RAG Practice：Build enterprise knowledge base

The biggest problem with LLMs is「hallucination」——it will confidently fabricate answers。RAG（Retrieval-Augmented Generation）by having AI first retrieve relevant documents, then answer，perfectly solves this problem。

What is RAG？

RAG = Retrieval Augmented Generation，Retrieval-Augmented Generation。Core concept：

1. Split enterprise documents into small chunks
2. use Embedding Model converts text to vectors
3. Store in vector database
4. When user asks a question，First retrieve relevant documents
5. documents + send questions toLLM

Tech stack

Complete code

1. Install dependencies

pip install langchain langchain-openai chromadb tiktoken

2. Load documents

from langchain_community.document_loaders import (

DirectoryLoader,

TextLoader,

PyPDFLoader

)

from langchain.text_splitter import RecursiveCharacterTextSplitter

Load docs all files in the directory

loader = DirectoryLoader("./docs", glob="*/.md")

documents = loader.load()

Split document

splitter = RecursiveCharacterTextSplitter(

chunk_size=500,

chunk_overlap=50,

separators=["\n\n", "\n", "。", "！", "？"]

)

chunks = splitter.split_documents(documents)

print(f"Split into {len(chunks)} document chunks")

3. Create vector database

from langchain_openai import OpenAIEmbeddings

from langchain_community.vectorstores import Chroma

embeddings = OpenAIEmbeddings(

base_url="https://www.ciyuano.com/v1",

api_key="sk-relay-Your API key",

model="text-embedding-3-small"

)

vectorstore = Chroma.from_documents(

chunks,

embeddings,

persist_directory="./chroma_db"

)

print("Vector database created！")

4. Build Q&A chain

from langchain.chains import RetrievalQA

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(

model="deepseek-v4-flash",

base_url="https://www.ciyuano.com/v1",

api_key="sk-relay-Your API key",

temperature=0

)

qa_chain = RetrievalQA.from_chain_type(

llm=llm,

retriever=vectorstore.as_retriever(

search_kwargs={"k": 3}

),

return_source_documents=True

)

Query

result = qa_chain.invoke({"query": "What is the company's leave policy？"})

print("Answer：", result["result"])

print("Source：", [doc.metadata["source"] for doc in result["source_documents"]])

Optimization tips

1. Splitting strategy

Choose splitting method based on document type：

• Markdown：Split by heading
• Code：By function/class-based splitting
• PDF：Split by paragraph

2. Retrieval optimization

• Use hybrid retrieval（vector + keywords）
• Adjust top_k count
• Use reranking model

3. Prompt Optimize

in Prompt explicitly requires：

• Answer only based on provided documents
• If no relevant information in documents，explicitly state「I'm not sure」
• Cite document sources

Summary

RAG is an enterprise AI best practices for applications。through LangChain + vector database，you can quickly build an accurate、reliable enterprise knowledge base system。