RAG & Evaluation for AI Features

RAG là gì

Retrieval-Augmented Generation — kết hợp retrieval (search trong knowledge base) với generation (LLM). Giải quyết 2 limit chính của LLM: hallucination và stale knowledge.

RAG pipeline

1. Ingestion — doc → chunk → embed → vector DB.
2. Retrieval — user query → embed → top-K similar chunks.
3. Re-ranking — refine top-K bằng cross-encoder.
4. Generation — LLM với context = chunks + user query.
5. Citation — link back to source.

Design decisions PM phải làm

• Chunk size — 256/512/1024 token. Trade-off: precision vs context.
• Chunk overlap — 10–20% để không cut mid-sentence.
• Embedding model — OpenAI text-embedding-3, Cohere, multilingual model cho VN.
• Vector DB — Pinecone, Weaviate, pgvector. Trade-off: cost, latency, scale.
• Hybrid search — combine semantic + keyword (BM25).

Evaluation framework

Retrieval metrics

• Recall@K — % relevant doc trong top-K.
• MRR — Mean Reciprocal Rank.
• NDCG — Normalized Discounted Cumulative Gain.

Generation metrics

• Faithfulness — output có ground trong retrieved chunks?
• Relevance — output có trả lời query?
• Answer correctness — vs ground truth.
• Citation quality — source link chính xác?

Eval tools

• RAGAS — framework Python open-source.
• LangSmith, Phoenix — trace + eval.
• LLM-as-judge — use GPT-4 score output.

Common pitfalls

• Test trên 10 example → claim "RAG works" → fail production scale.
• Chunk theo character count thay vì semantic boundary.
• Không track retrieval và generation độc lập — debug bị mù.
• Skip re-ranking → top-K nhiễu.