How It Works
Creating a Knowledge Base
Adding Documents
- Chunking — splits into overlapping segments with context preservation
- ML enrichment — optional table and layout extraction
- Embedding — generates vector representations
- Storage — persists in PostgreSQL with PGVector
Searching
Search Pipeline
| Stage | Description |
|---|---|
| Query expansion | Rewrites the query into multiple variants |
| Semantic search | Cosine similarity against embedded chunks |
| Hybrid search | Combines semantic with BM25 keyword matching |
| Reranking | LLM or cross-encoder reorders results by relevance |
Agent Integration
When an agent hasenable_retrieval=True and is linked to a knowledge base, retrieval happens automatically:
- The user sends a message
- The agent searches its knowledge bases
- Top results are injected into the prompt context
- The agent responds using the retrieved information