Production knowledge system
Building a citation-first research system for a knowledge domain where context, source quality and interpretation matter.
The system
Ask HipHop combines domain content, long-form media and structured data behind a retrieval system designed to preserve context and provenance. The backend coordinates ingestion, retrieval planning, vector search, reranking and grounded generation instead of treating RAG as a single similarity-search step.
Request path
User intent enters the system
Route the request before retrieval
Choose knowledge bases and constraints
Vector search plus metadata filters
Improve relevance before generation
Compose a grounded answer
Return source-aware output
Engineering decisions
A domain system benefits from knowing what kind of question it is answering before searching. Classification gives retrieval a narrower job and makes metadata constraints explicit instead of hoping similarity search discovers the right context on its own.
Documents, long-form video and structured entity data arrive through different pipelines. Asynchronous workers keep ingestion, parsing, embedding and indexing away from interactive request latency while making retries and failure states observable.
Retrieval is not useful enough if the product cannot explain where an answer came from. Source metadata survives the pipeline so the generated response can remain tied to inspectable evidence.
The system evolved beyond a single vector collection. Retrieval planning and metadata filters allow questions to be evaluated against the appropriate project or knowledge base rather than searching everything indiscriminately.
Backend
FastAPI · PostgreSQL · Redis · asynchronous workers
AI / Retrieval
Bedrock · embeddings · vector retrieval · reranking · metadata filters
Product
React · citation-aware chat · public entity experiences