31 July 2026
What a production RAG system actually costs to build
By Rasel Rana Rocky
A production RAG system typically takes eight to fourteen weeks to build and lands between $75,000 and $120,000 for mid-complexity work. Prototypes are much cheaper and much faster, which is exactly why so many teams are surprised by the gap.
The difference is not the retrieval code. It is everything required to trust it.
Why is the prototype so much cheaper than the production system?
Because the prototype does not have to be right. A demo that retrieves plausible chunks and produces a confident answer looks finished, and the work that separates it from a production system is invisible until you go looking: evaluation, ingestion reliability, and cost control under real traffic.
A prototype answers “can this work”. A production system answers “is this still working, right now, for every query we serve”.
Where does the time actually go?
Ingestion and data preparation, at 30–40% of the schedule — not where teams expect. In our experience the retrieval layer is rarely the long pole.
The breakdown below is drawn from our own delivered projects, not a published, externally measured dataset — nobody publishes one for RAG builds, ourselves included. Treat the shares as estimates, not measurements.
| Phase | Typical share | What dominates |
|---|---|---|
| Ingestion and data preparation | 30–40% | Source-system variety, document quality, permissions |
| Evaluation harness | 15–25% | Building a labelled set that reflects real queries |
| Retrieval and ranking | 15–20% | Chunking strategy, hybrid search, reranking |
| Application and integration | 15–20% | Auth, UI, streaming, error states |
| Cost and latency work | 10–15% | Caching, model routing, rate-limit handling |
Ingestion is consistently the largest and consistently the most underestimated. The documents are in six systems, three of them have no usable API, and a quarter of them are scanned PDFs.
What makes a RAG project expensive?
Four things, in roughly this order:
- Source data spread across many systems. Each new source is its own connector, its own auth, and its own failure mode.
- Permission-aware retrieval. If users may only see some documents, retrieval has to enforce that, and it has to do so without a per-query permission check that destroys latency.
- No existing evaluation set. Without labelled queries there is no way to know whether a change improved anything, so every change becomes an argument.
- Strict latency budgets. Sub-second end-to-end with reranking is achievable, but it constrains nearly every other decision.
Do you need a vector database?
Often not. If your corpus is under roughly a million chunks and you already run PostgreSQL, pgvector will very likely serve you, and it removes an entire system from your operational surface.
Dedicated vector databases start earning their cost at larger scale, with heavy filtered search, or when you need index types Postgres does not offer. Choosing one first, then discovering pgvector would have been sufficient, is a common and expensive ordering mistake.
What should you spend money on first?
The evaluation harness. It is the least exciting line item and the one that determines whether every subsequent decision is measured or guessed.
Teams that build evaluation first ship slower for three weeks and faster for the following six months. Teams that skip it spend those six months arguing about whether retrieval quality got better.