Build your stack, see the cost live
Pick each component — open-source self-hosted, hosted API, or managed. Set your traffic. Every choice updates the monthly total instantly. LLM prices are verified from Anthropic & OpenAI's live 2026 pages; other rates are editable defaults.
① Your traffic & tokens
② Embedding model
details →③ Vector database
details →④ Reranker
details →⑤ LLM that writes the answer ← 99% of your bill
details →Prices verified from official 2026 pricing pages.
⑥ Hosting (your VPS)
details →Quick presets
Total monthly cost
$0 per answer
Infra only
$0
LLM only
$0
Cost breakdown
Embeddings — turn text into vectors
Embeddings run on the question (~30 tokens) at query time, plus a one-time pass over your knowledge base. At your scale the per-query cost is tiny either way — the real choice is $0 self-hosted vs a small API bill for convenience and possibly higher accuracy.
| Option | Type | Cost | When to pick it |
|---|---|---|---|
| Open-source self-hosted (BGE, Arctic-Embed v2, nomic-v2) | OPEN | $0/query (uses VPS) | Rock-bottom budget; you control the model; accuracy-viable on a modest box. |
| OpenAI text-embedding-3-small | API | $0.02/1M tok (~$3.60/mo @ 6M) | You want zero ops and a proven model; cost is negligible. |
| OpenAI text-embedding-3-large | API | $0.13/1M tok | Slightly higher quality; still trivial cost at this token volume. |
| Cohere Embed v4 | API | ~$0.10/1M tok | If you already use Cohere; verify the live rate. |
⚠ Cross-model accuracy scores are vendor-self-reported — pick the final model with your own eval harness. Arctic-Embed v2 and nomic-v2 are strong open starting points.
Vector database — store & search the vectors
At under 100K chunks the index fits in RAM and answers in single-digit milliseconds on any option. So this is the least important cost decision — self-host and it's free.
| Option | Type | Cost | Notes |
|---|---|---|---|
| pgvector | OPEN | $0 | Best if you already run Postgres — one less service. Hybrid via Postgres full-text + vectors. |
| Qdrant (self-host) | OPEN | $0 | Cleaner native hybrid (dense + sparse) + built-in RRF fusion out of the box. |
| Qdrant Cloud | MANAGED | ~$25/mo | Managed ops + free tier for testing. |
| Supermemory | MANAGED | $19 + usage | Whole RAG pipeline managed for you — easiest, but a black box you can't tune. |
Reranker — the biggest accuracy lever
A reranker reorders the top candidates so the most relevant chunk lands in position 1–3. Verified to substantially cut retrieval failure. Self-host it — the API route is a hidden cost bomb at your volume.
Self-hosted cross-encoder
bge-reranker-v2-m3 on your VPS. $0/query. Keep top-k small (20→5) to stay under 2s.
Cohere Rerank API
~$2/1K searches = ~$12,000/mo at 6M queries. Easy to miss — try the calculator.
No reranker
Cheapest, but you lose the single biggest post-retrieval accuracy gain.
⚠ Specific vendor "lift %" and latency numbers were refuted in verification — the value of reranking is confirmed, the exact figures aren't. Measure your own.
The LLM — where your money actually goes
Everything else is ~$0. This is 99% of your bill, so this single choice is your cost strategy. Prices below are verified from Anthropic & OpenAI's official 2026 pages.
| Model | In /MTok | Out /MTok | @ 900K/mo | @ 6M/mo |
|---|---|---|---|---|
| GPT-5.4-Nano | $0.20 | $1.25 | ~$845 | ~$5,625 |
| Claude Haiku 4.5 | $1.00 | $5.00 | ~$3,825 | ~$25,500 |
| GPT-5.4-Mini | $0.75 | $4.50 | ~$3,105 | ~$20,700 |
| Claude Sonnet 4.6 | $3.00 | $15.00 | ~$11,475 | ~$76,500 |
Self-hosting an open LLM (Llama/Qwen) swaps per-token cost for a fixed GPU bill — cheaper than API once sustained volume is high enough. Toggle it in the calculator to find your break-even.
Hosting & caching
Your VPS
All self-hosted pieces (vector DB, embeddings, reranker) share one box. A $30–50/mo VPS comfortably holds <100K chunks in RAM and serves the retrieval stack. This is your entire "infra" line — well under $100/mo.
Semantic answer cache ★ biggest saver
Support/FAQ traffic repeats heavily. Cache answers by semantic similarity; a hit skips the LLM entirely. A 30–70% hit rate can roughly halve your largest cost. Move the slider in the calculator to see it.
Recommended build
For your profile (mixed business KB, <100K chunks, English, <2s, rock-bottom infra), this is the balanced pick. Hit "⚖️ Balanced" in the calculator to load it.