LangCache semantic cache calculator

See how much you can save by using semantic caching for LLM apps

Cache hit rate estimator

Query Source

Check cache hit rate with our sample queriesView sample query list

Upload My Own File

Cost Calculator

Cache Hit Rate (%)

Estimated percentage of cache hits

Daily Queries

Enter the number of queries per day

Input Tokens per Query

Average number of input tokens per query

Output Tokens per Query

Average number of output tokens per query

Cost assumptions

• LLM costs: $2.5 input/$10 output per 1M tokens (GPT-4o pricing) • LangCache service cost: $1.5 input/$0 output per 1M tokens (free in public preview) - illustrative pricing for demonstration, final GA pricing TBD • Storage: $100/month (may vary depending on size)

Cost details

Annual cost without LangCache

$3,741,250.00

Cost Calculation: • Annual Queries: 1,000,000/day × 365 = 365,000,000 • LLM Input Cost (GPT-4o): 365,000,000 × 100 × $2.5/1M = $91,250.00 • LLM Output Cost (GPT-4o): 365,000,000 × 1000 × $10/1M = $3,650,000.00 • Total Cost: $91,250.00 + $3,650,000.00 = $3,741,250.00

Annual cost with LangCache

$617,137.50

Cost Calculation: • LLM Cost: $3,741,250.00 × (1 - 85%) = $561,187.50 • Embedding Cost: 365,000,000 × 100 × $1.5/1M = $54,750.00 • Storage Cost (annual): $1,200.00 • Total Cost: $561,187.50 + $54,750.00 + $1,200.00 = $617,137.50

Annual savings

$3,124,112.50

Savings by cache hit rate