Home/Answers/Cached tokens
AI COST LAB / Cached tokens

Cached tokens: Local vs Online Tools

Local Cached tokens tools are more private and work offline; online tools save installation but files leave your device.

Updated 2026-11-01About 3 minLocal test
Quick answer

Local Cached tokens tools are more private and work offline; online tools save installation but files leave your device. Prompt caching can cut input cost, but hit rate depends on repeated context and cache policy.

Why it matters

Use local for sensitive files; online is fine for small temporary ones.

Only hits are cheaper.

Step by step

  1. Measure input, output and retry tokens per user/task.
  2. Separate cache, batch, embeddings and fine-tuning costs.
  3. Use 30 days of real requests for cost per user, then margin.
  4. Model prices change; baseline from official pricing pages and record the date.

Common mistakes

  • Estimating from input tokens only; output often costs more.
  • Forgetting retries, failures and tool calls.
  • Assuming cache hits by default.
  • Using stale prices for long-term margin.

Checklist

  • Input/output/cache/retry tokens are tracked separately.
  • Batch/async fit and latency requirements are clear.
  • Embeddings/fine-tuning/image/audio are separate.
  • Cost per user and margin use real usage.
  • Price date and fallback model are documented.

Key facts

  • Only hits are cheaper.
  • Caches have TTL.
  • Not all models support.

FAQ

Local or online tools for Cached tokens?

Local Cached tokens tools are more private and work offline; online tools save installation but files leave your device. Prompt caching can cut input cost, but hit rate depends on repeated context and cache policy.

How long does it take?

Usually 1-3 minutes; for intermittent issues, run 30 seconds and retest elsewhere.

Is data uploaded?

Runs locally and does not upload video/audio; extensions may still read the page.

Source: OpenAI