best practice

Automate Cost Savings with Cache Expiration Settings

Q: What prompt should I use?

// Example of creating a cache with a specific expiration const cache = await cacheManager.create({ model: "models/gemini-1.5-pro", contents: [{ role: "user", parts: [{ text: "[PASTE_LARGE_DATASET_HERE]" }] }], ttlSeconds: [INSERT_SECONDS_E.G._3600], });

Published 06:03 MYT · Jul 19

Use TTL (Time to Live) to manage your AI sessions.

Optimize your AI budget by setting specific expiration times (TTL) for cached content in Gemini.

Google Gemini

The Scenario

You are running a temporary workshop or a 2-hour analysis session where users query a specific dataset. You want the session to be fast for those two hours but ensure you aren't billed for storage afterward.

Before & after

The old way

Developers either manually deleted old context or let sessions expire randomly, leading to unpredictable billing. (Rough time cost: 10-20 minutes of manual monitoring).

With AI

Set a Time to Live (TTL) on your cache to automatically clear data after a specific duration, ensuring you only pay for what you need. (Rough time cost: 5 seconds to configure).

The Prompt

// Example of creating a cache with a specific expiration
const cache = await cacheManager.create({
  model: "models/gemini-1.5-pro",
  contents: [{ role: "user", parts: [{ text: "[PASTE_LARGE_DATASET_HERE]" }] }],
  ttlSeconds: [INSERT_SECONDS_E.G._3600], 
});

TTL manages the lifecycle of your cached context automatically, which is vital for cost control in production apps.

Source

Release notes | Gemini API | Google AI for Developers

"The typical solution is RAG... But for this use case I found a simpler, more efficient approach: Gemini’s Context Caching."