Harness

TL;DR

• The Problem: Coding agents easily accumulate 200K–300K tokens of tool outputs. Prompt caching keeps active turns cheap and fast, but provider caches expire after a short Time-To-Live (typically 5 minutes). Stepping away for coffee causes a cold-cache prefill on your next message—spiking latency and costing up to $10\times$ more.
• The Solution: TTL-Aware Idle Compression starts a background countdown after each turn (e.g., at ~4.5 minutes). Right before the cache expires, the harness appends a summarization prompt to the existing conversation—reusing the still-warm KV cache (~95% cache hit)—and shrinks the bloated history down to a lean ~19K-token context.
• Zero Disruption: It runs speculatively during human think-time. If you start typing while compression is mid-flight, the harness immediately aborts the background job and preserves your active flow with zero blocking latency.
• The Trade-off: Compaction is lossy and speculative. Summarizing drops verbatim details (potentially forcing file re-reads), and compacting an abandoned terminal tab burns output tokens on a session that never resumes—making its ROI sensitive to cache pricing cliffs and session resume rates.

The blog is co-authored with Gemini Carbon.