There’s a famous quote in computer science:
“There are only two hard things in Computer Science: naming things and cache invalidation.”
It’s funny because it’s painfully accurate. I ran straight into this problem while building an IAM system using Attribute-Based Access Control (ABAC). The policies endpoint became the most frequently called part of the system, and caching it felt both obvious and terrifying.
Here’s what went wrong, what we learned, and how a pragmatic caching strategy removed our biggest bottleneck without compromising safety.
Why cache invalidation is genuinely hard
Caching is easy. Invalidating correctly is not.
The moment you cache data, you’re making a deal with the devil: you get speed, but you now own correctness over time.
It’s hard because:
- Caches are copies of truth, and truth changes.
- Distributed systems don’t change state simultaneously across regions.
- You must choose between freshness and latency.
- Invalidation rules quickly become complex and brittle.
- One wrong decision can leak data, deny access, or break trust.
This is why many teams quietly avoid caching altogether, especially in security-sensitive systems like IAM.
Why teams skip caching (even when they shouldn’t)
Most engineers don’t skip caching because they’re lazy. They skip it because:
- It’s hard to reason about when to invalidate.
- Testing stale edge cases is painful.
- Debugging cache issues feels like chasing ghosts.
- Nobody wants to ship a “fast but wrong” system.
Unfortunately, by the time caching becomes unavoidable, the endpoint is already on fire.
The system I was working on
The setup looked like this:
- An IAM system using ABAC
- Used by multiple teams, apps, and regions
- Policies evaluated via a policies endpoint using an in-house policy engine
- Policies could be updated at runtime
- User attributes could be configured by users themselves
That last point made caching especially tricky. Policy decisions depended on attributes that were mutable and user-controlled.
And yet…
The policies endpoint was absolutely hammered.
The meme that inspired our solution
As we were struggling to figure out how to perfectly invalidate our policy caches, I saw this meme on reddit:
What if correctness didn’t have to be absolute to be safe?
Policy caching is often framed as a strict choice between speed and accuracy. In practice, the real decision is whether we can tolerate a bounded window of staleness in exchange for dramatically lower latency.
In our case, being fast and correct 99.999% of the time was not only acceptable… and it was optimal.
The approach we took: fast first, correctness guaranteed
We implemented a cache-first strategy with background refresh, combined with time-based expiration.
The flow looked like this:
- Always read from cache first.
- Serve the cached result immediately.
- Refresh the cache in the background.
- Apply a TTL so stale data can’t live forever.
- Accept that the first request after a change might be stale… briefly.
P.S. While writing this post, I got to know that this pattern is commonly known as stale-while-revalidate.
Why this worked for us
- The policies endpoint always hit the cache.
- Latency dropped dramatically.
- Cache freshness improved automatically over time.
- Staleness was bounded and predictable.
- Safety was preserved via TTLs and fallback logic.
The downside (and we accepted it knowingly)
We still accessed the database during refresh, meaning:
- Slightly higher DB load
- More moving parts
But the bottleneck was gone and that was the real win.
The real lesson learnt
Cache invalidation is hard because it forces you to confront reality: distributed systems are messy, and perfection is expensive.
But the goal isn’t perfection.
The goal is bounded incorrectness with predictable behavior. When done right, caching turns your slowest endpoint into your fastest one.
Or, to put it another way:
Fast and safe beats slow and perfect every time. 🚀
