Distributed Caching Strategies

The Cache-Aside Pattern

The most common distributed caching pattern. The application code first asks the cache (e.g., Redis). If the data is missing (Cache Miss), the application queries the database, writes the result to the cache, and then returns it. Subsequent requests hit the cache (Cache Hit).

The Thundering Herd Problem

When a popular, computationally expensive key expires in the cache, hundreds of concurrent requests might simultaneously experience a Cache Miss. All of them will hit the database at the exact same time, potentially taking the database offline.

Mitigation: Mutex Locks

Implement a distributed lock in Redis. When a cache miss occurs, the application attempts to acquire a lock for that specific key. If successful, it queries the database and updates the cache. If it fails to acquire the lock, it means another thread is already fetching the data, so it sleeps for 50ms and retries the cache.

// Conceptual pseudo-code
val = redis.get(key)
if (val == null) {
  if (redis.acquireLock(key + "_lock", 5s)) {
    val = db.query()
    redis.set(key, val, TTL)
    redis.releaseLock(key + "_lock")
  } else {
    sleep(50ms)
    return fetch(key) // Retry
  }
}
return val