Serve stale data instead of blocking with `lock=True`

[mrmachine]

@cached_as(..., lock=True) can be used to avoid dog-pile effect, but this still results in slow queries as clients wait to acquire lock while cache is being populated (but at least results in lower resource utilisation).

I'd like an option to immediately serve stale data (if available) from the cache instead of blocking while the cache is being populated. This was mentioned in https://github.com/Suor/django-cacheops/pull/134#issuecomment-76676304 as the second point of avoiding dog-pile.

Can you give some advice on how to approach this implementation in django-cacheops?

[Suor]

Now cacheops immediately erases data on event, so you can't serve anything stale. Need to change this first.

And there could be lots of strategies with that. You can move it, mark it stale or add to invalidated list. This depends on how you want to work it later.

[mrmachine]

@Suor thanks for the feedback. So something like this:

• Add a setting CACHEOPS_INVALIDATED_EXPIRY = 300
• Add a kwarg stale=False to @cached_as etc. (everywhere lock is currently accepted) and also to CacheopsRedis._get_or_lock()
• On invalidation event, instead of deleting data move it to a new key (same key with "stale" prefix) with CACHEOPS_INVALIDATED_EXPIRY as the expiry so it won't linger forever
• Here https://github.com/Suor/django-cacheops/blob/cfc836c7dc80fb03ba5673f0cb43850e23fa9d8f/cacheops/redis.py#L41 pass through stale kwarg to _get_or_lock()
• After here https://github.com/Suor/django-cacheops/blob/cfc836c7dc80fb03ba5673f0cb43850e23fa9d8f/cacheops/redis.py#L61-L65 add a new elif stale: and get/return the stale data (if available), else continue the loop

If I'm reading correctly, this should:

• Allow the first miss to set the lock and return, allowing that process to continue repopulating the cache
• Allow subsequent misses (while lock is set) to return stale data within the expiry period
• Allow subsequent misses (after the expiry period) to block until the lock has cleared and cache is repopulated
• Redis will cleanup the expired stale cache data for us automatically (only if we configure Redis with --maxmemory FOO --maxmemory-policy volatile-FOO?)

[Suor]

This could work. Have you tried implementing it?

Also, volatile keys are expired and deleted even without any max-memory setting.

[mrmachine]

@Suor Thanks for the feedback on the approach. I have not had a chance to try and implement it yet. I have currently just backported dogpile mitigation (locking) to an old version of django-cacheops for use in a Django 1.6 project. That is working to reduce load but is not improving response times. Time is now just spent with most requests waiting for the locks to release instead of recreating cache data in parallel. Next I will try to implement the above, and hopefully with minor tweaks it can be adapted for the current version of django-cacheops too, assuming it works.

[Suor]

This should be easier to do in new insideout mode. That ones doesn't drop the cache key on invalidation but only removes conj stamps.

[Suor]

I will leave a hint in case anyone wants to champion this:

1. In INSIDEOUT mode it should be pretty easy to add a flag like allow_stale to @getset.getting() and ignore stamps if that is passed.
2. Add the same flag to @cached_as(), QuerySet.cache() and even CACHEOPS setting's dicts. Pass it down.