Redis backend with CAS integration

[Messinger]

Hi

I wrote for a project using your cache a backend using Redis: https://github.com/Messinger/redis-activesupport-with-cas (based on redis-store)

I read somewhere that you want informed when someone did it.

My questions:

• may you give a hint in your readme on it? I think some developers want use redis instead of memcache, too
• is it possible that you may do a review? I had written it based on your cas methods, due missing description (is there anyone?) I implemented it along your'e tests for CAS. So I'm not sure if I miss something.
• do you have some hints, how to integrate it into your test-system? I would like that to be sure that it works like you expected.

Hope someone will find this gem usefull, too.

[dylanahsmith]

That's great! I think it would make sense to update the README to link to different backends and stop making it sounds like the gem only works with identity cache.

Sorry about the lack of documentation on the methods that the backend requires. I don't even see documentation in the memcached_store gem for it.

Basically the cas method yields the value at the cache key if it is present and sets the value with the result of the yield if it hasn't changed from the value yielded to the given block. cas doesn't yield or change the value if the cache key isn't present.

cas_multi differs in that it takes an array of keys, yields a hash of key-value pairs for the found cache keys and expects the given block to return a hash of key-value pairs to update if those keys hadn't been changed from the initial hash of key-value pairs that were yielded.

Ideally I would like to push those methods upstream to ActiveSupport::Cache::MemCacheStore so we don't depend on non-standard extensions to the rails cache store API. In the meantime, documentation for these methods would certainly be welcome.

Is redis-store meant to be used as a rails compatible cache store? If so, then I don't see why there would be a need for redis-activesupport-with-cas, since redis-store-with-cas should provide the desired interface for using with IdentityCache directly. Otherwise, it seems like redis-activesupport-with-cas would be providing a weird hybrid where cas and cas_multi might behave the right way but the delete and write methods wouldn't behave the same way as a rails cache store. E.g. if normalize_key(name, options) in redis-activesupport-with-cas is done in just cas and cas_multi then that normalize could cause cas to operate on a different key than write or delete and always be treated as a cache miss for modified keys.

If we want to test it as part of the identity cache backend, then we would probably want a way to swap out the memcached_store backend with other stores in a similar way to how we allow the test suite to be run with mysql or pg as a database.

[Messinger]

TL;DR: redis-store-with-cas is a patch for redis-store, while redis-activesupport-with-cas is an extension to redis-activesupport which of course uses normalize-keys etc ;) eg is a full implemented activesupport-cache

Well, redis-store is more an enhancement of low-level redis client for ruby independend of rails. I wrote redis-store-with-cas as a monkey patch for it 'cause there is a bug inside redis-store which makes problems with CAS when using the namespace feature. I hope one day this may complete merged into redis-store.

redis-activesupport-with-cas is re-using the activesupport-part of the redis-store-system (all features are separate gems). Due the described bug you had to integrate low-level functions into activesupport when not reusing redis-store-with-cas

In my project we use the CAS-workflow on a very raw level AND for activesupport eg rails. Thats the next reason why both.

so: redis-store is meant as extension to ruby itself, redis-activesupport is the riddle between rails-activesupport and redis-store. Same for the -with-cas variants. All are bundled for rails-projects due the redis-rails gem.

Yes, its a little bit confusing for the first time, but after a while it makes sense splitting all parts into separate gems.

[Messinger]

forgot: The whole redis projects with all assigned gems

[danielnc]

bump, any updates for rails 6?

[rafaelfranca]

@danielnc what do you mean for Rails 6? This gem already support Rails 6.

[danielnc]

@rafaelfranca question was related to a redis cache store + rails 6

[rafaelfranca]

Ah, the Redis cache store on Rails 6 doesn't support CAS, so it can't work with this gem. So far the only supported store is the memcache store.

[danielnc]

@rafaelfranca thanks, if there are any plans to support redis, let us know

best,

[haadfida]

@rafaelfranca Any updates on redis cache store and this gem for Rails 7? Is there any plans to add support for Redis? Thanks.

[dylanahsmith]

The more direct equivalent to CAS support in redis is the WATCH command (which is what redis-store-with-cas uses), except that isn't ideal for fetch_multi support, since it would cause an entire batch to be invalidated by a conflict in a single key.

Instead, it probably makes sense to version the cache invalidation value (will be needed for https://github.com/Shopify/identity_cache/issues/535 anyways) and to use a set_if_equal lua script to set the value if it is still equal to the given value. That will be functionally equivalent and would allow it to be efficiently pipelined. The single key transactions should even work with a distributed redis servers, since each transaction would only affect a single key.

We could have an extension for ActiveSupport::Cache::RedisCacheStore in a similar way as we do with IdentityCache::MemCacheStoreCAS extending ActiveSupport::Cache::MemCacheStore. Although, in this case it looks like there is a public redis method, so perhaps this can be extended in a cleaner way this time.

It will also make sense to take advantage of redis pipelining / multi commands for batching writes, such as cache invalidations, which is even something that the standard cache interface supports with write_multi. This should also make it easier to take advantage of in a memcached backend when there is a support for meta commands.