search

valkey-io / valkey

A flexible distributed key-value datastore that supports both caching and beyond caching workloads.
https://valkey.io
Other
16.86k stars 626 forks source link

Reliable Keyspace Notifications #28

Open hpatro opened 6 months ago

hpatro commented 6 months ago

We have a requirement that we need to get a notification on changes to a Redis data structure. Based on my research I found out that I can use Redis key space notifications for doing the same. However, Redis key space notifications send the events to Redis pub/sub channel which is fire and forget i.e once the clients lose the connections all the events till the connection is up again are lost.

Redis streams solve this problem. Also, I want to use consumer group feature of Redis streams. So is there any way that Redis key space notifications can be pushed to Redis streams instead of Redis pub/sub channel?

One of the feature request which I was interested in solving in the past, so filing an issue to have more discussion on this Ref: https://github.com/redis/redis/issues/5766

madolson commented 6 months ago

We had a summary for this didn't we? The conclusion was basically there was no good solution without have a "per-shard hash".

hpatro commented 6 months ago

I think we should re-explore and look for solution as we have the opportunity to think through and introduce breaking change with 1.0.

CharlesChen888 commented 6 months ago

This reminds me of an old request trying to subcribe to a stream (redis/redis#10809), and a reversed request that intended to persist PUBSUB data to streams.