marcoCasamento / Hangfire.Redis.StackExchange

HangFire Redis storage based on original (and now unsupported) Hangfire.Redis but using lovely StackExchange.Redis client
Other
462 stars 109 forks source link

Redis Cluster issue #42

Closed gzpbx closed 5 years ago

gzpbx commented 7 years ago

Do you encounter this error -

Error occurred during execution of 'BackgroundProcessingServer' process. Execution will be retried (attempt 2 of 2147483647) in 00:00:02 seconds. ==> StackExchange.Redis.RedisCommandException: Multi-key operations must involve a single slot; keys can use 'hash tags' to help this, i.e. '{/users/12345}/account' and '{/users/12345}/contacts' will always be in the same slot at StackExchange.Redis.ServerSelectionStrategy.Select(Message message) at StackExchange.Redis.ConnectionMultiplexer.TryPushMessageToBridge[T](Message message, ResultProcessor1 processor, ResultBox1 resultBox, ServerEndPoint& server) at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) at StackExchange.Redis.RedisTransaction.Execute(CommandFlags flags) at Hangfire.Redis.RedisConnection.AnnounceServer(String serverId, ServerContext context) at Hangfire.Server.BackgroundProcessingServer.Hangfire.Server.IBackgroundProcess.Execute(BackgroundProcessContext context) at Hangfire.Server.AutomaticRetryProcess.Execute(BackgroundProcessContext context)

gzpbx commented 7 years ago

I have a Pull Request to fix this issue.

marcoCasamento commented 7 years ago

Hi @gzpbx and thanks for the PR. It's a big one and I want to go through some integration tests first, especially regarding the "batch" involved methods. I'm also going to change the version you put and publish it as 1.7.0-beta as I'm trying to keep version somewhat aligned with the main product.

gzpbx commented 7 years ago

Hi @marcoCasamento , My fix in the PR has already worked on production. The solution for redis cluster is very easy:

sagart commented 7 years ago

Hi @marcoCasamento When is the next release scheduled ?

AndreSteenbergen commented 6 years ago

I am using a redis cluster as well. But as far as I can tell, the solution currently forces all keys to be in the same hash slot on redis. Am I correct? I only use redis for hangfire currently, but if all keys are in the same slot, a cluster would be overkill right? Maybe it would be better/ nicer if we could have a redis option to avoid multi key operations.

Redis Cluster supports multiple key operations as long as all the keys involved into a single command execution (or whole transaction, or Lua script execution) all belong to the same hash slot. The user can force multiple keys to be part of the same hash slot by using a concept called hash tags.

I guess all transactions are off the table when one would like to use sharding (based on the config), right?

marcoCasamento commented 5 years ago

@AndreSteenbergen absolutely right. I see no point in having sharding if all of the keys have to be in the same server. It can happens however that the redis instance is used for more than just Hangfire and in that case the {hangfire} prefix could be a valid compromise