Closed errcheckenjoyer closed 1 month ago
Hello @errcheckenjoyer
I recently noticed this myself and opened an issue in rueidis - https://github.com/redis/rueidis/issues/586
Need to find some time and contribute
Looking forward to it. Thanks for the quick reply!
@errcheckenjoyer hello, Centrifuge master branch now pins to rueidis version without such panic.
Thanks @FZambia, will try this update during the week.
Can you also help me to understand what happens to messages sent into redis during such disconnect-reconnect process? It seems to me that they won't be delivered without additional actions from client (reconnect with recovery or history api call), even if they are stored in history stream. And it is impossible to notice this on client side.
Yes, messages will be lost, but Centrifuge has message recovery feature which allows to deal with it if necessary – read more here:
When recovery enabled, to notice on the client-side you have wasRecovering and recovered flags.
v0.33.0 uses rueidis without panic, so I think good to close.
Describe the bug It looks like a race between using a redis connection and releasing it.
To Reproduce It is difficult to reproduce because you need a redis connection error under a heavy load.
We have experienced crashes of our centrifugo instances (but very few of them) caused by
that followed problems with the redis connection (we found this in logs right before the panic):
Expected behavior There should be no panic or service crash.
Versions (please complete the following information):