search

StackExchange / StackExchange.Redis

General purpose redis client
https://stackexchange.github.io/StackExchange.Redis/
Other
5.92k stars 1.51k forks source link

Need to implement WAIT , tips ? #1474

Open bklooste opened 4 years ago

bklooste commented 4 years ago

Have read the other thread but i need this anyway (not immediately but soon), don't need other blocking operations just WAIT

My options are

Just after some tips / suggestions of the best approach.

mgravell commented 4 years ago

We know that WAIT is a problem. We have investigated some alternative internal implementations that will enable WAIT and other blocking operations - essentially, switching to a connection pool internally. This also resolves a number of other outstanding problems. The prototype was very successful - but we have not yet merged this work, because: it takes time, and this isn't our day jobs. I have a few other plates that I'm spinning at the moment, but this is very high on my list.

Blocking operations are anathema to the current implementation; there are no supported workarounds; you can hack them in via Execute etc, but: this is not advisory, and no support will be offered to make it work, because it cannot work reliably.

My advice: wait

bklooste commented 4 years ago

Thank you much appreciated. I presume this will be a major release with a long cycle ?

mgravell commented 4 years ago

No, I was going to do this as 2.2, and the proof-of-concept is complete - it just needs integrating and testing, but: this isn't my only project and isn't my day job, so: it is tied on my availability. It is something I really want to get in, because it simplifies the IO path hugely, which should resolve a ton a network pain points.

MikaelElkiaer commented 3 years ago

Does this mean that there are currently no means of ensuring replication?

I am planning to set up master-replicant with sentinels. I suspect I have some data that I need to ensure is the same no matter which node you reach. It seems this is only possible with WAIT.

Is there anything going against an approach using a lua script to perform synchronous operations - i. e. using WAIT.

mgravell commented 3 years ago

That Lua script will block your entire server, and could even get forcibly terminated. I don't recommend that.

On Thu, 19 Aug 2021, 19:19 Mikael Elkiær, @.***> wrote:

Does this mean that there are currently no means of ensuring replication?

I am planning to set up master-replicant with sentinels. I suspect I have some data that I need to ensure is the same no matter which node you reach. It seems this is only possible with WAIT.

Is there anything going against an approach using a lua script to perform synchronous operations - i. e. using WAIT.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/StackExchange/StackExchange.Redis/issues/1474#issuecomment-902137843, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAEHMGXDM6NWWPNRQEKFTLT5VDKJANCNFSM4NLQYN3A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

MikaelElkiaer commented 3 years ago

That Lua script will block your entire server, and could even get forcibly terminated. I don't recommend that.

Ok, if that is more or less ruled out, then what else is there left to do? Even when sticking with DemandMaster, a failover will not be consistent either. Is there then only going full cluster and manually write all masters, while keeping DemandMaster?

Is this a problem of client and/or server blocking? I'd assume that server blocking always would be possible despite any limitations in the client - what am I missing here?

Edit: I see now that WAIT only knows which writes to confirm based on the current connection, and I guess that in a multiplexer scenario it would be hard to discern which writes to confirm, unless it should be all of them. However, in the context of a transaction (i. e. MULTI), wouldn't the WAIT command only confirm the writes within that transaction?

MikaelElkiaer commented 3 years ago

@mgravell Sorry for bumping, but I was wondering if you could come with some clarification regarding the last bit of my previous post. Wouldn't it be possible to do WAIT within the context of a MULTI - even in a multiplexer scenario?

mgravell commented 3 years ago

Honestly, until I get the time to implement pooled connections: I am not, in any scenario, going to advocate anything that introduces a blockage at the server, no matter how transient. I can see many ways it go go badly. Are there scenarios where it might work acceptably under modest load? Possibly, but I'm not going to try and think through what the limits are.

MikaelElkiaer commented 3 years ago

Honestly, until I get the time to implement pooled connections: I am not, in any scenario, going to advocate anything that introduces a blockage at the server, no matter how transient. I can see many ways it go go badly. Are there scenarios where it might work acceptably under modest load? Possibly, but I'm not going to try and think through what the limits are.

Fair enough. Thanks for the reply, will keep an eye out for connection pooling.