search

FoundatioFx / Foundatio.Redis

Foundatio Redis
Apache License 2.0
98 stars 31 forks source link

Random "Error getting queue payload" in RedisQueue under load #80

Closed Seadoo2017 closed 10 months ago

Seadoo2017 commented 11 months ago

After recent package update we started getting this error message, we tracked the issue down to a change done in RedisCacheClient. All the read operations now use CommandFlags.PreferReplica (changed in this commit: https://github.com/FoundatioFx/Foundatio.Redis/commit/8ea1bf1455761100a5186038ab61e554bc8c87ae)

Sometimes, the replica might not have the data when a read operation is performed right after write operation. So this cache change breaks RedisQueue randomly.

My suggestion is to revert the change or allow injecting the required behavior.

ejsmith commented 11 months ago

Would you mind creating a PR that adds a read mode to the redis options? So we can set it to PreferReplica or PreferMaster.

Seadoo2017 commented 11 months ago

Hi, added pull request (https://github.com/FoundatioFx/Foundatio.Redis/pull/81) I added read mode to options + reverted the default behavior to none as the new behavior is breaking existing code. Due to the nature of this issue, I didn't think of any good way to add unit test to test this change / demonstrate the breaking change in unit test, any ideas?

I gave access in the option to the original CommandFlags with all the available options to have full flexibility, do you prefer a new smaller enum / boolean option?

Thank you.

Seadoo2017 commented 10 months ago

@ejsmith I apologize but I had to open a new PR (https://github.com/FoundatioFx/Foundatio.Redis/pull/82) because I messed up the author at the first one so it didn't let me sign the CLA, this one is the same with the correct author.