search

redis / lettuce

Advanced Java Redis client for thread-safe sync, async, and reactive usage. Supports Cluster, Sentinel, Pipelining, and codecs.
https://lettuce.io
MIT License
5.3k stars 947 forks source link

When using the reactor client and performing Redis operations within the onErrorResume block, there is a chance of encountering thread deadlock. This can impact all threads at the entry points. #2905

Open project-snail opened 3 days ago

project-snail commented 3 days ago

Bug Report

When using the reactor client and performing Redis operations within the onErrorResume block, there is a chance of encountering thread deadlock. This can impact all threads at the entry points.

Reproduction steps are as follows:

  1. Start Redis normally
  2. Start the service
  3. ab -c 3 -n 100000 http://127.0.0.1:18080/lettuce/get (simulate normal traffic)
  4. Stop Redis (simulate Redis failure)
  5. Start any process on the Redis port (simulate Redis IP and port being accessible but Redis not yet providing service)
  6. Wait 30 seconds (wait for Lettuce to automatically reconnect to Redis, triggering the deadlock issue)
  7. After stopping ab, the HTTP service still does not work properly. And upon querying jstack, it is confirmed that a deadlock has occurred.

lockWritersExclusive

incrementWriters

Environment

Full demo code here

demo_code.zip

tishun commented 3 days ago

Quite possibly the same problem as the one in #2879

tishun commented 3 days ago

@project-snail thank you for the verbose bug report! I think we actually need to prioritize this fix for one of the coming versions