search

rabbitmq / rabbitmq-stream-rust-client

A client library for RabbitMQ streams
Other
152 stars 14 forks source link

Random infinite hangs with interacting with a RabbitMQ (Streams enabled) Docker container on macOS aarch64 #212

Closed nickgerace closed 7 months ago

nickgerace commented 7 months ago

Describe the bug

It appears to be random as to when it happens, but when running RabbitMQ with Streams on macOS aarch64 via Docker and using this library, there is sometimes an infinite hang. This is never seen on my Linux systems.

The container logs the following message:

Closing socket #Port<0.86>. Invalid transition from tuned to tuned.

I'm unsure if the client is causing the hang or the RabbitMQ instance is failing, but either way an infinite hang occurs and the client does not handle it.

Reproduction steps

  1. Use macOS aarch64 with Docker Desktop
  2. Run a RabbitMQ container with streams enabled (I use rabbitmq/3.12-management-alpine, publish port 5552:5552, enable the rabbitmq_stream plugin, and run with args RABBITMQ_SERVER_ADDITIONAL_ERL_ARGS=-rabbitmq_stream advertised_host localhost")
  3. Use the library's environment, publiush, and consume modules
  4. Observe infinite hang and logged error in container: Invalid transition from tuned to tuned.

Expected behavior

I expected the client to either not cause the hang or at least retain a responsive state and handle the hang/error from the RabbitMQ instance.

Additional context

No response

alfredotg commented 7 months ago

Hello. I think I found the problem.

Code below spawning task that send heartbeats: https://github.com/rabbitmq/rabbitmq-stream-rust-client/blob/7db8d4506dab0079680d7728b90765ae0d3806f1/src/client/mod.rs#L622

The problem is that the first heartbeat may be sent before the Tunes command. As far as I understand, this breaks the connection protocol and closes the RabbitMQ connection.

There are also several different problems that lead to infinity hangs. I will try to create PR with more information soon

alfredotg commented 7 months ago

PR #213 trying to fix two problems:

  1. Sending a Heartbeat before the Tunes command (and Open command) disrupts the connection protocol, leading RabbitMQ to terminate the connection.

  2. The client incorrectly manages the loss of connection. Requests continue to await a response indefinitely.

nickgerace commented 7 months ago

Thank you so much @alfredotg! Good catch :exploding_head: