sensu / sensu-transport

The Sensu transport abstraction library.
MIT License
14 stars 19 forks source link

Sensu does not resume activities after reconnecting to RabbitMQ transport #65

Closed cwjohnston closed 5 years ago

cwjohnston commented 5 years ago

Expected Behavior

When encountering a transport connection error, sensu-client will attempt to reconnect. Once reconnected, sensu-client will resume publishing keepalive messages, consuming check execution requests, and publishing check results.

Current Behavior

When using Sensu 1.6.0 with the RabbitMQ transport, the client will reconnect but does not resume activities, e.g. publishing keepalives, consuming check requests, publishing check results.

Steps to Reproduce (for bugs)

  1. Install 1.6.0-1 from unstable channel and (re)start the client
  2. Use pkill -TRAP sensu-client or similar to toggle debug-level logging on
  3. Observe normal client operation
  4. Run rabbitmqctl stop_app on the RabbitMQ broker where the client is connected
  5. Observe "tcp connection closed" and associated transport reconnect messages
  6. Run rabbitmqctl start_app to restore the RabbitMQ broker
  7. Observe client log indicates "transport connection open" and "transport connection ready" but client does not log any further activities, e.g. consuming and publishing to the transport.

Context

This seems to be a regression related to changes in #62 and/or #64. Restarting sensu-client restores normal operation. Regression likely applies to other Sensu daemons.

Your Environment

amdprophet commented 5 years ago

Fixed by #66 and will be shipped in 1.6.1.