search

sensu / sensu-transport

The Sensu transport abstraction library.
MIT License
14 stars 19 forks source link

Investigate lack of retry/back-off logic in DNS requests #60

Closed cwjohnston closed 5 years ago

cwjohnston commented 6 years ago

Per https://github.com/sensu/sensu/issues/1889 customers are seeing what amounts to a denial of service attack when Sensu services fail to resolve a DNS host name for their transport connection.

The issue linked above relates to Redis transport. We need to determine the behavior described there also applies to RabbitMQ transport, and, if so, implement some logic to make retries less aggressive.

portertech commented 5 years ago

Hostname resolution is done via https://github.com/sensu/sensu-transport/blob/c36d934a358d92df4597ac654b25d7df173042b8/lib/sensu/transport/base.rb#L152-L197

portertech commented 5 years ago

RabbitMQ periodic reconnect attempt logic uses a capped retry timer https://github.com/sensu/sensu-transport/blob/c36d934a358d92df4597ac654b25d7df173042b8/lib/sensu/transport/rabbitmq.rb#L283-L301

cwjohnston commented 5 years ago

Closing this as it appears RabbitMQ already follows the behavior we expect.