search

alanxz / rabbitmq-c

RabbitMQ C client
MIT License
1.77k stars 672 forks source link

Configurable timeout or non-blocking version for amqp_basic_publish() #519

Open micoq opened 6 years ago

micoq commented 6 years ago

Currently amqp_basic_publish() will always block indefinitely if RabbitMQ blocks the connection at the TCP level (in case of a memory alarm).

It could be a good idea to let the user to choose whether or not to block (ar add a timeout) when publishing a message so you can manage multiple producers with a single thread.

A memory alarm can be easily triggered by reducing the memory watermark on RabbitMQ: 

rabbitmqctl set_vm_memory_high_watermark absolute 10MiB
nh2d commented 5 years ago

What version of rabbitmq-c are you using? v0.8.0 What operating system and version are you running on? ubuntu 16.04 What compiler and version are you running? glibc which comes with ubuntu 16.04 (xenial xerus)

We run a Django application which publishes many celery tasks to RabbitMQ using librabbitmq v2.0.0. We use celery version 4.1.1. We recently experienced a RabbitMQ outage (due to some unrelated issues) which caused all of our task publishing to timeout. This resulted in a nearly full outage of our webapp, as many of our views synchronously publish celery tasks and hung trying to publish those tasks during the RabbitMQ outage.

We can mitigate this availability risk if we were able to configure a publishing timeout, basically having a maximum amount of time that client code will attempt to publish a message to RabbitMQ before failing fast. This would prevent the cascading failure scenario we experienced and provide greater flexibility for clients to control the publishing behavior.

Is this something we can achieve with librabbitmq currently? If not, does this seem like a reasonable feature request? Thank you for your consideration of this issue!