dashbitco / broadway_rabbitmq

A Broadway producer for RabbitMQ
193 stars 42 forks source link

Documentation for setting up dead-letter exchange is inaccurate and/or incomplete #133

Open fireproofsocks opened 1 month ago

fireproofsocks commented 1 month ago

If the example from https://hexdocs.pm/broadway_rabbitmq/BroadwayRabbitMQ.Producer.html#module-dead-letter-exchanges is run, it generates errors:

[error] GenServer Rarebit.Pipelines.Inspection.Broadway.Producer_0 terminating
** (stop) exited in: :gen_server.call(#PID<0.492.0>, {:subscribe, {:"basic.consume", 0, "my_queue.dlx", "", false, false, false, false, []}, #PID<0.407.0>}, 70000)
    ** (EXIT) shutdown: {:server_initiated_close, 404, "NOT_FOUND - no queue 'my_queue.dlx' in vhost '/'"}

This appears to be because the the my_queue.dlx is not yet created yet -- this makes sense because the code that creates that queue is in the :after_connect option. The errors appear only on the 1st start; subsequent restarts do not seem to have this issue.

Secondly, the code in the handle_message/3 is incomplete -- it should demonstrate a full working example, e.g.

  def handle_message(_processor, message, _context) do
    Broadway.Message.failed(message, "Error handling message; this should trigger a send to the dead letter queue")
  end
josevalim commented 1 month ago

Thank you, can you please send a pull request?

fireproofsocks commented 1 month ago

Yessir! I'm finding I am not a fan of the :declare, :bindings, or :after_connect options because it's difficult to express more complicated setups using those options (e.g. like having pre-requisites for a dead-letter queue). Secondly, I find that the :bindings in particular tend to be mis-interpreted because people think that Broadway somehow cares about the them. But really Broadway is only bound to the queue. So I might adjust the setup in the example a little bit and try to offer some clarifications in my PR.

I'm working on a book for Packt on concurrency with Elixir, so I've been digging into RabbitMQ and Broadway for one of the chapters. We'd love your feedback!

fireproofsocks commented 1 month ago

Here's the PR https://github.com/dashbitco/broadway_rabbitmq/pull/134