search

dashbitco / broadway_rabbitmq

A Broadway producer for RabbitMQ
193 stars 42 forks source link

Ability to retrieve the AMQP Channel used by the Producer #107

Closed basilenouvellet closed 2 years ago

basilenouvellet commented 3 years ago

First of all, thank you for this amazing library!

What do you think about adding the ability to retrieve the AMQP Channel used by the Producer?

I'm aware that the Channel is already available in the message metadata, but I'm talking here about having a dedicated method to query the Channel (from the Producer state) outside of a message consumption.

I see several use-cases for this:

  1. Publish RabbitMQ events: Broadway would be responsible for managing the AMQP Channel (connection, re-connection in case of failure, proper disconnection in case the process crashes, etc). This would save a lot of boilerplate code, avoiding the management of a dedicated Channel just to publish events.

  2. Send RabbitMQ RPC requests: building a RPC Client sending RPC requests with the Channel managed by Broadway, using Broadway as a reply-queue consumer for handling RPC responses. Kind of the same use-case as 1., but with the added complexity that we need to keep a mapping between the caller process PID and the RabbitMQ correlation ID, in order to send the response back to the caller process when the RPC response is consumed.

For now, I have circumvented this by having a dedicated process RequestHandler storing the Channel. I send the Channel to RequestHandler using the Broadway after_connect option. But this feels quite hacky to be honest.

Please tell me if you think this would be outside the scope of Broadway, as one could argue it is supposed to be essentially a consumer of messages, not a producer (publish events & RPC commands).

whatyouhide commented 3 years ago

Hey @BasileNouvellet! Thanks for the issue.

  1. I don't see how this is related to Broadway. If you want to publish messages on RabbitMQ outside of the scope of consuming them in a Broadway pipeline, then Broadway is not the right tool to use in my opinion. You would essentially just be using the Broadway producer as a "AMQP channel manager" as far as I understand. In that case, extracting the desired code out of Broadway and into your system or a library is what I would probably do.

  2. Here I'm not sure, what is the problem with the channel being stored in the message metadata?

To be honest, yes, I'm leaning on what you say here:

one could argue it is supposed to be essentially a consumer of messages, not a producer (publish events & RPC commands).

😄 I also don't think what you're doing with the RequestHandler process is hacky. after_connect is documented and public API, and sending messages between processes is nothing out of the ordinary. Are you seeing any problems with this setup?

basilenouvellet commented 3 years ago
  1. A RPC Client would need to publish a RPC message with the Channel first, then consume the response from the reply-queue. In my understanding, the Channel in the message metadata only allows building a RPC Server: consuming the request from the queue first, then publishing the response in the reply-queue.

My implementation of RequestHandler is still early, not in production yet, thus so far I don't have any problems with this setup, you're right!

I was asking the question here because a simple get_channel public API on Broadway RabbitMQ would enable production of messages (either as a RPC Client or Events publisher). But I totally get your point about this not being related to the goal of Broadway.

Thank you a lot for your time and your feedback!

whatyouhide commented 2 years ago

@BasileNouvellet closing this as is seems like we found an avenue to explore for now 😉