amqp-rs / lapin

AMQP client library in Rust, with a clean, futures based API
MIT License
1.05k stars 92 forks source link

Sharing `Channel` over async green threads #415

Open raui100 opened 3 months ago

raui100 commented 3 months ago

I have built a web-app using axum&tokio that creates a new green thread per request. I'm using lapin for RPC with RabbitMQ over a single channel which seems to work.
I've stumbled across the RabbitMQ documentation which discourages sharing Channels. Does this apply to lapin::Channel too and does this apply in an async context?

Is it recomment to share the connection and create a new lapin::Channel per incoming request?

Also it says in the documentation that closing a connection closes the underlying channels. Dropping a lapin::Connection doesn't close the underlying Connection which surprised me. Is it necessary to implement clean up code that closes all lapin::Connection and lapin::Channels at the end?

Keruspe commented 1 month ago

Hello, I'm so sorry, I was sure I answered you but I obviously haven't.

Basically, my point here will be: i'll always encourage you to follow the official documentation. It's there for a reason.

That being saidn I'm also well aware that this kind of specificities can be hard and/or counter intuitive. Especially when dealing with green threads and not OS threads.

Everything possible has been put into place in lapin to make sharing a Channel across threads (even OS threads) safe. Well, as safe as possible anyways. In high throughput context, with finely crafted interactions, I managed to actually make rabbitmq-server bug in several ways. It's actually possible to reproduce in a monothreaded env, but was easier with several threads. The issue got reported upstream but then I stopped being paid to work on it so I never got time to actually complete the debugging with upstream. A workaround got implemented in lapin to avoid hitting the faulty codepath in the server, forcing some ordering of frames even if not strictly necessary in theory according to the protocol specs.

WRT the connection and channel closing, you can see lapin::Connection and lapin::Channel like some kind of smart pointers, backed by std::sync::Arc. Basically, we'll automatically close them when the last ref is dropped (unless you already clsoed it before). Each Channel holds a ref to the Connection, so the Connection will get closed automatically only after all the Channels got closed themselves.

raui100 commented 1 month ago

Hi, thank you for your thorough answer :) To clarify:

  1. lapin::Channel and lapin::Connection can be shared in multiple threads, because they implement their own syncing mechanism.
  2. It is not necessary to clean up after them, because they implement their own RAII and clean up mechanisms.

This means the lapin::Channel and lapin::Connection have implementation details that makes them different from the Channel and Connection that are described in the RabbitMQ documentation.

I have another question regarding best practices. I have been using the connection pool from deadpool in combination with lapin. Whenever I need to send a new message I fetch a lapin::Connection from the pool, create a new lapin::Channel and use it to send a message. This seems to work really well. I had used long living lapin::Channels before and they sometimes timeout and loose their connection.

  1. Is it reasonable/idiomatic to create a lapin::Channel for each message I want to send? lapin::Connection::create_channel(..) seemed cheap to me. Is there a better way?
  2. Is it correct that lapin::Consumer have an heartbeat that keeps them alive forever even if they don't get any messages?
  3. Why do the examples of this crate often use multiplelapin::Channels created from the same lapin::Connection? Is this recommended? Is it bad to use a single lapin::Channel to create multiple lapin::Consumer and use it to send messages?