streadway / amqp

Go client for AMQP 0.9.1
http://godoc.org/github.com/streadway/amqp
BSD 2-Clause "Simplified" License
4.88k stars 621 forks source link

RabbitMQ has open connections(Too many File descriptors piling up on broker) for publish #527

Open raj-githubacc opened 2 years ago

raj-githubacc commented 2 years ago

Hi, We're using amqp library to publish messages to a Queue, We've encountered 2 issues here

  1. Same connection cannot be reused if we try to publish via goroutines. An error 504 connection/channel closed is returned.
  2. As a hotfix, we're creating new connections for each publish, but here somehow the connection destructor is not working and RabbitMQ node still has the connection channel opened. As a result too many file descriptors are being piled and node goes down.

Would appreciate any help/pointers

ingfdoaguirre commented 2 years ago

Hi, you can use this library https://github.com/houseofcat/turbocookedrabbit, I had the same problem as yours.

There you can create a "pool" of connections, so in your case, also mine, if there are no connections to handle your request, this library will create a new one and will close it if will never used again.

You can declare the max number of connections and channels to start in the pool.

I had an endpoint that ,in some cases, receives like 1k of requests per second, with vanilla library I got the same error, and is because the connection get saturated. With the library turbocookedrabbit I had no more problems.

ps: Internally uses this library to handle the basic logic.