golevelup / nestjs

A collection of badass modules and utilities to help you level up your NestJS applications 🚀
MIT License
2.28k stars 263 forks source link

Multiple RabbitSubscribe in one class, one looses connection #526

Open wodka opened 1 year ago

wodka commented 1 year ago

We have a class similar to this:

class Message {
  @RabbitSubscribe({
    exchange: 'message',
    queue: 'message-a-queue',
    queueOptions: {
      durable: true,
      maxPriority: 10,
    },
    routingKey: 'message-a',
  })
  async handleA() {}

  @RabbitSubscribe({
    exchange: 'message',
    queue: 'message-b-queue',
    queueOptions: {
      durable: true,
      maxPriority: 10,
    },
    routingKey: 'message-b',
  })
  async handleB() {}
}

after a few days running then message-a-queue consumer disconnects! There is nothing logged and I have no clue how to debug this. To me it looks like only the later one reconnects.

zimiovid commented 1 year ago

Same problem!

Andr-07 commented 1 year ago

Same for me -_-

underfisk commented 1 year ago

@wodka Could you provide a small repo (the same exact golevelup modules version) so that we can see what's going on?

wodka commented 1 year ago

I can try - problem is that this is only happening after running for a few days :/

underfisk commented 1 year ago

I can try - problem is that this is only happening after running for a few days :/

I believe there must be something that the module router that we have read might be throwing in which case the connection might be killed and not re-connected but I'd like to see if at some point I can play around with your repro and kinda attempt to force one handler to go down. If at some point you can give us more context, that would be great but feel free to create the repro when you can

EvgeniLeonti commented 1 year ago

we have a similar setup and also experience similar issues, in our case: this.amqpConnection.publish('EVENT_QUEUE', 'tx-created', {...}); throws an error

throw new Error('AMQP connection is not available');
/app/node_modules/@golevelup/nestjs-rabbitmq/lib/amqp/connection.js:192

this causes nestjs service to restart (if no catch handler exists) and seems like there has been a memory leak since then.

wodka commented 1 year ago

:/ trying to get it to reproduce I've now found one problem that was earlier not logged:

Error: Channel closed by server: 404 (NOT-FOUND)

-> the Channel exits and is filling and with a restart of the service process the entries...

more detailed logs ``` Failed to setup a RabbitMQ channel - name: AmqpConnection / error: Channel ended, no reply will be forthcoming Error: Channel ended, no reply will be forthcoming at rej (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:201:7) at C._rejectPending (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:207:42) at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:171:8) at C._closeChannels (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:394:18) at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:401:8) at C.onSocketError (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:355:10) at Connection.emit (node:events:513:28) at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:481:12) at Socket.emit (node:events:513:28) at emitReadable_ (node:internal/streams/readable:590:12) at process.processTicksAndRejections (node:internal/process/task_queues:81:21)"} Failed to setup a RabbitMQ channel - name: AmqpConnection / error: Operation failed: QueueDeclare; 404 (NOT-FOUND) with message "NOT_FOUND - home node 'rabbit@rabbitmq-0.rabbitmq.rabbitmq-35547411-production.svc.cluster.local' of durable queue 'pigeon-email-transactional-queue' in vhost '/' is down or inaccessible" Error: Operation failed: QueueDeclare; 404 (NOT-FOUND) with message "NOT_FOUND - home node 'rabbit@rabbitmq-0.rabbitmq.rabbitmq-35547411-production.svc.cluster.local' of durable queue 'pigeon-email-transactional-queue' in vhost '/' is down or inaccessible" at reply (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:134:29) at C.accept (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:417:7) at Connection.mainAccept [as accept] (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:64:33) at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:478:48) at Socket.emit (node:events:513:28) at emitReadable_ (node:internal/streams/readable:590:12) at process.processTicksAndRejections (node:internal/process/task_queues:81:21) Failed to setup a RabbitMQ channel - name: AmqpConnection / error: Channel ended, no reply will be forthcoming Error: Channel ended, no reply will be forthcoming at rej (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:201:7) at C._rejectPending (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:207:42) at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:171:8) at C._closeChannels (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:394:18) at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:401:8) at C.onSocketError (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:355:10) at Connection.emit (node:events:513:28) at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:481:12) at Socket.emit (node:events:513:28) at emitReadable_ (node:internal/streams/readable:590:12) at process.processTicksAndRejections (node:internal/process/task_queues:81:21) Successfully connected to RabbitMQ broker (default) Error: Channel closed by server: 404 (NOT-FOUND) with message "NOT_FOUND - home node 'rabbit@rabbitmq-0.rabbitmq.rabbitmq-35547411-production.svc.cluster.local' of durable queue 'pigeon-email-transactional-queue' in vhost '/' is down or inaccessible" at C.accept (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:422:17) at Connection.mainAccept (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:64:33) at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:478:48) at Socket.emit (node:events:513:28) at emitReadable_ (node:internal/streams/readable:590:12) at process.processTicksAndRejections (node:internal/process/task_queues:81:21) Unhandled rejection Error: Channel ended, no reply will be forthcoming at rej (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:201:7) at C._rejectPending (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:207:42) at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:171:8) at C._closeChannels (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:394:18) at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:401:8) at C.onSocketError (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:355:10) at Connection.emit (node:events:513:28) at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:481:12) at Socket.emit (node:events:513:28) at emitReadable_ (node:internal/streams/readable:590:12) at process.processTicksAndRejections (node:internal/process/task_queues:81:21) Unhandled rejection Error: Channel ended, no reply will be forthcoming at rej (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:201:7) at C._rejectPending (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:207:42) at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:171:8) at C._closeChannels (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:394:18) at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:401:8) at C.onSocketError (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:355:10) at Connection.emit (node:events:513:28) at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:481:12) at Socket.emit (node:events:513:28) at emitReadable_ (node:internal/streams/readable:590:12) at process.processTicksAndRejections (node:internal/process/task_queues:81:21) ```

Edit: not sure that error is really related, I keep trying

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 30 days with no activity.