search

dubek / rabbitmq-dump-queue

Dump messages from a RabbitMQ queue to files, without affecting the queue.
MIT License
217 stars 44 forks source link

RabbitMQ is down #10

Open darksmoke opened 7 years ago

darksmoke commented 7 years ago

I had 1.3 million messages in the queue. I ran it like this ./rabbitmq-dump-queue -uri = "amqp: // login:password@10.0.0.2: 5672 /% 2Fpaystatus" -queue = StatusQueueFromDLX -max-messages = 5000000 -output-dir = ./tmp 605.000 messages were subtracted and RabbitMQ is down

dubek commented 7 years ago

I'm sorry the server went down. I hope the messages are intact - rabbitmq-dump-queue never acks the messages, so they should stay in the queue if the client or server goes down.

Do you have RabbitMQ server logs from this incident?

magnusbaeck commented 4 years ago

I have no affiliation with @darksmoke, but I probably ran into the same problem without using rabbitmq-dump-queue; when a client with a large number of unacked messages disconnects is seems RabbitMQ (3.7 in our case) needs to make a large memory allocation. If that fails RabbitMQ will crash. Whether this is a bug in RabbitMQ (why would it need to allocate memory to hold the whole queue in RAM, if that indeed is what it's doing?) or a design flaw in rabbitmq-dump-queue is perhaps debatable.

dubek commented 4 years ago

Maybe we can add some warning in the README of this possible outcome.