rebus-org / Rebus.RabbitMq

:bus: RabbitMQ transport for Rebus
https://mookid.dk/category/rebus
Other
65 stars 45 forks source link

Application crashed when consume a long-running task #84

Closed volethanh closed 3 years ago

volethanh commented 3 years ago

Hi,

I'm facing a problem that I have a queue to handle long-running task, but it crashed and the message error show Though it was possible to receive, but the channel turned out to be closed... it will be renewed after a short wait

Screen Shot 2021-09-21 at 3 23 01 PM

Is there any way to configure timeout when queue consume message ?

Regards, Thanh

mookid8000 commented 3 years ago

The error message you're seeing is not caused by taking a long time to handle a message, it's caused by Rebus discovering that the channel was closed while trying to receive a message, which it'll know because it gets a ChannelClosedException from the RabbitMQ driver.

This is most likely just a sign that the connection between your Rebus instance and your RabbitMQ server has been broken and must be re-established (which Rebus will automatically attempt to do after a 30 s pause in this case).

With RabbitMQ, your message handler can take as long as you want to handle a message, as long as the connection to RabbitMQ is maintained.

If you want some kind of timeout on your message handling, you'll need to implement that yourself, which is fairly easy for async stuff in your Rebus handlers:

using var cancellationTokenSource = new CancellationTokenSource(TimeSpan.FromMinutes(2));

var cancellationToken = cancellationTokenSource.Token;

// pass cancellationToken to your async stuff
await DoWork(cancellationToken);

// check it manually
while (!cancellationToken.IsCancellationRequested)
{
    await DoStuff();
}

// or make it throw
foreach (var item in lotsOfItems)
{
    cancellationToken.ThrowIfCancellationRequested();

    await ProcessItem(item);
}
volethanh commented 3 years ago

The error message you're seeing is not caused by taking a long time to handle a message, it's caused by Rebus discovering that the channel was closed while trying to receive a message, which it'll know because it gets a ChannelClosedException from the RabbitMQ driver.

This is most likely just a sign that the connection between your Rebus instance and your RabbitMQ server has been broken and must be re-established (which Rebus will automatically attempt to do after a 30 s pause in this case).

With RabbitMQ, your message handler can take as long as you want to handle a message, as long as the connection to RabbitMQ is maintained.

If you want some kind of timeout on your message handling, you'll need to implement that yourself, which is fairly easy for async stuff in your Rebus handlers:

using var cancellationTokenSource = new CancellationTokenSource(TimeSpan.FromMinutes(2));

var cancellationToken = cancellationTokenSource.Token;

// pass cancellationToken to your async stuff
await DoWork(cancellationToken);

// check it manually
while (!cancellationToken.IsCancellationRequested)
{
  await DoStuff();
}

// or make it throw
foreach (var item in lotsOfItems)
{
  cancellationToken.ThrowIfCancellationRequested();

  await ProcessItem(item);
}

I've tried like above but no luck :(

mookid8000 commented 3 years ago

How would you suggest this problem is solved?