search

HangfireIO / Hangfire

An easy way to perform background job processing in .NET and .NET Core applications. No Windows Service or separate process required
https://www.hangfire.io
Other
9.41k stars 1.7k forks source link

Support automatic selection of queues (Enhancement) #879

Open YuriyZanichkovskyy opened 7 years ago

YuriyZanichkovskyy commented 7 years ago

It would be nice if instead of static list in BackgroundServerOptions for Queues we had a Func<string, bool> ProcessQueues, that is based on existing queues. That would allow processing for dynamic queues.

For example, we have a multi-tenancy application where each tenant has a separate queue and we would like to have an ability to scale processing per tenant. var backgroundServerOptions = new BackgroundServerOptions() { ProcessQueues = s => s.StartsWith("tenantcode") or s => config.Tenants.Contains(s) etc. };

dcorbett-radian commented 7 years ago

+1 the ability to prioritize / scale dynamically on a given machine ( and/or application-wide ) per queue would be awesome.

odinserj commented 7 years ago

Interesting idea, will review it for 1.7.0.

drungrin commented 6 years ago

I believe it is related to this enhancement.

We have the option to priorize the queues.

var options = new BackgroundJobServerOptions { Queues = new[] { "critical", "default" } };

But if the critical queue consume all the available Workers, everything else hangs.

For solving this issue, I create a lot of queues:

foreach (var queue in queues) { app.UseHangfireServer(new BackgroundJobServerOptions { WorkerCount = workerCount, ServerName = $"{serverName}_{queue}", Queues = new[] { queue } }); }

My suggestion is that you could have the option to set the workercount on the queue also, to make it even more dynamic.