Support for `async` background job methods

[odinserj]

Heavy I/O bound background jobs consume worker thread inefficiently – instead of doing the real work they wait for the completion of I/O operation(s). Currently if an application has a lot of such background jobs, it is better to increase the worker pool size to process jobs more efficiently. However increased worker pool causes performance problems for CPU-bound jobs.

Instead of waiting for the I/O completion, it would be useful to yield the worker thread to process another background job instead, for example, using the default mechanism with async and await keywords:

public async Task IOBoundMethod()
{
    var documents = await _documentsService.FetchAll();
    documents.Process();
}

But we should ensure that the completion is being called inside the Hangfire's worker thread, and not in CLR's thread pool thread (to not to compete with request processing pipeline, use additional storage connections and so on). Another consideration should be dedicated to job filters – CaptureCultureAttribute changes the thread context, and it should be changed again on each continuation. There are also problems with reliability (outstanding jobs should be requeued), shutdown events (outstanding operations should be canceled), and perhaps with something else.

[ghuntley]

Yo this would be amazing. +1

[devmondo]

cant wait to see this come really :+1:

[unt1tled]

+1 here too. Definitely a requirement on my end.

[marius-stanescu]

+1 this would be very nice (http://discuss.hangfire.io/t/async-task-jobs/73)

[marcoCasamento]

that would be great!

[itmeze]

+1 killer feature for me

[treymack]

:+1:

[Blackbaud-MichaelAndrews]

+1

[Isantipov]

As of now Hangfire does not protect you from incorrectly using async methods as background tasks. I.e. you can schedule an async operation which will run successfully (however, Hangfire will not await the result so esceptions will be lost + job execution statistics will be incorrect). If async support is far down the road, how about throwing an exception when trying to schedule/run a job which returns a Task?

[odinserj]

@Isantipov, this is a brilliant proposal, will schedule it for next minor release

[di97mni]

:+1:

[parliament718]

+1. Maybe this year?

[chfumero]

+1

[oliverkane]

+1

[mjohnson0580]

+1

[jbizkit]

+1

[levmatta]

+1 (luis.matta recent pro buyer)

[Otto404]

+1

[keesschollaart81]

+1

[AdityaSantoso]

+1

[odinserj]

Wow, such a popular feature, want it too :-) This feature requires a lot of breaking changes and it is postponed to the version 2.0, that will be released in the next year with async methods for storages as well.

[marcoCasamento]

@odinserj, do you already have any idea about the async method you need from the storage ?

[odinserj]

@marcoCasamento, please see #401.

[Gillardo]

Is there any news on this yet?

[dandcg]

Hi @odinserj - any idea when async will be here. Will definitely be moving up to the pro version when this happens?

[tuespetre]

+1

...except I'm not sure how you're going to pull off keeping it limited to pro. In my mind only two things need to be done:

1. Add overloads that accept Expression<Func<Task>>, including those variations that accept a generic instance argument, and variations that accept CancellationToken instances as well it looks like there is already support for CancellationToken
2. Add the bit of code that waits for the Task, if any, to complete and catches any AggregateException (possibly with some flattening.) (Link to relevant code)

[tuespetre]

Alright, so I've submitted a PR that adds support. I should note that for the time being, it's really not a big deal to go without this support -- all one needs to do is provide both SomeMethod and SomeMethodAsync:

public void SomeMethod()
{    
    try
    {
        SomeMethodAsync().Wait();
    }
    catch (AggregateException aggregate)
    {
        throw aggregate.InnerException;
    }
}

public async Task SomeMethodAsync()
{
    await ...
}

A little bit of churn, sure, but then "actual" async support requires one to put in those #pragma warning disable 4014 statements or deal with compiler warnings. Oh well. :beers:

[rosdi]

+1

[bezzad]

+1

[mwhitis]

+1

[odinserj]

Okay, guys. Thanks to @tuespetre, the first step was done via merging PR #540. It doesn't add real asynchrony to background processing itself (we still need to guess the worker count), but we can finally forget the headache with deciding what to do with async methods in Hangfire, async/await keywords are available now.

I'll release 1.6.0-beta1 next week that will include this change.

[odinserj]

Hangfire 1.6.0 is just released. Here is the sample from Hangfire.Highlighter:

[viktor-evdokimov]

@odinserj so is it available in Hangfire or only in Hangfire.PRO ?

[odinserj]

@ojosdegris, this feature is available in Hangfire 1.6.0-beta1, full async implementation will also be shipped in the open-source version, that's why I removed the pro tag.

[viktor-evdokimov]

@odinserj Thanks! noticed that but was not sure.

[kfrancis]

What's the timeline on 1.6.0 moving past beta?

[mlewski]

+1. @odinserj , there's a comment that this is scheduled for 2.0.0, could update the issue-milestone as well?

[Korayem]

as of v1.6.0, async support was added

@odinserj please close this issue

[Salgat]

@Korayem that is just a wrapper around async methods, not true async support (as noted in the issue).

[oguzhantopcu]

Is there any plan for real async jobs?

[davidrevoledo]

If I put this parameter PerformContext in the async code it fails is there any other way to get this context ?

[amatosolivo]

Is this issue FIXED in version 1.7.32?

[DMiradakis]

True async is still not supported, similar conversation happening on #1658 .

From what I have gathered, the main issues are:

1. .GetAwaiter() and other similar synchronous methods are used in the deeper parts of the execution engine, even when async background jobs are executed. Thus, their threads are synchronously blocked.
2. Sometimes thread pool starvation can still occur from the job storage APIs, such as Microsoft.Data.SqlClient. For example, in that library, some of the supposedly async methods are actually invoking their sync counterparts (they are superficially async).

I've been looking into this a lot because I am building the first ever dotnet job orchestrator, Didact, which is a little similar to Hangfire in some ways but very different in others. I think I will be able to solve 1 in Didact but not solve 2 since those are deeper library problems. I still very much appreciate Hangfire for the use cases it solves, though.

It's probably worth you going and adding your 👍 emoji to #1658 .