chrisckc
commented
5 years ago According to the Docs: https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-2.2&tabs=visual-studio#ihostedservice-interface
The cancellation token has a default five second timeout to indicate that the shutdown process should no longer be graceful. When cancellation is requested on the token:
Any remaining background operations that the app is performing should be aborted. Any methods called in StopAsync should return promptly.
However, tasks aren't abandoned after cancellation is requested—the caller awaits all tasks to complete.
I am not sure exactly what this means, the last sentence seems to contradict the previous statements.
To extend the default five second shutdown timeout, set:
ShutdownTimeout when using Generic Host. For more information, see .NET Generic Host.
In dotnet 2.2 there is no option to specify a shutdown timeout on either StopAsync() or WaitForShutdownAsync(), despite the Docs being for 2.2
If all stop operations ran concurrently, a low shutdown timeout should be ok, but as they seem to be running sequentially, a much longer timeout would be required.
analogrelay
Not sure if this is an issue or by design but i have been experimenting with the GenericHostSample code from here:
https://github.com/aspnet/Extensions/tree/master/src/Hosting/samples/GenericHostSample
and found a couple of things which may need to be addressed.
I have created a repo based on the above code to test and demonstrate the issues:
https://github.com/chrisckc/DotNetCoreGenericHostSample
The Readme.md explains the issues, but i will point them out here:
First issue:
The Sample does not indicate how to properly cleanup after running a task when using MyServiceA which inherits from BackgroundService. Using the Sample code as-is with just MyServiceA added, pressing Ctrl-C results in any code after
await Task.Delay(TimeSpan.FromSeconds(5), stoppingToken);not running due to the fact that cancelling a running Task raises an Exception, here is the console output from the repo i created above:
Notice that the "MyServiceA background task is stopping" message is not shown. The issue can be rectified by changing the Delay code to this so that the Exception is swallowed:
Resulting in the following console output, notice that the "MyServiceA background task is stopping" message is shown and clean-up completed.
Second Issue:
When the GenericHost is stopping multiple concurrent HostedServices due to Ctrl-C, the services are stopped sequentially, each stop operation seems to be blocked until the HostedService which is being stopped has finished stopping, which could include running any required clean-up code etc.
This does not appear to cause an issue as long as the Tasks being ran in each HostedService are setup to accept the cancellation token as per the sample, however the logs look rather odd as you can see that another service starts its background work cycle after one service has announced that it is stopping. If the long running Tasks in each HostedService accept cancellation tokens, they are immediately cancelled just after being started so everything stops quickly (as quick as the time taken for any clean-up code to run). Really they should not have been started in the first place because of the check in the while loop
while (!stoppingToken.IsCancellationRequested)but because each Stop operation is triggered sequentially, the cancellation token in each HostedService Instance is only triggered after the previous HostedService has finished stopping.I created 3 new Services, MyServiceX, MyServiceY, and MyServiceZ which are basically copies of MyServiceA in the repo, to test the above scenario you will need to uncomment the Delay line in each of those services which accepts the cancellation token and comment out the one which does not. Also go into ProgramHelloWorld and uncomment those 3 services instead of A.
Third issue:
If MyServiceX, MyServiceY, and MyServiceZ use a Delay which does not accept a cancellationToken, the GenericHost crashes shortly after pressing Ctrl-C. This only happens when more than 1 of the services is added in the DI config and so the GenericHost attempts to stop more than 1 service.
To test this use the original code in MyServiceX, MyServiceY, and MyServiceZ using the Delay which does not accept cancellation token.
Here is what the console output looks like:
To make this worse, when running in docker and inspecting to container logs after stopping the container, the Exception is not there, the last entry is
This can be fixed with a try catch and console log as mentioned in the Readme.md, but only for the ProgramFullControl setup.
Expected behavior
For the sample to demonstrate usage closer to real world and show how to properly stop and cleanup multiple HostedServices
Also real work usage will often require a delayed start of the HostedServices if they access databases and everything is running in docker due to startup timing issues. I have added a simple solution for this as mentioned in the Readme.m
I intend to add a copy of the BackgroundService base class in my repo and have a play around to find out the cause of the multiple services issue, but time has other demands at the moment.
Possibly loosely related:
https://github.com/aspnet/Extensions/issues/813
https://github.com/aspnet/Extensions/issues/1245
OS: MacOS 10.14.4 SDK 2.2.106