dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.97k stars 4.65k forks source link

UseWindowsService reports a gracefull ServiceStop to SCM even in case of exception, so it's impossible to schedule a restart #77381

Open beppemarazzi opened 1 year ago

beppemarazzi commented 1 year ago

Description

If there is an unexpected exception into a BackgroundService, the Host correctly tears down all the application by default. (see https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.hosting.hostoptions.backgroundserviceexceptionbehavior)

The problem is that when the application is executed in a WindowsService through UseWindowsService API, the ServiceControlManager is notified of a normal ServiceStop in this case. So it's impossible to configure the SCM to restart the service in case of unexpected termination.

Reproduction Steps

  1. Compile this simple program
    
    var options = new WebApplicationOptions
    {
    Args = args,
    ContentRootPath = AppContext.BaseDirectory
    };

var builder = WebApplication.CreateBuilder(options); builder.Host.UseWindowsService(); builder.Services.AddHostedService();

var app = builder.Build();

app.Run();

class MyHS : BackgroundService { private readonly ILogger _logger; public MyHS(ILogger logger) { _logger = logger; } protected override async Task ExecuteAsync(CancellationToken stoppingToken) { _logger.LogInformation("Started"); while(!stoppingToken.IsCancellationRequested) { if (File.Exists("c:\crash.txt")) throw new InvalidOperationException("CRASH");

        await Task.Delay(2000);
    }        
    _logger.LogInformation("Stopped");
}

}


2. Install it as a Windows Service (i.e. `sc create _myService binpath=C:\Devel\test\WebApplication2\WebApplication2\bin\Debug\net6.0\WebApplication2.exe`
3. Configure the SCM to restart the service in case of unexpected error (i.e. `sc failure _myService reset=0 actions=restart/5000`
4. Start the service
5. Create a new file `c:\crash.txt`
6. Look at the service status and wait until it's dead
7. Delete the file `c:\crash.txt`
8. Wait some seconds (i.e. 5)

### Expected behavior

The service will restart

### Actual behavior

Nothing happens

### Regression?

_No response_

### Known Workarounds

I've developed a dirty workaround, but IMHO is preferable to officially address this issue.
In few words:

1. i'm intercepting the Host Logger to detect if there is an event with EventId == 10 raised when the Host is stopped due to an unexpected exception (see (https://source.dot.net/#Microsoft.Extensions.Hosting/Internal/Host.cs,110) )
2. i'm overriding the registration of the service `WindowsServiceLifetime` with something like this:

```cs
  ...
   services.AddSingleton<IHostLifetime, WindowsServiceLifetimeEx>();
   ...

    internal class WindowsServiceLifetimeEx : WindowsServiceLifetime, IHostLifetime
    {
        private readonly HostLogger m_hostLogger; //this is my wrapper around ILogger that stores the exception logged with event 10. It's injected into Host overriding the registration for ILogger<>...
        public WindowsServiceLifetimeEx(HostLogger hostLogger, IHostEnvironment environment, IHostApplicationLifetime applicationLifetime, ILoggerFactory loggerFactory, IOptions<HostOptions> optionsAccessor)
            : base(environment, applicationLifetime, loggerFactory, optionsAccessor)
        {
            m_hostLogger = hostLogger;
        }

        protected override void OnStop()
        {
            var ex = m_hostLogger.BackgroundServiceStoppingHostException;
            if (ex != null)
                throw ex;
            base.OnStop();
        }
    }

Configuration

Microsoft.AspNetCore.App 6.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]

λ [System.Environment]::OSVersion.Version

Major Minor Build Revision


10 0 19043 0

Other information

No response

ghost commented 1 year ago

Tagging subscribers to this area: @dotnet/area-extensions-hosting See info in area-owners.md if you want to be subscribed.

Issue Details
### Description If there is an unexpected exception into a `BackgroundService`, the Host correctly tears down all the application by default. (see https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.hosting.hostoptions.backgroundserviceexceptionbehavior) The problem is that when the application is executed in a WindowsService through [`UseWindowsService`](https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.hosting.windowsservicelifetimehostbuilderextensions.usewindowsservice) API, the ServiceControlManager is notified of a normal ServiceStop in this case. So it's impossible to configure the SCM to restart the service in case of unexpected termination. ### Reproduction Steps 1. Compile this simple program ``` var options = new WebApplicationOptions { Args = args, ContentRootPath = AppContext.BaseDirectory }; var builder = WebApplication.CreateBuilder(options); builder.Host.UseWindowsService(); builder.Services.AddHostedService(); var app = builder.Build(); app.Run(); class MyHS : BackgroundService { private readonly ILogger _logger; public MyHS(ILogger logger) { _logger = logger; } protected override async Task ExecuteAsync(CancellationToken stoppingToken) { _logger.LogInformation("Started"); while(!stoppingToken.IsCancellationRequested) { if (File.Exists("c:\\crash.txt")) throw new InvalidOperationException("CRASH"); await Task.Delay(2000); } _logger.LogInformation("Stopped"); } } ``` 2. Install it as a Windows Service (i.e. `sc create _myService binpath=C:\Devel\test\WebApplication2\WebApplication2\bin\Debug\net6.0\WebApplication2.exe` 3. Configure the SCM to restart the service in case of unexpected error (i.e. `sc failure _myService reset=0 actions=restart/5000` 4. Start the service 5. Create a new file `c:\crash.txt` 6. Look at the service status and wait until it's dead 7. Delete the file `c:\crash.txt` 8. Wait some seconds (i.e. 5) ### Expected behavior The service will restart ### Actual behavior Nothing happens ### Regression? _No response_ ### Known Workarounds I've developed a dirty workaround, but IMHO is preferable to officially address this issue. In few words: 1. i'm intercepting the Host Logger to detect if there is an event with EventId == 10 raised when the Host is stopped due to an unexpected exception (see (https://source.dot.net/#Microsoft.Extensions.Hosting/Internal/Host.cs,110) ) 2. i'm overriding the registration of the service `WindowsServiceLifetime` with something like this: ``` ... services.AddSingleton(); ... internal class WindowsServiceLifetimeEx : WindowsServiceLifetime, IHostLifetime { private readonly IHostApplicationLifetime m_applicationLifetime; private readonly ILogger m_logger; private readonly HostLogger m_hostLogger; public WindowsServiceLifetimeEx(HostLogger hostLogger, IHostEnvironment environment, IHostApplicationLifetime applicationLifetime, ILoggerFactory loggerFactory, IOptions optionsAccessor) : base(environment, applicationLifetime, loggerFactory, optionsAccessor) { m_applicationLifetime = applicationLifetime; m_logger = loggerFactory.CreateLogger(); m_hostLogger = hostLogger; } async Task IHostLifetime.StopAsync(CancellationToken cancellationToken) { var cleanExit = !m_hostLogger.IsBackgroundServiceStoppingHost; if (cleanExit) { m_logger.LogDebug("Clean exiting..."); await base.StopAsync(cancellationToken); m_logger.LogDebug("Clean exit!"); } else { m_logger.LogDebug("NOT Clean exiting..."); m_applicationLifetime.StopApplication(); try { await Task.Delay(-1, m_applicationLifetime.ApplicationStopped); } catch (OperationCanceledException) { } m_logger.LogDebug("NOT Clean exit!"); } } } ``` ### Configuration Microsoft.AspNetCore.App 6.0.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] λ [System.Environment]::OSVersion.Version Major Minor Build Revision ----- ----- ----- -------- 10 0 19043 0 ### Other information _No response_
Author: beppemarazzi
Assignees: -
Labels: `untriaged`, `area-Extensions-Hosting`
Milestone: -
beppemarazzi commented 1 year ago

Probably related with #50019

ericstj commented 1 year ago

Perhaps the request here would be to instead have a new value for BackgroundServiceExceptionBehavior which would be to rethrow the exception and let the process exit.

KalleOlaviNiemitalo commented 10 months ago

Not sure about rethrowing… it should be enough if you run sc.exe failureflag SERVICENAME 1 at install time and then set ServiceBase.ExitCode = nonzero (e.g. Exception.HResult) during the IHostApplicationLifetime.ApplicationStopped notification if the stop was caused by an error. Hosting will log the original exception anyway, so you don't need to rethrow that for ServiceBase to catch and log.