Health Checks: publish health report on event

[dlyz]

Is there an existing issue for this?

• [X] I have searched the existing issues

Is your feature request related to a problem? Please describe the problem.

Assuming we use a health check publisher pattern, when we actively push health state to some external system. Let's consider an example from official docs:

public class StartupHealthCheck : IHealthCheck
{
    private volatile bool _isReady;

    public bool StartupCompleted
    {
        get => _isReady;
        set => _isReady = value;
    }

    public Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context, CancellationToken cancellationToken = default)
    {
        if (StartupCompleted)
        {
            return Task.FromResult(HealthCheckResult.Healthy("The startup task has completed."));
        }

        return Task.FromResult(HealthCheckResult.Unhealthy("That startup task is still running."));
    }
}

Important thing in this example is that we know exactly when the health state changes - when property StartupCompleted changes. But in the box there is only HealthCheckPublisherHostedService that pushes health reports to publishers only on timer (default interval is 30 seconds). So there might be a noticeable delay between health state change and report of this state.

Only available solution out of the box is to make publish intervals smaller, but it probably will waste the processor more that it should, considering that most of the time there are no problems with the health.

Describe the solution you'd like

It would be nice to have a way to (actively) notify HealthCheckPublisherHostedService when the health state changes as an addition to periodical reports.

For example we can use ChangeToken pattern from options:

interface IHealthChangeTokenSource
{
  IChangeToken GetChangeToken();
}

So HealthCheckPublisherHostedService will inject IEnumerable<IHealthChangeTokenSource> from the container, track change tokens and report whenever something changes.

In this case the example might look like this:

public class StartupHealthCheck : IHealthCheck, IHealthChangeTokenSource
{
    private volatile bool _isReady;

    public bool StartupCompleted
    {
        get => _isReady;
        set
        {
            _isReady = value;
            OnChange();
        }
    }

    private CancellationTokenSource _changeTokenSource = new();

    public IChangeToken GetChangeToken() => new CancellationChangeToken(_changeTokenSource.Token);

    private void OnChange()
    {
        var previousToken = Interlocked.Exchange(ref _changeTokenSource, new CancellationTokenSource());
        previousToken.Cancel();
    }

    public Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context, CancellationToken cancellationToken = default)
    {
        if (StartupCompleted)
        {
            return Task.FromResult(HealthCheckResult.Healthy("The startup task has completed."));
        }

        return Task.FromResult(HealthCheckResult.Unhealthy("That startup task is still running."));
    }
}

There even might be a helper abstract class that will implement change token update logic (provide protected OnChange method) to make it easier for health check implementers.

The registration of StartupHealthCheck will change from

builder.Services.AddSingleton<StartupHealthCheck>();

to

builder.Services.AddSingleton<StartupHealthCheck>();
builder.Services.AddSingleton<IHealthChangeTokenSource>(sp => sp.GetRequiredService<StartupHealthCheck>());

If this case will be considered too custom to support out of the box, it will be nice to have at least an opportunity to implement it in my own project. Now this requires a nasty workaround: I have to delete HealthCheckPublisherHostedService and implement my own with almost the same code (see also #29896).

Additional context

No response

[ghost]

We've moved this issue to the Backlog milestone. This means that it is not going to be worked on for the coming release. We will reassess the backlog following the current release and consider this item at that time. To learn more about our issue management process and to have better expectation regarding different types of issues you can read our Triage Process.

[JamesNK]

This is an interesting idea but a fair-sized change to health checks. I think we'd need to see if more people are interested in being able to receive health checks changes in real-time before we invest in this feature.

An alternative idea is reducing the interval of checks in HealthCheckPublisherHostedService.