JasperFx / wolverine

Supercharged .NET server side development!
https://wolverinefx.net
MIT License
1.25k stars 137 forks source link

Bring back inbox/outbox metrics #357

Open jeremydmiller opened 1 year ago

jeremydmiller commented 1 year ago

The metrics sampling for the inbox/outbox is gonna go all to hell with multi-tenancy. Just bring it back per database later.

See:

internal class MetricsCalculator : IDurabilityAction
{
    private readonly ObservableGauge<int> _incoming;
    private readonly ObservableGauge<int> _outgoing;
    private readonly ObservableGauge<int> _scheduled;

    public MetricsCalculator(Meter meter)
    {
        _incoming = meter.CreateObservableGauge(MetricsConstants.InboxCount, () => Counts.Incoming,
            MetricsConstants.Messages, "Inbox messages");
        _outgoing = meter.CreateObservableGauge(MetricsConstants.OutboxCount, () => Counts.Outgoing,
            MetricsConstants.Messages, "Outbox messages");
        _scheduled = meter.CreateObservableGauge(MetricsConstants.ScheduledCount, () => Counts.Scheduled,
            MetricsConstants.Messages, "Scheduled messages");
    }

    public PersistedCounts Counts { get; private set; } = new();

    public string Description { get; } = "Metrics collection of inbox and outbox";

    public async Task ExecuteAsync(IMessageDatabase database, IDurabilityAgent agent,
        IDurableStorageSession session)
    {
        var counts = await database.FetchCountsAsync();
        Counts = counts;
    }
}
dgrozenok commented 9 months ago

Would be great to have those metrics working. Need to monitor the inbox/outbox counters while troubleshooting the messages not being processed timely.

jeremydmiller commented 1 month ago

Sorry Denys, these are getting bounced back to 4.0:(

Also, add this back to the docs:

| wolverine-inbox-count        | [Observable Gauge](https://opentelemetry.io/docs/reference/specification/metrics/api/#asynchronous-gauge) | Samples the number of pending envelopes in the durable inbox (likely to change)                                                                                                                                                                                                        |
| wolverine-outbox-count       | Observable Gauge                                                                                          | Samples the number of pending envelopes in the durable outbox (likely to change)                                                                                                                                                                                                       |
| wolverine-scheduled-count    | Observable Gauge                                                                                          | Samples the number of pending scheduled envelopes in the durable inbox (likely to change)                                                                                                                                                                                              |