Staging - [Alerting] Build Analysis: Exceptions and Errors Alert

[dotnet-eng-status-staging[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

• Message 11

Go to rule

@dotnet/dnceng, @dotnet/prodconsvcs, please investigate

Automation information below, do not change

Grafana-Automated-Alert-Id-6fe0b7b34a004f0bad0064a42f9b9135

[dotnet-eng-status-staging[bot]]

:green_heart: Metric state changed to ok

Description and instructions for this alert

Go to rule

[dotnet-eng-status-staging[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

• Message 11

Go to rule

[meghnave]

These were all the following error - "An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full. (api.github.com:443)" - which seems pretty consistent in the previous occurrences of this alert as well. And the service has been running fine since with bouts of this. Looking at old (2023) occurrences, this could be related to lack of storage in helix staging? So maybe there's not much to do here.

[garath]

It wouldn't be disk storage, but possibly socket exhaustion maybe?