Production - [Alerting] Android devices disconnected

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

@dotnet/dnceng, please investigate

Automation information below, do not change

Grafana-Automated-Alert-Id-35f560112f7a4bfabf9fd69bc1bd76fa

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-070} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-079} 100
• FailureRate {Machine=DNCENGWIN-091} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-079} 100
• FailureRate {Machine=DNCENGWIN-091} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[AlitzelMendez]

I put the machines offline and opened an IcM

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-079} 100
• FailureRate {Machine=DNCENGWIN-091} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-079} 100
• FailureRate {Machine=DNCENGWIN-091} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-079} 100
• FailureRate {Machine=DNCENGWIN-091} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-079} 100
• FailureRate {Machine=DNCENGWIN-091} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-025} 100
• FailureRate {Machine=DNCENGWIN-047} 100
• FailureRate {Machine=DNCENGWIN-070} 100
• FailureRate {Machine=DNCENGWIN-079} 100
• FailureRate {Machine=DNCENGWIN-091} 100
• FailureRate {Machine=DNCENGWIN-093} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-108} 100
• FailureRate {Machine=DNCENGWIN-118} 100

Go to rule

[dotnet-eng-status[bot]]

:green_heart: Metric state changed to ok

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-005} 100
• FailureRate {Machine=DNCENGWIN-085} 100
• FailureRate {Machine=DNCENGWIN-103} 100

Go to rule

[AlitzelMendez]

new machines already offline and added to IcM: https://portal.microsofticm.com/imp/v3/incidents/incident/413800516/summary

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-005} 100
• FailureRate {Machine=DNCENGWIN-085} 100
• FailureRate {Machine=DNCENGWIN-103} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-005} 100
• FailureRate {Machine=DNCENGWIN-085} 100
• FailureRate {Machine=DNCENGWIN-103} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-005} 100
• FailureRate {Machine=DNCENGWIN-085} 100
• FailureRate {Machine=DNCENGWIN-103} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-005} 100
• FailureRate {Machine=DNCENGWIN-085} 100
• FailureRate {Machine=DNCENGWIN-103} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-005} 100
• FailureRate {Machine=DNCENGWIN-085} 100
• FailureRate {Machine=DNCENGWIN-103} 100

Go to rule

[AlitzelMendez]

I got an update on the IcM that the following machines has been fixed: DNCENGWIN-025 DNCENGWIN-047 DNCENGWIN-070 DNCENGWIN-079 DNCENGWIN-091 DNCENGWIN-093 DNCENGWIN-104 DNCENGWIN-108 DNCENGWIN-118

putting above machines online and still waiting for DNCENGWIN-005 DNCENGWIN-085 DNCENGWIN-103

[dougbu]

nice @AlitzelMendez maybe we can see this alert die down for more than a day 😦

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-005} 100
• FailureRate {Machine=DNCENGWIN-085} 100
• FailureRate {Machine=DNCENGWIN-103} 100

Go to rule

[dotnet-eng-status[bot]]

:green_heart: Metric state changed to ok

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100
• FailureRate {Machine=DNCENGWIN-057} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100
• FailureRate {Machine=DNCENGWIN-057} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100
• FailureRate {Machine=DNCENGWIN-057} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100
• FailureRate {Machine=DNCENGWIN-057} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100
• FailureRate {Machine=DNCENGWIN-057} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100
• FailureRate {Machine=DNCENGWIN-057} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100
• FailureRate {Machine=DNCENGWIN-057} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100
• FailureRate {Machine=DNCENGWIN-057} 100

Go to rule

[chcosta]

Filed ICM - https://portal.microsofticm.com/imp/v3/incidents/details/418013216/home

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-001} 100
• FailureRate {Machine=DNCENGWIN-057} 100

Go to rule

[dotnet-eng-status[bot]]

:green_heart: Metric state changed to ok

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

Go to rule