Production - [Alerting] Android devices disconnected

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-008} 100
• FailureRate {Machine=DNCENGWIN-009} 100
• FailureRate {Machine=DNCENGWIN-011} 100
• FailureRate {Machine=DNCENGWIN-015} 100
• FailureRate {Machine=DNCENGWIN-020} 100
• FailureRate {Machine=DNCENGWIN-073} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-113} 100
• FailureRate {Machine=DNCENGWIN-133} 100
• FailureRate {Machine=DNCENGWIN-142} 100
• FailureRate {Machine=DNCENGWIN-147} 100

Go to rule

@dotnet/dnceng, @dotnet/prodconsvcs, please investigate

Automation information below, do not change

Grafana-Automated-Alert-Id-35f560112f7a4bfabf9fd69bc1bd76fa

[ilyas1974]

ICM created - https://portal.microsofticm.com/imp/v5/incidents/details/528502232/summary

[oleksandr-didyk]

@pavel-purma is on FR this week since I was sick for the fist part of the week, so I'll re-assign it to him

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-008} 100
• FailureRate {Machine=DNCENGWIN-009} 100
• FailureRate {Machine=DNCENGWIN-015} 100
• FailureRate {Machine=DNCENGWIN-020} 100
• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-073} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-113} 100
• FailureRate {Machine=DNCENGWIN-121} 88
• FailureRate {Machine=DNCENGWIN-133} 100
• FailureRate {Machine=DNCENGWIN-142} 100
• FailureRate {Machine=DNCENGWIN-147} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-113} 100
• FailureRate {Machine=DNCENGWIN-121} 88
• FailureRate {Machine=DNCENGWIN-147} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-104} 100
• FailureRate {Machine=DNCENGWIN-113} 100
• FailureRate {Machine=DNCENGWIN-121} 88
• FailureRate {Machine=DNCENGWIN-147} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 88

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 88

Go to rule

[dotnet-eng-status[bot]]

:green_heart: Metric state changed to ok

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-124} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-124} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-124} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-124} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-124} 100
• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-124} 100
• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-124} 100
• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-124} 100
• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-124} 100
• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-055} 100
• FailureRate {Machine=DNCENGWIN-121} 100
• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[dotnet-eng-status[bot]]

:broken_heart: Metric state changed to alerting

Description and instructions for this alert

Please note that this alert will fire every 12 hours as the list of machines can change while the alert is alive. So please keep an eye on the list of machines in the comment.

• FailureRate {Machine=DNCENGWIN-145} 100

Go to rule

[MilenaHristova]

I disabled DNCENGWIN-145 and opened https://portal.microsofticm.com/imp/v5/incidents/details/533188979/summary?tmpl=21c3we

[MilenaHristova]

https://portal.microsofticm.com/imp/v5/incidents/details/528502232/summary is resolved, I enabled all machines

• DNCENGWIN-008
• DNCENGWIN-009
• DNCENGWIN-011
• DNCENGWIN-015
• DNCENGWIN-020
• DNCENGWIN-073
• DNCENGWIN-104
• DNCENGWIN-113
• DNCENGWIN-133
• DNCENGWIN-142
• DNCENGWIN-147
• DNCENGWIN-145

Closing this as resolved