🛑 Dashboards-NBG is down

[thomasmerz]

In 36d27b7, Dashboards-NBG ($URL_DASHBOARDS_DE) was down:

• HTTP code: 0
• Response time: 0 ms

[thomasmerz]

Resolved: Dashboards-NBG is back up in 55adc06.

[thomasmerz]

ssh not responding… 😢 had to powercycle this VM…

[thomasmerz]

RCA: Gap in dnspingtest_rrd.ubuntu-nbg1-1/dashboard_day.html indicates that there has been some problem starting after 6am.

[thomasmerz]

Starting at 06:09 Netdata shows anomalies:

• load is exploading:
• processes are exploding:
• committed memory is exploding:
• tcp sockets is exploding:
• apps open file descriptors limit: (those from user root)
• "Anomaly Detection" didn't find anything 😞