legacy_watchdog.py: Add legacy KernelCI watchdog

kernelci / kernelci-deploy

Tools to deploy and update kernelci.org

GNU Lesser General Public License v2.1

4 stars 14 forks source link

legacy_watchdog.py: Add legacy KernelCI watchdog #118

Closed nuclearcat closed 5 months ago

nuclearcat commented 6 months ago

As we have endless issues with "dashboard is down", we were able to identify 2 reasons: 1)Celery queue getting stuck by some reason 2)OOM This is workaround of celery queue issues.

This script checks celery queue, and if there is no abnormal load average, it clearly means just celery issue.

nuclearcat commented 5 months ago

Tested on staging and production