search

web-platform-tests / pulls.web-platform-tests.org

[Deprecated] Some functionalities are now provided by wpt-pr-bot https://github.com/web-platform-tests/wpt-pr-bot
7 stars 23 forks source link

pulls.web-platform-tests.org is down #39

Closed foolip closed 6 years ago

foolip commented 6 years ago

504 Gateway Time-out. Oops?

mattl commented 6 years ago

Looking now.

foolip commented 6 years ago

I noticed because of https://bit.ly/ecosystem-infra-status which @mdittmer created. Yay monitoring!

mattl commented 6 years ago

Yay monitoring. Also, getting a different error now. SSHing in to take a look.

mattl commented 6 years ago

Load is very high, restarted some services and I'll keep an eye on it for the next 15 minutes and make sure things come back up.

mattl commented 6 years ago

Things are back. Going to continue to monitor this machine until our meeting in about 20 minutes.

mattl commented 6 years ago

postgres is having some issues on this machine, and downtime is intermittent. Keeping this open for now until I can completely resolve.

mattl commented 6 years ago

Noticed the site was down again. Have increased the resources available on this VM from 512mb RAM to 2GB RAM, also upgraded postgresql to the latest Ubuntu packages. Will continue to keep an eye on it.

foolip commented 6 years ago

There was 10 minutes of downtime today as well: https://bit.ly/ecosystem-infra-status

mattl commented 6 years ago

Things are looking good. We're monitoring it from Uptrends and https://github.com/w3c/wpt-pullresults/pull/40 is the PR to allow the site to use an external DB (RDS in this case)

foolip commented 6 years ago

Is there a public dashboard for that uptrends monitoring? Would be interesting to see if it disagrees in any way with https://bit.ly/ecosystem-infra-status, which I'd kind of expect since status cake has a 15 minute ping interval.