search

ESMValGroup / ESMValCore

ESMValCore: A community tool for pre-processing data from Earth system models in CMIP and running analysis scripts.
https://www.esmvaltool.org
Apache License 2.0
42 stars 38 forks source link

Test fail if CEDA ESGF archive is down #1637

Closed valeriupredoi closed 2 years ago

valeriupredoi commented 2 years ago

See test fails - all of them and their dog have died when CEDA archive was down a couple days ago - I thought we fixed this behaviour and got the rerouting done if one ESGF archive was down, I might be slightly out the loop if this was done, but clearly is still causing total test failures. @bouweandela what you reckon? :beer:

zklaus commented 2 years ago

To be clear, in this case this was a planned maintenance at CEDA. I saw the fails and double-checked this. It did make me wonder: Should we have some kind of status page or log of past CI failures?

valeriupredoi commented 2 years ago

good clarification, Klaus - indeed, Jassie eloped that day :grin: And a very good point about the log of past fails too, I'll look into setting that up!

bouweandela commented 2 years ago

Should we have some kind of status page or log of past CI failures?

It is available here: https://app.circleci.com/insights/github/ESMValGroup/ESMValCore/workflows/nightly/overview?branch=main&reporting-window=last-90-days

bouweandela commented 2 years ago

The originally reported problem was fixed in #1561, but I suspect that there is a different exception now. I'll make a new pull request to catch that one too.

valeriupredoi commented 2 years ago

Should we have some kind of status page or log of past CI failures?

It is available here: https://app.circleci.com/insights/github/ESMValGroup/ESMValCore/workflows/nightly/overview?branch=main&reporting-window=last-90-days

thanks, Bouwe! That is returning a Server Error for me at the mo, but also I'd be looking into a log of the GA tests - those are a bit harder to track after they've failed since one needs to scroll through a lot more test results (since they are 8-10x more numerous)

bouweandela commented 2 years ago

Maybe you need to sign in on CircleCI to see the test results? You can sign in with your GitHub account.

valeriupredoi commented 2 years ago

I did manage to get through - the Server Error is intermittent though - Circle may be having a bad hair day, but those metrics do look good and helpful; I'll see if GA offers something similar, maybe we should add those links somewhere in our docs?

bouweandela commented 2 years ago

maybe we should add those links somewhere in our docs?

Pull requests improving the docs are always very welcome ;-)

valeriupredoi commented 2 years ago

so I've set this up https://meercode.io with ESMValCore's Github Actions as Integration (it has many more CI integrations available) - it's pretty nifty - see attached example screenshot - but this is set up as a user so I am not sure how we can integrate it inside our organization. Just a pointer if you guys want to have a look at it GA-Dashboard