search

ESMValGroup / ESMValTool

ESMValTool: A community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
https://www.esmvaltool.org
Apache License 2.0
227 stars 128 forks source link

Try to fix failure of conda-lock in CI #3773

Closed maresb closed 1 month ago

maresb commented 1 month ago

I can trigger workflow dispatch from my fork, but I'm getting a strange Bash error. I don't have any more time at the moment, but please try with mamba<2 and see if that solves it. If not we'll go deeper.

valeriupredoi commented 1 month ago

Hi Ben, many thanks for dropping by! I am not sure you are allowed to run our GAs off a fork PR, but I ported your changes in #3771 - try triggering the GAs in your fork too, but for them to run you'll have to delete the Mambaforge line since they are serving us a brownout today to make us move to Miniforge3 (which I am right now) :beer:

valeriupredoi commented 1 month ago

I can trigger workflow dispatch from my fork, but I'm getting a strange Bash error. I don't have any more time at the moment, but please try with mamba<2 and see if that solves it. If not we'll go deeper.

ah I was just typing meself, many thanks! I'll take it from 'ere, mate :beer:

valeriupredoi commented 1 month ago

ported this off-fork in regular PR #3771 - great idea! :beer:

maresb commented 1 month ago

Thanks for putting me out of my misery here, I was wondering why everything was failing.

you'll have to delete the Mambaforge line since they are serving us a brownout today

Ah, I was unaware that Mambaforge was deprecated. I would rather recommend migrating to setup-micromamba, at least once they get these v2 issues sorted. This is the only time I've had a bad experience with it.

I want to release a new version of conda-lock, but I decided to postpone due to issues with mamba v2, and not wanting to add any more variables into the mix before that's all stabilized. I'd be interested in making a MWE out of this and associating it to one of the open issues in mamba-org/mamba (or creating a new one in the unlikely event that this is not yet reported). So it would be good to get something under my control to run.

valeriupredoi commented 1 month ago

yeah I postponed retiring Mambaforge until someone told me about the brownouts today, when it was clear I have to act :hammer: We've been using conda-incubator/setup-miniconda for a long while, am quite happy with that action, stable and with good support (eg the guy who pointed me to those brownouts being announced is from conda-incubator

maresb commented 1 month ago

Actually, even better than Mamba is pixi, a new project-centric Conda environment manager with built-in lockfile support. It's a lot more stable than conda-lock, especially for mixed conda/pip environments.

maresb commented 1 month ago

@valeriupredoi, I wasn't able to reproduce the failure locally. Could I please ask a favor from you?

Could you reproduce the KeyError, and but with the following modifications?

valeriupredoi commented 1 month ago

Sure thing! Will do that tomorrow, am off home for the day, mate 👍

valeriupredoi commented 1 month ago

@valeriupredoi, I wasn't able to reproduce the failure locally. Could I please ask a favor from you?

Could you reproduce the KeyError, and but with the following modifications?

* Just before running `conda-lock` could you run `mamba env export`

* Inside the conda-lock command could you add `--log-level=DEBUG`

Ben, see https://github.com/ESMValGroup/ESMValTool/pull/3780 (sorry, took me a while, emergency getting us rid of Mambaforge)