Failing CI after Release 6

pgierz commented 2 years ago

Hello,

This issue is for discussion and does not really report a "problem", but maybe something we should be aware of. The following models and components are currently failing the compile tests. In some cases, it is because my CI dummy account doesn't have access everywhere, in others, I am a bit confused:

Full Setups

AWIESM

[x] AWIESM 2.1: Could not read username (but why then does it work below?)
[x] AWIESM 2.1 Wiso: As above
[x] AWIESM 2.1 Recom: As above

Standalone Components

ECHAM

[ ] 6.3.04p1-esm_interface: Tries to compile as standalone, but probably is not a standalone valid component
[ ] 6.3.04p1-recom-awicm: As above
[ ] 6.3.05p2-concurrent_radiation: Can't tell yet why
[x] 6.3.05p2-concurrent_radiation-paleodyn: Can't tell yet why
[ ] 6.3.05p2-foci: Permission error of git clone, and not a standalone model
[x] 6.3.05p2-wiso: No Username for "https://gitlab.awi.de". That's from the same repository as the paleodyn variant, so, why is it able to download one and not the other?

PISM:

[x] Snowflake 0.7: No longer exists after swrepo.awi.de retired
[x] Snowflake 1.0: As above

FESOM

[ ] fesom-1.4-recom-awicm: Not a standalone model
[ ] fesom-1.4-recom-mocsy-slp: Not a standalone model
[ ] fesom-2.0-jio: FORTRAN internal error?
[ ] fesom-2.0-o: Error during get phase (permission denied?)
[ ] fesom-2.0-paleodyn: Fortran interal error (or, model not designed as standalone?)

General ideas

One idea. The following:

python/2020.2.902/lib/python3.7/subprocess.py", line 512, in run
119
    output=stdout, stderr=stderr)
120
subprocess.CalledProcessError: Command '['git', 'clone', '-b', 'oifs-deck', 'https://github.com/FESOM/fesom2.git', 'fesom-2.0-o']' returned non-zero exit status 128.
121
Error: Process completed with exit code 1.

is super unhelpful. It just says "ok, you got back first a 128 and then a 1". I have absolutely no way of seeing what didn't work without more specific info. I guess we should make esm-master spit out something more sensible.

pgierz commented 2 years ago

Here are also links to the various CI jobs:

AWIESM: https://github.com/esm-tools/esm_tools/actions/runs/1591738747

ECHAM: https://github.com/esm-tools/esm_tools/actions/runs/1591738748

PISM: https://github.com/esm-tools/esm_tools/actions/runs/1591738749

FESOM: https://github.com/esm-tools/esm_tools/runs/4559260911

mandresm commented 2 years ago

AWIESM

Many of the AWIESM versions use an ECHAM version that is retrieved from the gitlab.awi instead of the gitlab.dkrz, and when you manually install, you not only need to provide the password for awi (as you'll do for dkrz), but also your awi user (ESM-Tools only asks you for your dkrz user during installation and it never saves your awi-user).

pgierz commented 2 years ago

I think we should at some point entirely remove this esmtoolsrc file and get everything from env variables if (and only if) it does not exist. @seb-wahl will hopefully approve.

I need to test by hand, but I have the feeling that something during git clone messes up with tokens as a replacement for the password. The failing AWI-ESM test never happens to anyone in real life, but I think it would still be good to have it automatically.

nwieters commented 1 year ago

Hi @pgierz, was this only an issue on ollie? Or does it also apply to other machines and needs to kept open?

pgierz commented 1 year ago

Can be closed :-)

esm-tools / esm_tools