ESMValGroup / ESMValTool

ESMValTool: A community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
https://www.esmvaltool.org
Apache License 2.0
217 stars 127 forks source link

[Github Actions] install git in OSX and add environment inspection #3581

Closed valeriupredoi closed 5 months ago

valeriupredoi commented 5 months ago

Description

It appears that the lack of git on OSX machines is now chronic and getting critical, so may as well plop an install; I am also adding calls to conda list to inspect the environment in light of the new mamba not printing out to screen what it installs.


Before you get started

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the ๐Ÿ›  Technical or ๐Ÿงช Scientific review.

schlunma commented 5 months ago

Why is this only a problem for some OSX tests? Shouldn't the CI machines use a common configuration? I.e., why is git missing on only some of them?

valeriupredoi commented 5 months ago

that's a very good question, Manu! OSX machines are different than the regular Linux ones - usually lower specc-ed, and generally annoying (but that's just me complaining about Apple stuff haha). The node software list does indeed contain git 2.44 in the Utilities list BUT that's an arm64 node, and condamamba don't like that and get their own osx-64 packages, probably git gets lost somewhere between system install and condamamba. Check out Set up job of a job that passes vs Set up job of a job that fails and you see exactly the same OS parameters, docker container ID, signature etc, but git clearly is not in the condamamba environment

valeriupredoi commented 5 months ago

also, frankly - I am usually happier with a git installed from conda-forge than a system git - system git can always be tricky (either too old, or too new, or not working if one is on a Rocky 9 etc etc)

schlunma commented 5 months ago

Are we 100% that this error is cause by a missing git? The error message doesn't really say that, right? I would find it really weird if the CI machines do not include the advertised software...

valeriupredoi commented 5 months ago

Manu - good points! I realized I wrote my comments above with me feet and thought with me arse, here's an improved pitch:

valeriupredoi commented 5 months ago

on a related topic - we should most definitely include git in our direct dependencies - we have gitpython via some dep, but no git as it's needed by scm - @bouweandela would you agree?

mo-gill commented 5 months ago

Hi, we are currently working on the ESMValTool release for v2.11.0. We're wondering if you'd be able to complete this PR by the end of next week (Friday 10th May).

Otherwise, please let us know, and we'll move it into the next milestone for you :slightly_smiling_face:

valeriupredoi commented 5 months ago

@mo-gill cheers - we'll aim to get this in asap - wink, Manu, wink :grin:

valeriupredoi commented 5 months ago

I honestly still don't understand how git can be missing on some machines, but that shouldn't hurt

no it doesn't miss - it's a mangled paths problem - most probably Mac-related guff - the system git doesn't get seen by setuptools_scm in fact it's not available in the condamamba environment where the scm Subprocess is calling it. Beats me what the actual issue is - not having a Mac to test is not helping. But, alas, this will not hurt - if at all, it may even be better to use the latest git, the system one is pretty old. Thanks a lot for the review, Manu! @bouweandela are you aware of setuptools_scm involking git via a Subprocess that calls the git executable? I've only seen this in the OSX case, the root cause of this issue here