Track/work around actions/setup-python#682

iiasa / ixmp

The ix modeling platform for integrated and cross-cutting scenario analysis

https://docs.messageix.org/ixmp

Apache License 2.0

37 stars 110 forks source link

Track/work around actions/setup-python#682 #484

Closed khaeru closed 1 year ago

khaeru commented 1 year ago

Due to actions/setup-python#682, we see failures of the "pytest" CI workflow in ixmp (e.g. here) and message_ix (e.g. here).

These occur only on macos-latest for Python 3.7, specifically 3.7.17 as noted upstream.
The cause is that the Python build is missing _bz2, which is imported by pandas.

This issue is to track the upstream one.

As mitigation, we could temporarily modify the job matrix in the CI workflow to add:
```
exclude:
# Work around #XXX / actions/setup-python#682
- {os: macos-latest, python-version: "3.7"}
```
…and then later remove it.
We can also simply tolerate the failure and merge PRs despite it, referencing this issue.
The solution will/should be delivered upstream.

khaeru commented 1 year ago

A fix is apparently made upstream and is being deployed gradually across the network of GHA runners. One of the devs made a suggestion https://github.com/actions/setup-python/issues/682#issuecomment-1604319803 on how to force the refresh, so we could add that to the affected workflows if needed.

glatterf42 commented 1 year ago

Yes, I've seen that, too. We would only need these steps for one case of our matrix workflows, so it might be simpler to wait until the change reaches us naturally. How would we see that, though? The macos-py3.7 tests should pass again, but the fix was apparently published three days ago, so shouldn't the fix have arrived at our tests in the meantime?

khaeru commented 1 year ago

How would we see that, though? The macos-py3.7 tests should pass again

Yes exactly.

the fix was apparently published three days ago, so shouldn't the fix have arrived at our tests in the meantime?

Not necessarily. I guess the runners in their various data centres, and the software on them, updates or replicates on some periodic schedule, maybe there is caching involved, and perhaps this one fix didn't go through a fast/urgent channel or involve a forced cache refresh. It's a big system!

glatterf42 commented 1 year ago

So do you think it's worth it to update the workflows and remove the fix in a week or two (if that is a more appropriate time frame)?

khaeru commented 1 year ago

@glatterf42 noted that the fix made it to ixmp as of 2023-06-29, and we don't see the issue on message_ix any more, either, as of today. Thus, I'll close this tracking issue.