Error: Process completed with exit code 143. (on linux64 Github Actions)

[valeriupredoi]

Hey guys, we're seeing this rather non-descriptive fail - see the failed Create conda-lock file for linux-64 step in our GA flow, and am not 100% sure what's the underlying cause. To note that after moving to conda=23.3.1 things run fine locally, but not on the CI - I guess the bottom line here is if you guys know what the error code means for you, so I can get a clue about what to look for on the CI machine/stack, if not I'll just close this and poke around the CI regardless. Very many thanks in advance :beer:

[maresb]

Hmm, it's exiting with code 143 which is a Linux exit code indicating that the process quit after receiving a SIGTERM signal. The question is then where does this SIGTERM come from? I'm wondering if the solver could be hitting a memory limit which is triggering a SIGTERM from the kernel's Out-Of-Memory Killer.

[valeriupredoi]

@maresb very cool indeed, cheers! I wasn't sure if that was an internal code from conda-lock or a system one - it could well be it's a memory issue (allowed max memory is pretty scant on the GA machines), I'll poke around and see - do you want me to close this not to clutter your spec? I can post here once I find out more, if that'd help you in any way :+1:

[maresb]

Hi @valeriupredoi, feel free to keep this open for now until you learn more info. I'm not sure how to triage this best. It's unfortunate if the memory on a typical runner is being exceeded. Maybe it's a memory leak, or maybe it's that conda-forge has been growing so much. Whatever the cause, I hope we can mitigate it. Although my best guess is that the memory usage of conda-lock itself is dominated by that of the mamba subprocess.

[valeriupredoi]

@maresb cheers, mate, I'll be on to this next week, apologies for replying late, am out to a conference this week, and things are bit hectic around here. But will defo update this issue once I figure out things 🍺

[valeriupredoi]

@maresb my apologies for the radio silence - conference done but managed to catch the blithering Covid from there so I had take it easy while I battled (and won) the virus :grin: Good news - I tested the configuration and conda-lock installed with mamba is not doing anything bad: with Python=3.10 it needs about 0.8GB of memory, very similar to Python=3.11 as well. But the solution for me is to to install it with mamba and use Mambaforge as deps channel - that does the trick, lock file gets generated in no time, and all is fine and dandy. The previous issue was with a conda installation, and not only the lock file would take ages to create, but also I was getting this internal error - quite possibly due to high memory consumption since I don't think mamba was kicking in and the solver was still conda's inefficient solver. Quite happy - and very many thanks for you communication and support :beers:

[valeriupredoi]

BTW here's the workflow file I am using now, if it's useful for any of your users https://github.com/ESMValGroup/ESMValTool/actions/runs/4938066725/jobs/8827374040?pr=3174

[maresb]

Hey, congrats @valeriupredoi on surviving Covid! It's really remarkable how differently it affects different people, so good to hear that you're already victorious.

Thanks a lot for the detailed report. This will surely be helpful to other users.

Thanks also for the workflow; coincidentally, I need to build something similar, like today, so this is really perfect timing. :grin:

[valeriupredoi]

ah cheers muchly, mate! Yeah, this virus is bizarre - fortunately got over it (the second time, first time for me, yet unofficially proven, was in December 2019 - I must have been one of the first cases in the UK haha). Glad that helped! And keep up the awesome work on conda-lock - let me know anytime if I can help any way :beer: