Binder services running out of memory

jnywong commented 2 months ago

Context

I keep running into OOM errors when trying to build this environment.yml.

I have tried building this environment some of our available binder services, e.g.

and run into the same error:

Step 39/50 : RUN TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file "binder/environment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '
 ---> Running in 5ef5962be9b5
Channels:
 - conda-forge
 - defaults
Platform: linux-64
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... bash: line 1:     8 Killed                  ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file "binder/environment.yml"
time: 36.818
 ---> Removed intermediate container 5ef5962be9b5
The command '/bin/sh -c TIMEFORMAT='time: %3R' bash -c 'time ${MAMBA_EXE} env update -p ${NB_PYTHON_PREFIX} --file "binder/environment.yml" && time ${MAMBA_EXE} clean --all -f -y && ${MAMBA_EXE} list -p ${NB_PYTHON_PREFIX} '' returned a non-zero code: 137

I've also checked Grafana and I can see pods getting killed :(

Screenshot 2024-07-19 at 11 49 32