search

mkandes / galyleo

A shell utility to help you launch Jupyter notebooks on high-performance computing systems in a simple, secure way.
36 stars 7 forks source link

Running in debug queue needs specifying amount of memory #14

Closed zonca closed 2 years ago

zonca commented 2 years ago

the sample script you provide in the README works fine even if I do not specify memory for compute and for shared. Instead if I use debug:

galyleo launch --account sds166 --partition debug --cpus 1 --time-limit 00:30:00 --env-modules cpu,gcc,anaconda3

I get:

sbatch: error: Batch job submission failed: Requested node configuration is not available

workaround is to always specify --memory

mkandes commented 2 years ago

@zonca - This is more an Expanse Slurm configuration issue with the 'debug' partition than a galyleo one. In general, I did not specify a default memory setting for galyleo since each system we use it on had their own default setting via the scheduler configuration.

mkandes commented 2 years ago

@zonca - I've added some system-wide variables that can be set in the galyleo.conf file moving forward. This includes a default memory setting, if not explicitly provided by user. As such, this should resolve the problem with the 'debug' partition on Expanse once I deploy the new version on galyleo to the system. I'm trying to get a few more issues resolved before deploying the new version, but it should be by December 1st --- that is my plan anyway.

See https://github.com/mkandes/galyleo/issues/3 for fix.