Jobs exceed allocated memory - Githubissues

jupyterhub / batchspawner

Custom Spawner for Jupyterhub to start servers in batch scheduled systems

BSD 3-Clause "New" or "Revised" License

190 stars 134 forks source link

Jobs exceed allocated memory #201

Closed statiksof closed 3 years ago

statiksof commented 3 years ago

Bug description

Batchspawner jobs exceed allocated memory until available node's memory.

Added srun didn't change the behavior. cgroups not available on the cluster at the moment.

Expected behaviour

Slurm kills corresponding job.

Actual behaviour

Batchspawner jobs continue running until reaching available memory on the allocated node.
Kernel restarts
No significant error message from Slurm/Batchspawner

How to reproduce

start a jupyterhub session
allocate more memory than allocated (using simple numpy script)

Your personal set up

Jupyterhub: 1.1.0
Batchspawner: 1.0.1

Any thoughts on this?