search

hpc / charliecloud

Lightweight user-defined software stacks for high-performance computing.
https://hpc.github.io/charliecloud
Apache License 2.0
308 stars 61 forks source link

mksquashfs parallel processing is uncontrolled #1556

Open rarensu opened 1 year ago

rarensu commented 1 year ago

When I do ch-convert to sqfs, mksquashfs defaults to use all the cores on the machine. This is actually just the normal, expected behavior of mkquashfs as far as I know. This is bad because in a shared cluster environment, usually you don't have or even want all the cores on a machine. Singularity had the same issue so they implemented an environment variable that would get passed to mksquashfs as a command line option to limit the number of cores it would use. I'm not saying that's the correct solution, I'm just saying the problem is previously known.

reidpr commented 4 months ago

Brief search reveals that -processors was available at least as far back as SquashFS-Tools 4.2 (in Debian Jessie), released 2011. So we can probably just use it unconditionally.