carstenbauer
commented
1 year ago SLURM_JOB_IDfor checking if we are in a job/allocation.SLURM_CPU_BIND_LIST=0x000000000000000000000000FFFFFFFForSLURM_CPU_BIND=verbose,mask_cpu:0x000000000000000000000000FFFFFFFFfor the mask.
During the NHR course week a participant got an unexpected result when running
pinthreads(:numa)inside of SLURM allocation that didn't comprise an entire compute node (128 cores) but only a fraction of it (32 cores). In this case, SLURM (rightfully) restricted the access to the first 32 cores. This led to the situation thatpinthreads(:numa), while not showing any error or similar, was actually only pinning to the first two NUMA domains (corresponding to the first 32 cores).Maybe this is fine. But we could also think about adding "SLURM awareness" in some form. For example, we could check SLURM environment variables to figure out A) if we are running within a SLURM allocation and B) what cores are actually accessible (we should be able to figure this out from the mask). Wrt. TP.jl we could then
threadinfo()only show the accessible cores (maybe grey out the inaccessible ones?)