Open timothyfrankdavies opened 7 months ago
Currently we use os.cpu_count() in wise_wsol.py to decide how many batches of jobs to launch. It's also used in some debug output.
os.cpu_count()
wise_wsol.py
That returns the number of CPU cores on hardware, but some job scheduling tools like Slurm limit the number of CPUs available to each process.
I tried using len(os.sched_getaffinity(0)), which worked on OzStar, but crashes on macOS.
len(os.sched_getaffinity(0))
There are a few changes that may be worth making:
os.sched_getaffinity
And at the same time, it might be worth letting multithread be set by command line instead of/as well as the json configs.
multithread
For each of these options there's some decisions to be made on where to put the helper function, and when to call it.
All in all not too much work, but too much for #32.
Currently we use
os.cpu_count()
inwise_wsol.py
to decide how many batches of jobs to launch. It's also used in some debug output.That returns the number of CPU cores on hardware, but some job scheduling tools like Slurm limit the number of CPUs available to each process.
I tried using
len(os.sched_getaffinity(0))
, which worked on OzStar, but crashes on macOS.There are a few changes that may be worth making:
len(os.sched_getaffinity(0))
toos.cpu_count()
os.sched_getaffinity
on Unix platforms.And at the same time, it might be worth letting
multithread
be set by command line instead of/as well as the json configs.For each of these options there's some decisions to be made on where to put the helper function, and when to call it.
All in all not too much work, but too much for #32.