dftlibs / numgrid

Numerical integration grid for molecules.
Mozilla Public License 2.0
47 stars 14 forks source link

How to specify number of cores for parallel grid generation? #56

Closed manassharma07 closed 1 month ago

manassharma07 commented 1 year ago

In the readme it is mentioned that

The Becke partitioning step is parallelized using Rayon. In other words, this step should be able to use all available cores on the computer or computing node. Since grids are currently generated atom by atom, it is also possible to parallelize "outside" by the caller.

Indeed when I run the code, the program has 1600% CPU usage on my 8 core machine, indicating that it is trying to use all the threads possible.

Such a behavior is not very desirable as first of all, parallelizing over 16 threads on an 8 core machine is not very efficient as in my experience. Furthermore, this very problematic when submitting jobs as you need to specify a number of cores and this leads to oversubscription.

manassharma07 commented 1 year ago

Well actually, I found the solution as soon as I posted. I just googled about how to limit the number of threads used by RAYON, and the official FAQ (https://github.com/rayon-rs/rayon/blob/master/FAQ.md) has the solution.

If you want to alter the number of threads spawned, you can set the environmental variable RAYON_NUM_THREADS to the desired number of threads or use the ThreadPoolBuilder::build_global function method

So I just needed to do export RAYON_NUM_THREADS=4

bast commented 1 year ago

Thanks for reporting! I am reopening because I would like to add your solution to the README and will close as soon as I do that.