Closed manassharma07 closed 1 month ago
Well actually, I found the solution as soon as I posted. I just googled about how to limit the number of threads used by RAYON, and the official FAQ (https://github.com/rayon-rs/rayon/blob/master/FAQ.md) has the solution.
If you want to alter the number of threads spawned, you can set the environmental variable RAYON_NUM_THREADS to the desired number of threads or use the ThreadPoolBuilder::build_global function method
So I just needed to do
export RAYON_NUM_THREADS=4
Thanks for reporting! I am reopening because I would like to add your solution to the README and will close as soon as I do that.
In the readme it is mentioned that
Indeed when I run the code, the program has 1600% CPU usage on my 8 core machine, indicating that it is trying to use all the threads possible.
Such a behavior is not very desirable as first of all, parallelizing over 16 threads on an 8 core machine is not very efficient as in my experience. Furthermore, this very problematic when submitting jobs as you need to specify a number of cores and this leads to oversubscription.