Closed atrettin closed 3 years ago
Would be good to throw a warning in __init__.py
for when users set PISA_NUM_THREADS
> 1, but PISA_TARGET
= cpu
otherwise looks to be working correctly
Would be good to throw a warning in
__init__.py
for when users setPISA_NUM_THREADS
> 1, butPISA_TARGET
= cpu
Okay, done!
This PR allows one to set the number of threads to be used for multithreading with one central environment variable
PISA_NUM_THREADS
. This variable is used bynumba
as well as any "custom" multithreading implementation (of which there is currently only one, for fast histograms).Multithreading can be made much more simple than we used to: All one needs to do is to implement things with simple for-loops and then use numba's
prange
instead ofrange
in for-loops that can be naively parallelized. This is done for themceq_barr_red
stage as well as the fast lookup procedures. I would recommend going this route as well forprob3
, but I don't think that I'm going to do this because I'm not using it myself. I had to remove thetarget
keyword in some places because it no longer works.nuSQuIDS already support multithreading for the most expensive operation of propagating state densities through matter potentials, and here, too, the
PISA_NUM_THREADS
variable is used.Multithreading for fast histograms is possible because the Global Interpreter Lock is released during the main loop, allowing Python to continue during the calculation. The input sample is simply split up in chunks and the histogram is calculated for every chunk, and the outputs are added together. The speedup is not entirely a factor of N for N cores, but there is benefit in it.