The parallelism in math enabled by PYGAMA_PARALLEL=True by default creates as many threads as possible. This has a couple of problems. First, it can slow things down in some situations due to overhead associated with moving things to threads. Second, NERSC doesn't want us doing this on login/jupyter nodes.
For now the solution will be to default it to False. However, in the future, we should add a way to control the number of threads used (perhaps replace PYGAMA_PARALLEL with PYGAMA_NTRHEADS). This can be controlled in numba using numba.set_num_threads() as discussed here: https://numba.readthedocs.io/en/stable/user/threading-layer.html. For default behavior we have a couple of options:
1) Default to threading off. The disadvantage of this approach is that it means people will be testing their code single-threaded and may miss bugs. For this reason, we would want to make sure tests still ran multi-threaded in some instances
2) Choose a smaller number of threads as the default.
The parallelism in math enabled by
PYGAMA_PARALLEL=Trueby default creates as many threads as possible. This has a couple of problems. First, it can slow things down in some situations due to overhead associated with moving things to threads. Second, NERSC doesn't want us doing this on login/jupyter nodes.For now the solution will be to default it to False. However, in the future, we should add a way to control the number of threads used (perhaps replace
PYGAMA_PARALLELwithPYGAMA_NTRHEADS). This can be controlled in numba usingnumba.set_num_threads()as discussed here: https://numba.readthedocs.io/en/stable/user/threading-layer.html. For default behavior we have a couple of options: 1) Default to threading off. The disadvantage of this approach is that it means people will be testing their code single-threaded and may miss bugs. For this reason, we would want to make sure tests still ran multi-threaded in some instances 2) Choose a smaller number of threads as the default.