ATTPC / Spyral

A Python analysis library for AT-TPC data
GNU General Public License v3.0
2 stars 2 forks source link

Disable implicit multithreading in libraries #195

Closed gwm17 closed 1 week ago

gwm17 commented 3 weeks ago

Libraries like numpy, scipy, and polars utilize tools like BLAS or rayon which are often implicitly multithreaded for things like matrix decomposition. However this poses an issue when used in the context of a multiprocessing environment like we use, where each Spyral process might now try to spawn its own independent BLAS threads and you'll quickly run out of resources. We can use tools like environment variables or threadpoolctl to disable these features.

gwm17 commented 3 weeks ago

threadpoolctl is out. Only works as a context guard (we'd have to apply the context everywhere multithreading is possible). Env variables is is

gwm17 commented 1 week ago

Done in 0.13.0