search

rajanil / msCentipede

A hierarchical multiscale model for inferring transcription factor binding from chromatin accessibility data.
MIT License
25 stars 6 forks source link

limit the number of CPU cores / threads #16

Open balwierz opened 7 years ago

balwierz commented 7 years ago

Hello,

msCentipede seems to use up to 32 CPU cores in my setup. There should be an option to limit the number of cores, so that the code would behave well when run on a cluster or on a multi-user machine.

rdalbanus commented 5 years ago

Same issue here. For future reference, I've looked at the source code and found that multi-threading was coming from numpy. Running export OPENBLAS_NUM_THREADS=1 and export MKL_NUM_THREADS=1 before submitting jobs solved this issue in my server. Another option is to directly modify call_binding.py to add a line with os.environ[VAR] = '1', where VAR is a string with the openBLAS or MKL variable above. If understood correctly, numpy may invoke other libraries on different systems, so make sure to check the links below in case this solution doesn't work.

Further reading: [1] https://stackoverflow.com/questions/17053671/python-how-do-you-stop-numpy-from-multithreading [2] The Use parallel primitives section of Parallel Programming with numpy and scipy