search

apcamargo / genomad

geNomad: Identification of mobile genetic elements
https://portal.nersc.gov/genomad/
Other
168 stars 17 forks source link

Too many threads used in numpy and tensorflow #17

Closed tsp-kucbd closed 1 year ago

tsp-kucbd commented 1 year ago

Currently numpy and tensor flow are not honouring the number of threads given. When restricting genomad to 1 thread with '-t 1', we can see CPU spikes to almost the full amount of CPUs available on the node. An temporary but partly fix is to restrict the number of threads numpy can use via these environment variables:

export OMP_NUM_THREADS=1
export MKL_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
export OPENBLAS_NUM_THREADS=1
export VECLIB_MAXIMUM_THREADS=1
export NUMEXPR_NUM_THREADS=1

But the for tensor flow in nn_classification.py we haven't found any environment variables yet, one might need to restrict tf directly in the python code. (damn you, autocorrect changing numpy to dumpy)

apcamargo commented 1 year ago

Thanks for the report, @tsp-kucbd. I just pushed a new release that allows the user to control the number of threads used for classification in the nn-classification module. Let me know if that solves your issue.