Open Microbion opened 1 year ago
I think you have to ask the phage prediction tool developer directly as I cannot adjust the code of these tools
Are these commands you would use in a bash terminal before executing e.g. deepvirfinder? export MKL_NUM_THREADS=1
If yes I think its possible then
I modified line 10 in workflows/process/deepvirfinder/deepvirfinder.wf, by OMP_NUM_THREADS=${task.cpus} dvf.py -c ${task.cpus} -i ${fasta} -o ${name}
, and it worked.
Consider of some tools' can't limit resource by itself or can but not correctly (not just deepvirfinder, I meeted similar troubles in other tools), I think it's necessary to limit resource manurally to improve robustness of workflow.
nice! by chance do you know which tools have this issue?
I would update the code accordingly and create a new release candidate
--max_cores parameter can't limit deepvirfinder, because the theano backend to keras automatically uses all available cores for the internal matrix operations. as mentioned here https://github.com/jessieren/DeepVirFinder/issues/4 I wonder other tools have the similar issue, such as a numpy issue mentioned here https://stackoverflow.com/questions/30791550/limit-number-of-threads-in-numpy. Is it necessary and possible to add some environment variables for avoid using all cores?