search

bokulich-lab / q2-assembly

QIIME 2 plugin for (meta)genome assembly.
BSD 3-Clause "New" or "Revised" License
4 stars 12 forks source link

default values for number of CPU threads #29

Closed gregcaporaso closed 1 year ago

gregcaporaso commented 1 year ago

In doing some experiments with q2-assembly, I noticed that the default number of CPUs is set a few different ways. For example:

qiime assembly assemble-megahit: --p-num-cpu-threads Number of CPU threads. Default: # of logical processors. qiime assembly assemble-spades: --p-threads Number of threads. Default: 16.

In the core distribution QIIME 2 plugins, we tend to set these types of values with a default of 1, forcing the user to intentionally request more resources. This is because users will often just go with the default setting. If it's set to 1, they'll notice that it's too slow and increase the value. If it's set high by default though, it becomes easy for users to not notice and overload a system. For example, if they request a single CPU on their cluster, and then 16 subprocesses spin up, that can overload the cluster node and get them in trouble with the sys admin (and potentially give QIIME 2 a bad reputation with the sys admin if it happens regularly).

I recommend always setting the defaults for these parameters to 1, and letting the user override them.

misialq commented 1 year ago

Hey @gregcaporaso, thanks for pointing that out - will adapt!