Can you go into more detail about the --threads option? What does "experimental" mean?

broadinstitute / pilon

Pilon is an automated genome assembly improvement and variant detection tool

GNU General Public License v2.0

342 stars 60 forks source link

Can you go into more detail about the --threads option? What does "experimental" mean? #127

Open lfaller-zymergen opened 4 years ago

lfaller-zymergen commented 4 years ago

I am trying to speed up pilon and am wondering if increasing that value will be a viable path forward.

In the documentation, it says the following:

--threads
          Degree of parallelism to use for certain processing (default 1). Experimental.

Does "experimental" mean it's not stable? What does "certain processing" refer to?

Thanks for any additional information!

w1bw commented 3 years ago

It processes certain scaffolds (or chunks of scaffolds) in parallel with this option. However, it is a pretty crude mechanism which doesn't provide much speedup, and it really causes the garbage collector to kick in. Also, because of changes to the scala language and runtime, I will be removing support for --threads it in the next release; I might revisit it in the future.

fergsc commented 3 years ago

Does that mean we are best to run Pilon on a sinlge thread for a week? That is not possible for many of us using HPCs.