search

yyoshiaki / VIRTUS

A bioinformatics pipeline for viral transcriptome detection and quantification considering splicing.
https://github.com/yyoshiaki/VIRTUS/wiki
Other
33 stars 9 forks source link

Parallel processing #39

Open gailrosen opened 4 months ago

gailrosen commented 4 months ago

Hi,

I have 800 samples to run. I have calculated that the processing may take 50 days on my system. Is there a way to break up the runs into smaller runs and then recalculate the overall statistics later? What do you recommend?

Thanks, GLR

yyoshiaki commented 4 months ago

Hi GLR,

That's a huge project! CWL offers many options for parallelization or specifications for resources; I would recommend looking through the options first. e.g. https://cwltool.readthedocs.io/en/latest/cli.html#cmdoption-cwltool-strict-cpu-limit