search

bxlab / metaWRAP

MetaWRAP - a flexible pipeline for genome-resolved metagenomic data analysis
MIT License
389 stars 189 forks source link

check point function available? #185

Open ganiatgithub opened 5 years ago

ganiatgithub commented 5 years ago

Hello there,

Thank you for the cool program. I ran the assembly function on a server but killed due to not enough walltime after 72 hours. Is it possible to pick up a job from some check points? i.e. == Running assembler: K55

That would be fantastic.

Cheers

ganiatgithub commented 5 years ago

If it helps, my command is:

metaWRAP assembly \
-1 /Path_to_reads/reads_1.fastq \
-2 /Path_to_reads/reads_2.fastq \
-o /Path_to_output/ \
-m 500 \
-t 24 \
--metaspades

Last few lines of the log:

== Running assembler: K55

  0:00:00.000     4M / 4M    INFO    General                 (main.cpp                  :  74)   Loaded config from /30days/uqgni1/jiaoyang/metaWRAP/assembly_out/metaspades/K55/configs/config.info
  0:00:00.000     4M / 4M    INFO    General                 (main.cpp                  :  74)   Loaded config from /30days/uqgni1/jiaoyang/metaWRAP/assembly_out/metaspades/K55/configs/mda_mode.info
  0:00:00.000     4M / 4M    INFO    General                 (main.cpp                  :  74)   Loaded config from /30days/uqg
.........

  3:25:45.269     1G / 13G   INFO   K-mer Index Building     (kmer_index_builder.hpp    : 336)   Index built. Total 1746267506 bytes occupied (3.70972 bits per kmer).
  3:25:47.379     5G / 13G   INFO   DeBruijnExtensionIndexBu (kmer_extension_index_build:  99)   Building k-mer extensions from k+1-mers
  3:28:23.328     5G / 13G   INFO   DeBruijnExtensionIndexBu (kmer_extension_index_build: 103)   Building k-mer extensions from k+1-mers finished.
  3:28:23.329     5G / 13G   INFO    General                 (stage.cpp                 : 101)   PROCEDURE == Condensing graph
  3:28:23.332     5G / 13G   INFO   UnbranchingPathExtractor (debruijn_graph_constructor: 355)   Extracting unbranching paths
ursky commented 5 years ago

For metaSPAdes there are already checkpoints implemented if you are using metawrap v>1.1.5. If the job times out just re-run it on the same output folder and it should continue where it left off.

ganiatgithub commented 5 years ago

It worked, thank you!