search

lentendu / DeltaMP

A flexible, reproducible and resource efficient metabarcoding amplicon pipeline for HPC
GNU General Public License v3.0
2 stars 1 forks source link

Checkpointing with only a new database should start at id step #154

Closed lentendu closed 2 years ago

lentendu commented 2 years ago

Not the case yet, starts at trim step as DB_CHOP is used for preclustering with MOTHUR (silva based SOP) Checkpointing should only consider part of the code relevant for the two compared subproject (need to resolve all ifelse in the step scripts)

lentendu commented 2 years ago

One solution would be to "make" (i.e. resolve if-else statements) all the scripts to keep only the relevant part of each at startup, e.g. with: awk '{if(/^\tif/ || /^\tthen/ || /^\telse/ || /^\tfi/){print $0} else {print "echo \""$0"\""}}' Illumina_fastq.sh

However, this would need a significant rewriting of all scripts to avoid trying resolving if else statements involving files created during the pipeline execution, or inside for/while loops. It would be needed to differentiate between generic if-else statements of the whole subproject and if-else statements which can change inside the subproject and would not need to be resolved at that step.