uclahs-cds / pipeline-call-gSV

Nextflow pipeline to call germline structural variants and copy number variants using DELLY and Manta
https://uclahs-cds.github.io/pipeline-call-gSV/
GNU General Public License v2.0
2 stars 1 forks source link

Use `methods.setup_process_afterscript()` for process logs #157

Closed nwiltsie closed 3 months ago

nwiltsie commented 4 months ago

We should incorporate the new methods.setup_process_afterscript() from https://github.com/uclahs-cds/pipeline-Nextflow-config/pull/64 into this pipeline.

Under the new method, log files are automatically saved to ${params.log_output_dir}/process-log/${task.ext.log_dir}${task.ext.log_dir_suffix}/ (no need to specify a publishDir directive).

${task.ext.log_dir} defaults to ${task.process.replace(':', '/')}. ${task.ext.log_dir_suffix} defaults to a blank string.

The key steps are:

  1. Add methods.setup_process_afterscript() to methods.setup().
  2. For each Nextflow process:
    1. Remove the file ".command.*" output directive.
    2. Add ext log_dir { ... } and ext log_sur_suffix { ... } directives, if appropriate.
    3. Remove the publishDir directive with pattern: ".command.*".

As an example, this publishDir directive... https://github.com/uclahs-cds/pipeline-call-gSV/blob/994f46de6c347999dfe2afb5fc1c5d271ef881d8/module/sha512.nf#L18-L21

... should be replaced with this directive:

ext log_dir { "${task.process}/${task.process}-${task.index}" }

(Just a note, most of our other pipelines use "${task.process.replace(':', '/')}" - I don't know if that `:`` replacement is relevant here.)

See https://github.com/uclahs-cds/pipeline-recalibrate-BAM/pull/76/files for a full example.