search

nextflow-io / nextflow

A DSL for data-driven computational pipelines
http://nextflow.io
Apache License 2.0
2.76k stars 630 forks source link

Add ability to log each command run #108

Closed tomsitter closed 8 years ago

tomsitter commented 8 years ago

It would be helpful if nextflow could create a log file of every command run similar to snakemake or ruffus.

For example, snakemake can print out a "dry run" of a workflow

> snakemake -np
rule bwa_map:
        input: data/genome.fa, data/samples/A.fastq
        output: mapped_reads/A.bam
bwa mem data/genome.fa data/samples/A.fastq | samtools view -Sb - > mapped_reads/A.bam
rule samtools_sort:
        input: mapped_reads/A.bam
        output: sorted_reads/A.bam
samtools sort -T sorted_reads/A -O bam mapped_reads/A.bam > sorted_reads/A.bam
rule samtools_index:
        input: sorted_reads/A.bam
        output: sorted_reads/A.bam.bai
samtools index sorted_reads/A.bam
rule report:
        input: calls/all.vcf
        output: report.html
localrule all:
        input: report.html
Job counts:
        count   jobs
        1       all
        1       bcftools_call
        1       bwa_map
        1       report
        1       samtools_index
        1       samtools_sort
        6

Even an ability to automatically traverse the work/ directory and compile all of the commands after the workflow was executed would be helpful.

Thanks

pditommaso commented 8 years ago

There are several ways this feature could be implemented. For example it could be possible to implement a task completion notification handler, something similar to the workflow onComplete, that would allow the user to create its own report easily.

Another option could be to add a kind of log command to the nextflow command line that would print to the stdout the tasks execution directories. Then the user should process them to read the command script.

pditommaso commented 8 years ago

This is available in version 0.22.0-RC1

santiagorevale commented 7 years ago

Hi NextFlow team! @pditommaso

I understand that this enhancement has been implemented. However, I was unable to find any documentation on how to implement it (log the actual command of each process). Could you give me an example on how to do that?

Thank you very much in advance!

Cheers

pditommaso commented 7 years ago

Unfortunately it hasn't documented yet, but you can find a brief description here.

Feel free to comment on that thread for more details.

santiagorevale commented 7 years ago

Hi there! That was it! Thanks!