search

nextflow-io / nextflow

A DSL for data-driven computational pipelines
http://nextflow.io
Apache License 2.0
2.66k stars 619 forks source link

Pipeline skips modules when running with `-dump-channels` #3528

Closed nvnieuwk closed 1 year ago

nvnieuwk commented 1 year ago

Bug report

Expected behavior and actual behavior

When running my pipeline in a normal setting (stubs for making it run quicker) using nextflow run CenterForMedicalGeneticsGhent/nf-cmgg-germline -r dev -latest -stub -profile test,docker --outdir results. I get the expected trace:

[5f/2d3292] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:COMPOSESTRTABLEFILE (GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set_chr21.fna) [100%] 1 of 1 ✔
[64/8ce4b0] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:INDEX_TO_BED (fasta_bed)                                                              [100%] 1 of 1 ✔
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:SAMTOOLS_MERGE                                               
[45/879655] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:SAMTOOLS_INDEX (DNA049572)                                   [100%] 1 of 1 ✔
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:MERGE_BEDS                                                   
[63/1b7f38] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:CALIBRATEDRAGSTRMODEL (NA24385D2_NVQ_034)                    [100%] 2 of 2 ✔
[bf/17c1de] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:HAPLOTYPECALLER (NA24385D2_NVQ_034_1)                        [100%] 8 of 8 ✔
[f0/f389c0] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:VCF_GATHER_BCFTOOLS:BCFTOOLS_CONCAT (NA24385D2_NVQ_034)      [100%] 2 of 2 ✔
[63/780514] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:VCF_GATHER_BCFTOOLS:TABIX_TABIX (NA24385D2_NVQ_034)          [100%] 2 of 2 ✔
[38/b8dcdf] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:BCFTOOLS_STATS_INDIVIDUALS (NA24385D2_NVQ_034)               [100%] 2 of 2 ✔
[b2/6a5a8a] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:REBLOCKGVCF (NA24385D2_NVQ_034)                              [100%] 2 of 2 ✔
[09/ef54b2] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:MERGE_BEDS (Proband_12345)                                           [100%] 1 of 1 ✔
[ef/5f416a] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:GENOMICSDBIMPORT (Proband_12345)                                     [100%] 1 of 1 ✔
[7b/cd106b] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:GENOTYPE_GVCFS (Proband_12345_1)                                     [100%] 2 of 2 ✔
[6c/349989] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:VCF_GATHER_BCFTOOLS:BCFTOOLS_CONCAT (Proband_12345)                  [100%] 1 of 1 ✔
[8e/637f93] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:VCF_GATHER_BCFTOOLS:BCFTOOLS_SORT (Proband_12345)                    [100%] 1 of 1 ✔
[86/c9a29b] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:VCF_GATHER_BCFTOOLS:TABIX_TABIX (Proband_12345)                      [100%] 1 of 1 ✔
[bd/f75fdb] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:FILTER_SNPS (Proband_12345)                                                           [100%] 1 of 1 ✔
[35/7d0279] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:FILTER_INDELS (Proband_12345)                                                         [100%] 1 of 1 ✔
[21/99b898] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:VCF_EXTRACT_RELATE_SOMALIER:TABIX_TABIX (Proband_12345)                               [100%] 1 of 1 ✔
[fd/f4c9dd] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:VCF_EXTRACT_RELATE_SOMALIER:SOMALIER_EXTRACT (Proband_12345)                          [100%] 1 of 1 ✔
[e2/23e195] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:VCF_EXTRACT_RELATE_SOMALIER:SOMALIER_RELATE (Proband_12345)                           [100%] 1 of 1 ✔
[08/7176f3] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ADD_PED_HEADER:RTGTOOLS_PEDFILTER (Proband_12345)                                     [100%] 1 of 1 ✔
[4d/2275e5] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ADD_PED_HEADER:BCFTOOLS_REHEADER (Proband_12345)                                      [100%] 1 of 1 ✔
[b3/d02c31] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:ENSEMBLVEP (Proband_12345)                                                 [100%] 1 of 1 ✔
[eb/b9a004] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:TABIX_ENSEMBLVEP (Proband_12345)                                           [100%] 1 of 1 ✔
[f1/ea2c15] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:BCFTOOLS_CONCAT (Proband_12345)                                            [100%] 1 of 1 ✔
[90/650a22] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:VCFANNO (Proband_12345)                                                    [100%] 1 of 1 ✔
[b8/5fe6ad] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:BGZIP_ANNOTATED_VCFS (Proband_12345)                                       [100%] 1 of 1 ✔
[18/39dfb9] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:BCFTOOLS_STATS_FAMILY (Proband_12345)                                                 [100%] 1 of 1 ✔
[ff/fefb5c] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:VCF2DB (Proband_12345)                                                                [100%] 1 of 1 ✔
[e5/1ea099] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:CUSTOM_DUMPSOFTWAREVERSIONS (1)                                                       [100%] 1 of 1 ✔
[27/c6038f] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:MULTIQC                                                                               [100%] 1 of 1 ✔

But when I run it with -dump-channels, I get the following trace:

[33/c4ef6a] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:COMPOSESTRTABLEFILE (GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set_chr21.fna) [100%] 1 of 1 ✔
[63/a46df7] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:INDEX_TO_BED (fasta_bed)                                                              [100%] 1 of 1 ✔
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:SAMTOOLS_MERGE                                               -
[5d/5a9e3f] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:SAMTOOLS_INDEX (DNA049572)                                   [100%] 1 of 1 ✔
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:MERGE_BEDS                                                   -
[19/07102e] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:CALIBRATEDRAGSTRMODEL (DNA049572)                            [100%] 1 of 1 ✔
[d7/06f91e] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:HAPLOTYPECALLER (DNA049572_3)                                [100%] 4 of 4 ✔
[09/a67b08] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:VCF_GATHER_BCFTOOLS:BCFTOOLS_CONCAT (DNA049572)              [100%] 1 of 1 ✔
[b0/de6ba4] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:VCF_GATHER_BCFTOOLS:TABIX_TABIX (DNA049572)                  [100%] 1 of 1 ✔
[a3/ec083a] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:BCFTOOLS_STATS_INDIVIDUALS (DNA049572)                       [100%] 1 of 1 ✔
[13/ffe89b] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:GERMLINE_VARIANT_CALLING:REBLOCKGVCF (DNA049572)                                      [100%] 1 of 1 ✔
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:MERGE_BEDS                                                           -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:GENOMICSDBIMPORT                                                     -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:GENOTYPE_GVCFS                                                       -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:VCF_GATHER_BCFTOOLS:BCFTOOLS_CONCAT                                  -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:VCF_GATHER_BCFTOOLS:BCFTOOLS_SORT                                    -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:JOINT_GENOTYPING:VCF_GATHER_BCFTOOLS:TABIX_TABIX                                      -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:FILTER_SNPS                                                                           -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:FILTER_INDELS                                                                         -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:VCF_EXTRACT_RELATE_SOMALIER:TABIX_TABIX                                               -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:VCF_EXTRACT_RELATE_SOMALIER:SOMALIER_EXTRACT                                          -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:VCF_EXTRACT_RELATE_SOMALIER:SOMALIER_RELATE                                           -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ADD_PED_HEADER:RTGTOOLS_PEDFILTER                                                     -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ADD_PED_HEADER:BCFTOOLS_REHEADER                                                      -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:ENSEMBLVEP                                                                 -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:TABIX_ENSEMBLVEP                                                           -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:BCFTOOLS_CONCAT                                                            -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:VCFANNO                                                                    -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:ANNOTATION:BGZIP_ANNOTATED_VCFS                                                       -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:BCFTOOLS_STATS_FAMILY                                                                 -
[-        ] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:VCF2DB                                                                                -
[5e/a3eb42] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:CUSTOM_DUMPSOFTWAREVERSIONS (1)                                                       [100%] 1 of 1 ✔
[a2/3e3e85] process > CMGG_CMGGGERMLINE:CMGGGERMLINE:MULTIQC                                                                               [100%] 1 of 1 ✔

Steps to reproduce the problem

Run following command: nextflow run CenterForMedicalGeneticsGhent/nf-cmgg-germline -r dev -latest -stub -profile test,docker --outdir results -dump-channels input_beds with and without -dump-channels to see the difference. I'll see if I can create a minimal example, but have been unable to do so for now.

Program output

nextflow.log

Environment

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.