seqeralabs / nf-tower

Nextflow Tower system
https://tower.nf
Mozilla Public License 2.0
146 stars 52 forks source link

Duplicate Logging Message #155

Open apeltzer opened 5 years ago

apeltzer commented 5 years ago

I'm running some tests with tower and the nf-core/rnaseq pipeline and have a lot of these messages in the nextflow log - not sure whether this is a problem with tower or Nextflow and/or intended:

Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (38)
[b7/0568c4] process > get_software_versions                                       [100%] 1 of 1 ✔
[25/a4f77e] process > makeBED12 (genes.gtf)                                       [100%] 1 of 1 ✔
[65/4def0b] process > makeHisatSplicesites (genes.gtf)                            [100%] 1 of 1 ✔

This is just an example, I typically see (depending on the pipeline) dozens of these entries. Running on OSX with test,docker and newest Edge Release of NXF.

pditommaso commented 5 years ago

Ouch, the log is printed regularly without tower? may it be possible there's some println in the pipeline script breaking the ANSI rendering?

apeltzer commented 5 years ago

Yes, without tower it's not printed too often but I can check tomorrow morning! Will update here tomorrow morning!

pditommaso commented 5 years ago

not too often is suspect .. :)

apeltzer commented 5 years ago

Without tower:

 nextflow run . -profile test,docker --skipQC -resume
N E X T F L O W  ~  version 19.09.0-edge
Launching `./main.nf` [prickly_hopper] - revision: ebcf453221
Prefer GTF over GFF, so ignoring provided GFF in favor of GTF
[2m----------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/rnaseq v1.4dev
----------------------------------------------------
Run Name          : prickly_hopper
Reads             : data/*{1,2}.fastq.gz
Data Type         : Single-End
Gzipped Refs?     : false
Strandedness      : None
Trimming          : 5'R1: 0 / 5'R2: 0 / 3'R1: 0 / 3'R2: 0 / NextSeq Trim: 0
Aligner           : STAR
Fasta Ref         : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genome.fa
GTF Annotation    : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gtf
GFF3 Annotation   : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gff
Biotype GTF field : gene_biotype
Save prefs        : Ref Genome: No / Trimmed FastQ: No / Alignment intermediates: No
Max Resources     : 6 GB memory, 2 cpus, 2d time per job
Container         : docker - nfcore/rnaseq:dev
Output dir        : ./results
Launch dir        : /Users/alexanderpeltzer/IDEA/nf-core/rnaseq
Working dir       : /Users/alexanderpeltzer/IDEA/nf-core/rnaseq/work
Script dir        : /Users/alexanderpeltzer/IDEA/nf-core/rnaseq
User              : alexanderpeltzer
Config Profile    : test,docker
Config Description: Minimal test dataset to check pipeline function
[2m----------------------------------------------------
executor >  local (11)
[7e/475bae] process > get_software_versions     [100%] 1 of 1 ✔
[13/cbd850] process > makeBED12 (genes.gtf)     [100%] 1 of 1 ✔
executor >  local (12)
[7e/475bae] process > get_software_versions     [100%] 1 of 1 ✔
[13/cbd850] process > makeBED12 (genes.gtf)     [100%] 1 of 1 ✔
executor >  local (12)
[7e/475bae] process > get_software_versions     [100%] 1 of 1 ✔
[13/cbd850] process > makeBED12 (genes.gtf)     [100%] 1 of 1 ✔
executor >  local (12)
[7e/475bae] process > get_software_versions     [100%] 1 of 1 ✔
[13/cbd850] process > makeBED12 (genes.gtf)     [100%] 1 of 1 ✔
executor >  local (14)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
[13/cbd850] process > makeBED12 (genes.gtf)                                [100%] 1 of 1 ✔
[f7/0e21c1] process > makeSTARindex (genome.fa)                            [100%] 1 of 1 ✔
[-        ] process > fastqc                                               

With tower:

N E X T F L O W  ~  version 19.09.0-edge
Launching `./main.nf` [astonishing_feynman] - revision: ebcf453221
Prefer GTF over GFF, so ignoring provided GFF in favor of GTF
[2m----------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/rnaseq v1.4dev
----------------------------------------------------
Run Name          : astonishing_feynman
Reads             : data/*{1,2}.fastq.gz
Data Type         : Single-End
Gzipped Refs?     : false
Strandedness      : None
Trimming          : 5'R1: 0 / 5'R2: 0 / 3'R1: 0 / 3'R2: 0 / NextSeq Trim: 0
Aligner           : STAR
Fasta Ref         : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genome.fa
GTF Annotation    : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gtf
GFF3 Annotation   : https://github.com/nf-core/test-datasets/raw/rnaseq/reference/genes.gff
Biotype GTF field : gene_biotype
Save prefs        : Ref Genome: No / Trimmed FastQ: No / Alignment intermediates: No
Max Resources     : 6 GB memory, 2 cpus, 2d time per job
Container         : docker - nfcore/rnaseq:dev
Output dir        : ./results
Launch dir        : /Users/alexanderpeltzer/IDEA/nf-core/rnaseq
Working dir       : /Users/alexanderpeltzer/IDEA/nf-core/rnaseq/work
Script dir        : /Users/alexanderpeltzer/IDEA/nf-core/rnaseq
User              : alexanderpeltzer
Config Profile    : test,docker
Config Description: Minimal test dataset to check pipeline function
[2m----------------------------------------------------
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (12)
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (14)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (14)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (15)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (16)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (18)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (19)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (20)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (20)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (20)
[7e/475bae] process > get_software_versions                                [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (22)
[7e/475bae] process > get_software_versions                                                            [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (22)
[7e/475bae] process > get_software_versions                                                            [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (22)
[7e/475bae] process > get_software_versions                                                            [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (23)
[7e/475bae] process > get_software_versions                                                            [100%] 1 of 1 ✔
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
Monitor the execution with Nextflow Tower using this url https://tower.nf/watch/XXXXXX
executor >  local (23)
[7e/475bae] process > get_software_versions                                                            [100%] 1 of 1 ✔
[a4/f62576] process > makeBED12 (genes.gtf)                                                            [100%] 1 of 1 ✔
[73/65e02d] process > makeSTARindex (genome.fa)                                                        [100%] 1 of 1 ✔
[-        ] process > fastqc                                                                           -
[51/e00082] process > trim_galore (SRR4238379)                                                         [100%] 4 of 4 ✔
[b4/2e0908] process > star (SRR4238355)                                                                [100%] 4 of 4 ✔
[-        ] process > rseqc                                                                            -
[-        ] process > preseq                                                                           -
[-        ] process > markDuplicates                                                                   -
[-        ] process > qualimap                                                                         -
[-        ] process > dupradar                                                                         -
[ed/773669] process > featureCounts (SRR4238355_subsampAlignedByCoord.out)                             [100%] 4 of 4 ✔
pditommaso commented 5 years ago

yeah, but there's something wrong with the pipeline script. It should repeat these blocks:

executor >  local (11)
[7e/475bae] process > get_software_versions     [100%] 1 of 1 ✔
[13/cbd850] process > makeBED12 (genes.gtf)     [100%] 1 of 1 ✔
pditommaso commented 5 years ago

For example this.