search

PoonLab / gromstole

Quantifying SARS-CoV-2 VoCs from NGS data of wastewater samples
MIT License
3 stars 2 forks source link

Implement pipeline from Ewels et al. #41

Closed Abayomi-Olabode closed 2 years ago

Abayomi-Olabode commented 2 years ago

Installed nextflow and moved it to /usr/local/bin. Running pipeline using 2 paired fastq files in ./test_viralrecon/.

Command for Illumina amplicon analysis:

abayomi@Langley:~$ nextflow run nf-core/viralrecon       --input ./test_viralrecon/samplesheet.csv       --platform illumina       --protocol amplicon       --genome 'MN908947.3'       --primer_set artic       --primer_set_version 3       --skip_assembly       -profile docker
N E X T F L O W  ~  version 21.10.6
Launching `nf-core/viralrecon` [jolly_lamport] - revision: 2ebae61442 [master]

------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/viralrecon v2.2
------------------------------------------------------
Core Nextflow options
  revision          : master
  runName           : jolly_lamport
  containerEngine   : docker
  launchDir         : /home/abayomi
  workDir           : /home/abayomi/work
  projectDir        : /home/abayomi/.nextflow/assets/nf-core/viralrecon
  userName          : abayomi
  profile           : docker
  configFiles       : /home/abayomi/.nextflow/assets/nf-core/viralrecon/nextflow.config

Input/output options
  input             : ./test_viralrecon/samplesheet.csv
  platform          : illumina
  protocol          : amplicon

Reference genome options
  genome            : MN908947.3
  fasta             : https://github.com/artic-network/artic-ncov2019/raw/master/primer_schemes/nCoV-2019/V3/nCoV-2019.reference.fasta
  gff               : https://github.com/nf-core/test-datasets/raw/viralrecon/genome/MN908947.3/GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz
  primer_bed        : https://github.com/artic-network/artic-ncov2019/raw/master/primer_schemes/nCoV-2019/V3/nCoV-2019.primer.bed
  primer_set        : artic
  primer_set_version: 3

Illumina de novo assembly options
  skip_assembly     : true

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/viralrecon for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.3901628

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/viralrecon/blob/master/CITATIONS.md
------------------------------------------------------
executor >  local (3)
[39/2d13c1] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)     [  0%] 0 of 1
[e7/8c6f20] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (nCoV-2019.reference.fasta)                        [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:COLLAPSE_PRIMERS                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD                                                      [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                       -
[f2/90d9f7] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                   [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                          -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                     -
executor >  local (3)
[39/2d13c1] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)     [  0%] 0 of 1
[e7/8c6f20] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (nCoV-2019.reference.fasta)                        [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:COLLAPSE_PRIMERS                                                   [  0%] 0 of 1
[97/4886ea] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (nCoV-2019.reference.fasta)                          [100%] 1 of 1, failed: 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                       -
[f2/90d9f7] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                   [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                          -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                     -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:KRAKEN2_KRAKEN2                                                                   -
executor >  local (3)
[39/2d13c1] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)     [100%] 1 of 1 ✔
[e7/8c6f20] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (nCoV-2019.reference.fasta)                        [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:COLLAPSE_PRIMERS                                                   [  0%] 0 of 1
[97/4886ea] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (nCoV-2019.reference.fasta)                          [100%] 1 of 1, failed: 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                       -
[f2/90d9f7] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                   [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                          -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                     -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:KRAKEN2_KRAKEN2                                                                   -
executor >  local (3)
[39/2d13c1] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)     [100%] 1 of 1 ✔
[e7/8c6f20] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (nCoV-2019.reference.fasta)                        [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:COLLAPSE_PRIMERS                                                   [  0%] 0 of 1
[97/4886ea] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (nCoV-2019.reference.fasta)                          [100%] 1 of 1, failed: 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                       -
[f2/90d9f7] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                   [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                          -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                     -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:KRAKEN2_KRAKEN2                                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BOWTIE2_ALIGN                                                       -
executor >  local (3)
[39/2d13c1] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)     [100%] 1 of 1 ✔
[e7/8c6f20] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (nCoV-2019.reference.fasta)                        [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:COLLAPSE_PRIMERS                                                   [  0%] 0 of 1
[97/4886ea] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (nCoV-2019.reference.fasta)                          [100%] 1 of 1, failed: 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                       -
[f2/90d9f7] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                   [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                          -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                     -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:KRAKEN2_KRAKEN2                                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BOWTIE2_ALIGN                                                       -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:SAMTOOLS_SORT                                     -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:SAMTOOLS_INDEX                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS                 -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_MAPPED                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:IVAR_TRIM                                                        -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:SAMTOOLS_SORT                                  -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:SAMTOOLS_INDEX                                 -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PICARD_COLLECTMULTIPLEMETRICS                                                     -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MOSDEPTH_GENOME                                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PLOT_MOSDEPTH_REGIONS_GENOME                                                      -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MOSDEPTH_AMPLICON                                                                 -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PLOT_MOSDEPTH_REGIONS_AMPLICON                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:IVAR_VARIANTS                                                       -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:IVAR_VARIANTS_TO_VCF                                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VCF_BGZIP_TABIX_STATS:TABIX_BGZIP                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VCF_BGZIP_TABIX_STATS:VCF_TABIX_STATS:TABIX_TABIX                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VCF_BGZIP_TABIX_STATS:VCF_TABIX_STATS:BCFTOOLS_STATS                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:IVAR_CONSENSUS                                                      -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:PLOT_BASE_DENSITY                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:QUAST                                                               -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:PANGOLIN                                                            -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:NEXTCLADE                                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:SNPEFF_SNPSIFT:SNPEFF_ANN                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:SNPEFF_SNPSIFT:VCF_BGZIP_TABIX_STATS:TABIX_BGZIP                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:SNPEFF_SNPSIFT:VCF_BGZIP_TABIX_STATS:VCF_TABIX_STATS:TABIX_TABIX    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:SNPEFF_SNPSIFT:VCF_BGZIP_TABIX_STATS:VCF_TABIX_STATS:BCFTOOLS_STATS -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:SNPEFF_SNPSIFT:SNPSIFT_EXTRACTFIELDS                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:ASCIIGENOME                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_IVAR_NEXTCLADE                                                 -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:GET_SOFTWARE_VERSIONS                                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC                                                                           -
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/viralrecon] Pipeline completed with errors-
Error executing process > 'NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (nCoV-2019.reference.fasta)'

Caused by:
  Process requirement exceed available memory -- req: 72 GB; avail: 62.8 GB

Command executed:

  mkdir bowtie2
  bowtie2-build --seed 1 --threads 12 nCoV-2019.reference.fasta bowtie2/nCoV-2019.reference
  echo $(bowtie2 --version 2>&1) | sed 's/^.*bowtie2-align-s version //; s/ .*$//' > bowtie2.version.txt

Command exit status:
  -

Command output:
  (empty)

Work dir:
  /home/abayomi/work/97/4886ea787ad62b4dae9c06ca189d2c

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
Abayomi-Olabode commented 2 years ago

Command for Illumina shotgun analysis analysis:

abayomi@Langley:~$ nextflow run nf-core/viralrecon       --input ./test_viralrecon/samplesheet.csv       --platform illumina       --protocol metagenomic       --genome 'MN908947.3'       -profile docker
N E X T F L O W  ~  version 21.10.6
Launching `nf-core/viralrecon` [boring_shannon] - revision: 2ebae61442 [master]

------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/viralrecon v2.2
------------------------------------------------------
Core Nextflow options
  revision       : master
  runName        : boring_shannon
  containerEngine: docker
  launchDir      : /home/abayomi
  workDir        : /home/abayomi/work
  projectDir     : /home/abayomi/.nextflow/assets/nf-core/viralrecon
  userName       : abayomi
  profile        : docker
  configFiles    : /home/abayomi/.nextflow/assets/nf-core/viralrecon/nextflow.config

Input/output options
  input          : ./test_viralrecon/samplesheet.csv
  platform       : illumina
  protocol       : metagenomic

Reference genome options
  genome         : MN908947.3
  fasta          : https://github.com/nf-core/test-datasets/raw/viralrecon/genome/MN908947.3/GCA_009858895.3_ASM985889v3_genomic.200409.fna.gz
  gff            : https://github.com/nf-core/test-datasets/raw/viralrecon/genome/MN908947.3/GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/viralrecon for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.3901628

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/viralrecon/blob/master/CITATIONS.md
------------------------------------------------------
executor >  local (5)
[25/3b02d9] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_FASTA (GCA_009858895.3_ASM985889v3_genomic.200409.fna.gz)       [100%] 1 of 1 ✔
[89/1d2437] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)         [  0%] 0 of 1
[23/5c35c0] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (GCA_009858895.3_ASM985889v3_genomic.200409.fna)       [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                       -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD                                                          [  0%] 0 of 1
[d4/9e211c] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BLAST_MAKEBLASTDB (GCA_009858895.3_ASM985889v3_genomic.200409.fna)     [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                           -
[99/ac0dac] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                       [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                               -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                              -
executor >  local (5)
[25/3b02d9] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_FASTA (GCA_009858895.3_ASM985889v3_genomic.200409.fna.gz)       [100%] 1 of 1 ✔
[89/1d2437] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)         [100%] 1 of 1 ✔
[23/5c35c0] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (GCA_009858895.3_ASM985889v3_genomic.200409.fna)       [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                       -
[22/973105] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (GCA_009858895.3_ASM985889v3_genomic.200409.fna)         [100%] 1 of 1, failed: 1
[d4/9e211c] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BLAST_MAKEBLASTDB (GCA_009858895.3_ASM985889v3_genomic.200409.fna)     [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                           [  0%] 0 of 1
[99/ac0dac] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                       [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                               -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                         -
executor >  local (5)
[25/3b02d9] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_FASTA (GCA_009858895.3_ASM985889v3_genomic.200409.fna.gz)       [100%] 1 of 1 ✔
[89/1d2437] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)         [100%] 1 of 1 ✔
[23/5c35c0] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (GCA_009858895.3_ASM985889v3_genomic.200409.fna)       [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                       -
[22/973105] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (GCA_009858895.3_ASM985889v3_genomic.200409.fna)         [100%] 1 of 1, failed: 1
[d4/9e211c] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BLAST_MAKEBLASTDB (GCA_009858895.3_ASM985889v3_genomic.200409.fna)     [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                           [  0%] 0 of 1
[99/ac0dac] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                       [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                               -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                         -
executor >  local (5)
[25/3b02d9] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_FASTA (GCA_009858895.3_ASM985889v3_genomic.200409.fna.gz)       [100%] 1 of 1 ✔
[89/1d2437] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)         [100%] 1 of 1 ✔
[23/5c35c0] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (GCA_009858895.3_ASM985889v3_genomic.200409.fna)       [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                       -
[22/973105] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (GCA_009858895.3_ASM985889v3_genomic.200409.fna)         [100%] 1 of 1, failed: 1
[d4/9e211c] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BLAST_MAKEBLASTDB (GCA_009858895.3_ASM985889v3_genomic.200409.fna)     [  0%] 0 of 1
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                           [  0%] 0 of 1
[99/ac0dac] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                       [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                               -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                         -
executor >  local (5)
[25/3b02d9] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_FASTA (GCA_009858895.3_ASM985889v3_genomic.200409.fna.gz)       [100%] 1 of 1 ✔
[89/1d2437] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)         [100%] 1 of 1 ✔
[23/5c35c0] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (GCA_009858895.3_ASM985889v3_genomic.200409.fna)       [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                       -
[22/973105] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (GCA_009858895.3_ASM985889v3_genomic.200409.fna)         [100%] 1 of 1, failed: 1
[d4/9e211c] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BLAST_MAKEBLASTDB (GCA_009858895.3_ASM985889v3_genomic.200409.fna)     [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                           [  0%] 0 of 1
[99/ac0dac] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                       [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                               -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:KRAKEN2_KRAKEN2                                                                       -
executor >  local (5)
[25/3b02d9] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_FASTA (GCA_009858895.3_ASM985889v3_genomic.200409.fna.gz)       [100%] 1 of 1 ✔
[89/1d2437] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)         [100%] 1 of 1 ✔
[23/5c35c0] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GET_CHROM_SIZES (GCA_009858895.3_ASM985889v3_genomic.200409.fna)       [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB                                                       -
[22/973105] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (GCA_009858895.3_ASM985889v3_genomic.200409.fna)         [100%] 1 of 1, failed: 1
[d4/9e211c] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BLAST_MAKEBLASTDB (GCA_009858895.3_ASM985889v3_genomic.200409.fna)     [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD                                                           [  0%] 0 of 1
[99/ac0dac] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (samplesheet.csv)                                       [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW                                                               -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP                                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM                                                              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_READS                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:KRAKEN2_KRAKEN2                                                                       -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BOWTIE2_ALIGN                                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:SAMTOOLS_SORT                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:SAMTOOLS_INDEX                                        -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS                     -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT                  -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS                  -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_FAIL_MAPPED                                                        -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PICARD_COLLECTMULTIPLEMETRICS                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MOSDEPTH_GENOME                                                                       -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PLOT_MOSDEPTH_REGIONS_GENOME                                                          -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:BCFTOOLS_MPILEUP                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:MAKE_CONSENSUS:BEDTOOLS_GENOMECOV                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:MAKE_CONSENSUS:BEDTOOLS_MERGE                                       -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:MAKE_CONSENSUS:MAKE_BED_MASK                                        -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:MAKE_CONSENSUS:BEDTOOLS_MASKFASTA                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:MAKE_CONSENSUS:BCFTOOLS_CONSENSUS                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:MAKE_CONSENSUS:PLOT_BASE_DENSITY                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:QUAST                                                               -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:PANGOLIN                                                            -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:NEXTCLADE                                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:SNPEFF_SNPSIFT:SNPEFF_ANN                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:SNPEFF_SNPSIFT:VCF_BGZIP_TABIX_STATS:TABIX_BGZIP                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:SNPEFF_SNPSIFT:VCF_BGZIP_TABIX_STATS:VCF_TABIX_STATS:TABIX_TABIX    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:SNPEFF_SNPSIFT:VCF_BGZIP_TABIX_STATS:VCF_TABIX_STATS:BCFTOOLS_STATS -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:SNPEFF_SNPSIFT:SNPSIFT_EXTRACTFIELDS                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_BCFTOOLS:ASCIIGENOME                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_CUSTOM_TSV_BCFTOOLS_NEXTCLADE                                                 -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ASSEMBLY_SPADES:SPADES                                                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ASSEMBLY_SPADES:BANDAGE_IMAGE                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ASSEMBLY_SPADES:ASSEMBLY_QC:BLAST_BLASTN                                              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ASSEMBLY_SPADES:ASSEMBLY_QC:FILTER_BLASTN                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ASSEMBLY_SPADES:ASSEMBLY_QC:QUAST                                                     -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ASSEMBLY_SPADES:ASSEMBLY_QC:ABACAS                                                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:ASSEMBLY_SPADES:ASSEMBLY_QC:PLASMIDID                                                 -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:GET_SOFTWARE_VERSIONS                                                                 -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC                                                                               -
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/viralrecon] Pipeline completed with errors-
Error executing process > 'NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (GCA_009858895.3_ASM985889v3_genomic.200409.fna)'

Caused by:
  Process requirement exceed available memory -- req: 72 GB; avail: 62.8 GB

Command executed:

  mkdir bowtie2
  bowtie2-build --seed 1 --threads 12 GCA_009858895.3_ASM985889v3_genomic.200409.fna bowtie2/GCA_009858895.3_ASM985889v3_genomic.200409
  echo $(bowtie2 --version 2>&1) | sed 's/^.*bowtie2-align-s version //; s/ .*$//' > bowtie2.version.txt

Command exit status:
  -

Command output:
  (empty)

Work dir:
  /home/abayomi/work/22/9731054594e7c7b5ee7a9650891b0f

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
Abayomi-Olabode commented 2 years ago

The main issue so has to do with available memory on Langley!

Caused by:
  Process requirement exceed available memory -- req: 72 GB; avail: 62.8 GB
Abayomi-Olabode commented 2 years ago

Trying some work arounds

ArtPoon commented 2 years ago

I can't conceive of how that step bowtie2-build could run out of RAM on such a small reference genome sequence. You should be able to run this command:

bowtie2-build --seed 1 --threads 12 GCA_009858895.3_ASM985889v3_genomic.200409.fna bowtie2/GCA_009858895.3_ASM985889v3_genomic.200409

on its own without any problems. Something else must be going on.

Abayomi-Olabode commented 2 years ago

The command below seems to be running fine so far. Will keep an eye!

abayomi@Langley:~/test_viralrecon$ /usr/local/bin/nextflow run nf-core/viralrecon --platform illumina --protocol amplicon --input /home/abayomi/test_viralrecon/samplesheet.csv --genome 'MN908947.3' --primer_bed /home/abayomi/test_viralrecon/nCoV-2019.primer.bed --primer_left_suffix '_L' --primer_right_suffix '_R' --ivar_trim_offset 5 --skip_kraken2 -profile test,docker
N E X T F L O W  ~  version 21.10.6
Launching `nf-core/viralrecon` [happy_caravaggio] - revision: 2ebae61442 [master]

------------------------------------------------------
                                        ,--./,-.
        ___     __   __   __   ___     /,-._.--~'
  |\ | |__  __ /  ` /  \ |__) |__         }  {
  | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                        `._,._,'
  nf-core/viralrecon v2.2
------------------------------------------------------
Core Nextflow options
  revision                  : master
  runName                   : happy_caravaggio
  containerEngine           : docker
  launchDir                 : /home/abayomi/test_viralrecon
  workDir                   : /home/abayomi/test_viralrecon/work
  projectDir                : /home/abayomi/.nextflow/assets/nf-core/viralrecon
  userName                  : abayomi
  profile                   : test,docker
  configFiles               : /home/abayomi/.nextflow/assets/nf-core/viralrecon/nextflow.config

Input/output options
  input                     : /home/abayomi/test_viralrecon/samplesheet.csv
  platform                  : illumina
  protocol                  : amplicon

Reference genome options
  genome                    : MN908947.3
  fasta                     : https://github.com/artic-network/artic-ncov2019/raw/master/primer_schemes/nCoV-2019/V1/nCoV-2019.reference.fasta
  gff                       : https://github.com/nf-core/test-datasets/raw/viralrecon/genome/MN908947.3/GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz
  primer_bed                : /home/abayomi/test_viralrecon/nCoV-2019.primer.bed
  primer_set                : artic
  primer_set_version        : 1
  primer_left_suffix        : _L
  primer_right_suffix       : _R

Illumina QC, read trimming and filtering options
  kraken2_db                : https://raw.githubusercontent.com/nf-core/test-datasets/viralrecon/genome/kraken2/kraken2_hs22.tar.gz
  skip_kraken2              : true

Illumina variant calling options
  callers                   : ivar,bcftools
  ivar_trim_offset          : 5

Illumina de novo assembly options
  assemblers                : spades,unicycler,minia
  skip_plasmidid            : true

Max job request options
  max_cpus                  : 2
  max_memory                : 6 GB
  max_time                  : 6h

Institutional config options
  config_profile_name       : Test profile
  config_profile_description: Minimal test dataset to check pipeline function

!! Only displaying parameters that differ from the pipeline defaults !!
------------------------------------------------------
If you use nf-core/viralrecon for your analysis please cite:

* The pipeline
  https://doi.org/10.5281/zenodo.3901628

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

* Software dependencies
  https://github.com/nf-core/viralrecon/blob/master/CITATIONS.md
------------------------------------------------------
Abayomi-Olabode commented 2 years ago

Still running but results file contains a couple of output subdirectories:

assembly fastp fastqc pipeline_info variants

Abayomi-Olabode commented 2 years ago

Variants contains the following subdirectories:

bcftools bowtie2 intersect ivar

Abayomi-Olabode commented 2 years ago

ivar contains the following subdirectories:

asciigenome bcftools_stats consensus log nextclade pangolin SAMPLE_1.tsv SAMPLE_2.tsv SAMPLE_2.vcf.gz SAMPLE_2.vcf.gz.tbi snpeff

Abayomi-Olabode commented 2 years ago

Pipeline still running. Will wait for run to be completed!

ArtPoon commented 2 years ago

Let's not post actual data on the issue tracker @Abayomi-Olabode

Abayomi-Olabode commented 2 years ago

Thanks @ArtPoon

Abayomi-Olabode commented 2 years ago 
abayomi@Langley:~/test_viralrecon$ /usr/local/bin/nextflow run nf-core/viralrecon --platform illumina --protocol amplicon --input /home/abayomi/test_viralrecon/samplesheet.csv --genome 'MN908947.3' --primer_bed /home/abayomi/test_viralrecon/nCoV-2019.primer.bed --primer_left_suffix '_L' --primer_right_suffix '_R' --ivar_trim_offset 5 --skip_kraken2 -profile test,docker

-[nf-core/viralrecon] 2/2 samples passed Bowtie2 1000 mapped read threshold:
    4341: SAMPLE_2
    3808762: SAMPLE_1

-[nf-core/viralrecon] Pipeline completed successfully-
Completed at: 25-Jan-2022 01:45:28
Duration    : 55m 53s
CPU hours   : 3.9
Succeeded   : 149
abayomi@Langley:~/test_viralrecon$ /usr/local/bin/nextflow run nf-core/viralrecon --platform illumina --protocol amplicon --input /home/abayomi/test_viralrecon/samplesheet.csv --genome 'MN908947.3' --primer_bed /home/abayomi/test_viralrecon/nCoV-2019.primer.bed --primer_left_suffix '_L' --primer_right_suffix '_R' --ivar_trim_offset 5 --skip_assembly --skip_kraken2 -profile test,docker

-[nf-core/viralrecon] 2/2 samples passed Bowtie2 1000 mapped read threshold:
    4341: SAMPLE_2
    3808762: SAMPLE_1
-
-[nf-core/viralrecon] Pipeline completed successfully-
Completed at: 01-Feb-2022 02:24:15
Duration    : 30m 11s
CPU hours   : 1.4
Succeeded   : 112
Abayomi-Olabode commented 2 years ago

Fixed the memory allocation issue and brought time down to ~20 mins. 

/usr/local/bin/nextflow run nf-core/viralrecon --platform illumina --protocol amplicon --input /home/abayomi/test_viralrecon/samplesheet.csv --genome 'MN908947.3' --primer_set artic --primer_set_version 3 --max_time 1.h --max_cpus 8 --max_memory 20GB --skip_assembly --skip_kraken2 -profile docker

-[nf-core/viralrecon] 2/2 samples passed Bowtie2 1000 mapped read threshold:
    4341: SAMPLE_2
    3808762: SAMPLE_1
-
-[nf-core/viralrecon] Pipeline completed successfully-
Completed at: 02-Feb-2022 21:02:39
Duration    : 19m 52s
CPU hours   : 2.5
Succeeded   : 78
Abayomi-Olabode commented 2 years ago

abayomi@Langley:~$ /usr/local/bin/nextflow run nf-core/viralrecon --platform illumina --protocol amplicon --input /home/abayomi/test_viralrecon/samplesheet.csv --genome 'MN908947.3' --primer_set artic --primer_set_version 3 --max_time 1.h --max_cpus 8 --max_memory 40GB --skip_assembly --skip_kraken2 -profile docker

-[nf-core/viralrecon] 2/2 samples passed Bowtie2 1000 mapped read threshold: 4341: SAMPLE_2 3808762: SAMPLE_1

-[nf-core/viralrecon] Pipeline completed successfully- Completed at: 02-Feb-2022 23:45:40 Duration : 25m 17s CPU hours : 2.5 Succeeded : 78


- --max_memory 5GB

/usr/local/bin/nextflow run nf-core/viralrecon --platform illumina --protocol amplicon --input /home/abayomi/test_viralrecon/samplesheet.csv --genome 'MN908947.3' --primer_set artic --primer_set_version 3 --max_time 1.h --max_cpus 8 --max_memory 5GB --skip_assembly --skip_kraken2 -profile docker

-[nf-core/viralrecon] 2/2 samples passed Bowtie2 1000 mapped read threshold: 4341: SAMPLE_2 3808762: SAMPLE_1

-[nf-core/viralrecon] Pipeline completed successfully- Completed at: 03-Feb-2022 10:25:06 Duration : 20m 25s CPU hours : 2.6 Succeeded : 78


- --max_memory 10GB

/usr/local/bin/nextflow run nf-core/viralrecon --platform illumina --protocol amplicon --input /home/abayomi/test_viralrecon/samplesheet.csv --genome 'MN908947.3' --primer_set artic --primer_set_version 3 --max_time 1.h --max_cpus 8 --max_memory 10GB --skip_assembly --skip_kraken2 -profile docker

-[nf-core/viralrecon] 2/2 samples passed Bowtie2 1000 mapped read threshold: 4341: SAMPLE_2 3808762: SAMPLE_1

-[nf-core/viralrecon] Pipeline completed successfully- Completed at: 03-Feb-2022 10:49:32 Duration : 20m 18s CPU hours : 2.5 Succeeded : 78

Abayomi-Olabode commented 2 years ago

Need to update this 

N E X T F L O W  ~  version 21.10.6
Launching `nf-core/viralrecon` [suspicious_goldwasser] - revision: 2ebae61442 [master]
NOTE: Your local project version looks outdated - a different revision is available in the remote repository [fc9fece226]
Abayomi-Olabode commented 2 years ago

Allocating a maximum of 4 cpus speeds up the run times ranging from 32m 26s - 1h 33m 24s

ArtPoon commented 2 years ago

Can you write a script to convert the tsv "VCF-like" output into something that resembles the mapped.csv file we generate with minimap.py? That will make comparison a lot easier

ArtPoon commented 2 years ago

also is it possible to extract coverage for all reference positions?

Abayomi-Olabode commented 2 years ago

Sure, on it

Abayomi-Olabode commented 2 years ago

A new version ( v 2.3) of nf-core/viralrecon has just been released. It has some major changes that impacts our previous run results.

[2.3] - 2022-02-04 :warning: Major enhancements

- Please see Major updates in v2.3 for a more detailed list of changes added in this version.
- When using --protocol amplicon, in the previous release, iVar was used for both the variant calling and consensus  --- - sequence generation. The pipeline will now perform the variant calling and consensus sequence generation with iVar and BCFTools/BEDTools, respectively.
- Bump minimum Nextflow version from 21.04.0 -> 21.10.3

Enhancements & fixes

- Port pipeline to the updated Nextflow DSL2 syntax adopted on nf-core/modules
- Updated pipeline template to nf-core/tools 2.2
- [#209] - Check that contig in primer BED and genome fasta match
- [#218] - Support for compressed FastQ files for Nanopore data
- [#232] - Remove duplicate variants called by ARTIC ONT pipeline
- [#235] - Nextclade version bump
- [#244] - Fix BCFtools consensus generation and masking
- [#245] - Mpileup file as output
- [#246] - Option to generate consensus with BCFTools / BEDTools using iVar variants
- [#247] - Add strand-bias filtering option and codon fix in consecutive positions in ivar tsv conversion to vcf
- [#248] - New variants reporting table
Abayomi-Olabode commented 2 years ago

Now repeating the previous runs with the latest version as follows:

nextflow run nf-core/viralrecon -r 2.3 --platform illumina --protocol amplicon --input ./samplesheet.csv --genome 'MN908947.3' --primer_set artic --primer_set_version 3 --max_time 12.h --max_cpus 4 --max_memory 5GB --skip_assembly --skip_kraken2 -profile docker