search

epi2me-labs / wf-metagenomics

Metagenomic classification of long-read sequencing data
Other
51 stars 23 forks source link

Error message about Matplotlib and anytree #83

Open weishwu opened 6 months ago

weishwu commented 6 months ago

Operating System

Other Linux (please specify below)

Other Linux

RedHat 7.9 (Maipo)

Workflow Version

v2.9.0-gd9e8593

Workflow Execution

Command line

EPI2ME Version

No response

CLI command run

export NXF_SINGULARITY_CACHEDIR=./env/nextflow_singularity_images/ export MPLCONFIGDIR=./outputs/wf_metagenomics/KP4/mplconfigdir/

nextflow run epi2me-labs/wf-metagenomics \ --fastq KP4.pass.filt.fastq \ --classifier minimap2 \ --sample KP4 \ --reference 741_wzi_types.fa \ --ref2taxid 741_wzi_types.ref2taxid.tab \ --keep_bam \ --minimap2_by_reference \ --out_dir wf_metagenomics/KP4 \ --threads 40 \ -profile singularity \ -c ${nf_conf}

Workflow Execution - CLI Execution Profile

singularity

What happened?

Got an error:

Command error:
  [15:31:44 - matplotlib] Matplotlib created a temporary cache directory at /tmp/matplotl
ib-clbezbtj because the default path (/home/weishwu/.config/matplotlib) is not a writable
 directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a wr
itable directory, in particular to speed up the import of Matplotlib and to better suppor
t multiprocessing.
  [15:31:45 - matplotlib.font_manager] generated new fontManager
  /home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/__init__.p
y:30: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    logger.warn(f"Could not load {name} due to missing module {e.name}")
  [15:31:46 - workflow_glue] Could not load abundance_tables due to missing module anytre
e

Relevant log output

N E X T F L O W  ~  version 23.10.1
Launching `https://github.com/epi2me-labs/wf-metagenomics` [suspicious_feynman] DSL2 - revision: d9e8593411 [master]
WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-metagenomics v2.9.0-gd9e8593
--------------------------------------------------------------------------------
Core Nextflow options
  revision             : master
  runName              : suspicious_feynman
  containerEngine      : singularity
  container            : [withLabel:wfmetagenomics:ontresearch/wf-metagenomics:sha44a6dacff5f2001d917b774647bb4cbc1b53bc76, withLabel:wf_common:ontresearch/wf-common:sha1c5febff9f75143710826498b093d9769a5edbb9, withLabel:amr:ontresearch/abricate:sha2c763f19fac46035437854f1e2a5f05553542a78]
  launchDir            : /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/outputs/wf_metagenomics/KP4
  workDir              : /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/outputs/wf_metagenomics/KP4/work
  projectDir           : /home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics
  userName             : weishwu
  profile              : singularity
  configFiles          : /home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/nextflow.config, /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/env/nextflow_resource.cfg

Input Options
  fastq                : /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/outputs/fastq_pass_trimmed/KP4.pass.filt.fastq
  classifier           : minimap2

Sample Options
  sample               : KP4

Reference Options
  reference            : /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/ref_data/741_wzi_types.fa
  ref2taxid            : /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/ref_data/741_wzi_types.ref2taxid.tab
  database_sets        : [ncbi_16s_18s:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_16s_18s.fna, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_kraken2.tar.gz, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ref2taxid.targloci.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], ncbi_16s_18s_28s_ITS:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS.fna, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS_kraken2.tar.gz, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ref2taxid.ncbi_16s_18s_28s_ITS.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], SILVA_138_1:[database:null], Standard-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_standard_08gb_20231009.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip], PlusPF-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_pluspf_08gb_20230314.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip], PlusPFP-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_pluspfp_08gb_20230314.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip]]

Minimap2 Options
  keep_bam             : true
  minimap2_by_reference: true

Output Options
  out_dir              : /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/outputs/wf_metagenomics/KP4

Advanced Options
  threads              : 40

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-metagenomics for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-metagenomics v2.9.0-gd9e8593.
--------------------------------------------------------------------------------
Checking inputs.
Note: Reference/Database are custom.
Note: Memory available to the workflow must be slightly higher than size of the database custom index.
Note: Memory available to the workflow must be slightly higher than size of the database Standard-8 index (8GB) or consider to use --kraken2_memory_mapping
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
[-        ] process > fastcat                                 -
[-        ] process > prepare_databases:download_unpack_ta... -
[-        ] process > prepare_databases:check_reference_re... -
[-        ] process > minimap_pipeline:run_common:getVersions -
[-        ] process > minimap_pipeline:run_common:getParams   -
[-        ] process > minimap_pipeline:minimap                -
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[-        ] process > minimap_pipeline:output_results         -
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (5)
[6c/31c405] process > fastcat (1)                             [  0%] 0 of 1
[25/079f7e] process > prepare_databases:download_unpack_ta... [  0%] 0 of 1
[81/6bfd53] process > prepare_databases:check_reference_re... [  0%] 0 of 1
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [  0%] 0 of 1
[83/157458] process > minimap_pipeline:run_common:getParams   [  0%] 0 of 1
[-        ] process > minimap_pipeline:minimap                -
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[-        ] process > minimap_pipeline:output_results         -
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (5)
[6c/31c405] process > fastcat (1)                             [  0%] 0 of 1
[25/079f7e] process > prepare_databases:download_unpack_ta... [  0%] 0 of 1
[81/6bfd53] process > prepare_databases:check_reference_re... [  0%] 0 of 1
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [  0%] 0 of 1
[83/157458] process > minimap_pipeline:run_common:getParams   [  0%] 0 of 1
[-        ] process > minimap_pipeline:minimap                -
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[-        ] process > minimap_pipeline:output_results         -
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (6)
[6c/31c405] process > fastcat (1)                             [  0%] 0 of 1
[25/079f7e] process > prepare_databases:download_unpack_ta... [  0%] 0 of 1
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [  0%] 0 of 1
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:minimap                -
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[77/806902] process > minimap_pipeline:output_results (1)     [  0%] 0 of 1
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (6)
[6c/31c405] process > fastcat (1)                             [  0%] 0 of 1
[25/079f7e] process > prepare_databases:download_unpack_ta... [  0%] 0 of 1
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [  0%] 0 of 1
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:minimap                -
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[77/806902] process > minimap_pipeline:output_results (1)     [  0%] 0 of 1
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (6)
[6c/31c405] process > fastcat (1)                             [  0%] 0 of 1
[25/079f7e] process > prepare_databases:download_unpack_ta... [  0%] 0 of 1
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [  0%] 0 of 1
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:minimap                -
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[77/806902] process > minimap_pipeline:output_results (1)     [100%] 1 of 1
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (6)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [  0%] 0 of 1
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [  0%] 0 of 1
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:minimap                -
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[77/806902] process > minimap_pipeline:output_results (1)     [100%] 1 of 1
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (7)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [  0%] 0 of 1
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:minimap                -
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[08/f5cdb2] process > minimap_pipeline:output_results (2)     [ 50%] 1 of 2
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (7)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [  0%] 0 of 1
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:minimap                -
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[08/f5cdb2] process > minimap_pipeline:output_results (2)     [100%] 2 of 2
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (7)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:minimap                [  0%] 0 of 1
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[08/f5cdb2] process > minimap_pipeline:output_results (2)     [100%] 2 of 2
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (8)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [  0%] 0 of 1
[-        ] process > minimap_pipeline:getAlignmentStats      -
[-        ] process > minimap_pipeline:createAbundanceTables  -
[-        ] process > minimap_pipeline:makeReport             -
[08/f5cdb2] process > minimap_pipeline:output_results (2)     [100%] 2 of 2
Minimap2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom reference exists
Checking custom ref2taxid mapping exists

executor >  local (11)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:getAlignmentStats      [  0%] 0 of 1
[58/34875d] process > minimap_pipeline:createAbundanceTables  [  0%] 0 of 1
[-        ] process > minimap_pipeline:makeReport             -
[97/81465c] process > minimap_pipeline:output_results (3)     [ 50%] 2 of 4

executor >  local (11)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:getAlignmentStats      [  0%] 0 of 1
[58/34875d] process > minimap_pipeline:createAbundanceTables  [  0%] 0 of 1
[-        ] process > minimap_pipeline:makeReport             -
[65/127cab] process > minimap_pipeline:output_results (4)     [100%] 4 of 4

executor >  local (11)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:getAlignmentStats      [  0%] 0 of 1
[58/34875d] process > minimap_pipeline:createAbundanceTables  [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:makeReport             -
[65/127cab] process > minimap_pipeline:output_results (4)     [100%] 4 of 4

executor >  local (12)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[e0/64a392] process > minimap_pipeline:getAlignmentStats (... [  0%] 0 of 1
[58/34875d] process > minimap_pipeline:createAbundanceTables  [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:makeReport             -
[65/127cab] process > minimap_pipeline:output_results (4)     [ 80%] 4 of 5

executor >  local (13)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[e0/64a392] process > minimap_pipeline:getAlignmentStats (... [100%] 1 of 1 ✔
[58/34875d] process > minimap_pipeline:createAbundanceTables  [100%] 1 of 1 ✔
[-        ] process > minimap_pipeline:makeReport             -
[ae/de7959] process > minimap_pipeline:output_results (5)     [ 80%] 4 of 5

executor >  local (14)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[e0/64a392] process > minimap_pipeline:getAlignmentStats (... [100%] 1 of 1 ✔
[58/34875d] process > minimap_pipeline:createAbundanceTables  [100%] 1 of 1 ✔
[96/6b7bcc] process > minimap_pipeline:makeReport (1)         [  0%] 0 of 1
[ae/de7959] process > minimap_pipeline:output_results (5)     [ 80%] 4 of 5

executor >  local (14)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[e0/64a392] process > minimap_pipeline:getAlignmentStats (... [100%] 1 of 1 ✔
[58/34875d] process > minimap_pipeline:createAbundanceTables  [100%] 1 of 1 ✔
[96/6b7bcc] process > minimap_pipeline:makeReport (1)         [  0%] 0 of 1
[ae/de7959] process > minimap_pipeline:output_results (5)     [100%] 5 of 5

executor >  local (14)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[e0/64a392] process > minimap_pipeline:getAlignmentStats (... [100%] 1 of 1 ✔
[58/34875d] process > minimap_pipeline:createAbundanceTables  [100%] 1 of 1 ✔
[96/6b7bcc] process > minimap_pipeline:makeReport (1)         [  0%] 0 of 1
[ae/de7959] process > minimap_pipeline:output_results (5)     [100%] 5 of 5
ERROR ~ Error executing process > 'minimap_pipeline:makeReport (1)'

Caused by:
  Process `minimap_pipeline:makeReport (1)` terminated with an error exit status (1)

Command executed:

  workflow-glue report         "wf-metagenomics-report.html"         --workflow_name wf-metagenomics         --versions versions         --params params.json         --read_stats read_stats/*         --lineages lineages         --abundance_table "abundance_table_species.tsv"         --taxonomic_rank "S"         --pipeline "minimap2"         --abundance_threshold "0"        --n_taxa_barplot "9"        --align_stats alignment_stats

Command exit status:
  1

Command output:
  (empty)

Command error:
  [15:31:44 - matplotlib] Matplotlib created a temporary cache directory at /tmp/matplotlib-clbezbtj because the default path (/home/weishwu/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
  [15:31:45 - matplotlib.font_manager] generated new fontManager
  /home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/__init__.py:30: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    logger.warn(f"Could not load {name} due to missing module {e.name}")
  [15:31:46 - workflow_glue] Could not load abundance_tables due to missing module anytree
  [15:31:46 - workflow_glue] Starting entrypoint.
  [15:31:47 - workflow_glue.Report    ] Sample KP4.
  [15:31:47 - EChrtPlotr] Skipping axis label fixing
  [15:31:47 - workflow_glue.Report    ] rank phylum.
  [15:31:47 - workflow_glue.Report    ] rank class.
  [15:31:47 - workflow_glue.Report    ] rank order.
  [15:31:47 - workflow_glue.Report    ] rank family.
  [15:31:47 - workflow_glue.Report    ] rank genus.
  [15:31:47 - workflow_glue.Report    ] rank species.
  Traceback (most recent call last):
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow-glue", line 7, in <module>
      cli()
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/__init__.py", line 72, in cli
      args.func(args)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report.py", line 419, in main
      samples_references[s] = report_utils.load_alignment_data(
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 385, in load_alignment_data
      align_df = alignment_metrics(depth, stats)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 300, in alignment_metrics
      metrics = stats.apply(lambda row: coverage_dispersion(
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/frame.py", line 9568, in apply
      return op.apply().__finalize__(self, method="apply")
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/apply.py", line 764, in apply
      return self.apply_standard()
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/apply.py", line 891, in apply_standard
      results, res_index = self.apply_series_generator()
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/apply.py", line 907, in apply_series_generator
      results[i] = self.f(v)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 300, in <lambda>
      metrics = stats.apply(lambda row: coverage_dispersion(
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 280, in coverage_dispersion
      all_depth_pos = nreads_all_positions(depth_ref, ref_len)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 263, in nreads_all_positions
      all_depth_pos = pd.DataFrame(0, index=range(1, ref_len + 1), columns=["depth"])
  TypeError: can only concatenate str (not "int") to str

Work dir:
  /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/outputs/wf_metagenomics/KP4/work/96/6b7bccf947bde70d53445a2401772e

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details

executor >  local (14)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[e0/64a392] process > minimap_pipeline:getAlignmentStats (... [100%] 1 of 1 ✔
[58/34875d] process > minimap_pipeline:createAbundanceTables  [100%] 1 of 1 ✔
[96/6b7bcc] process > minimap_pipeline:makeReport (1)         [100%] 1 of 1, failed: 1 ✘
[ae/de7959] process > minimap_pipeline:output_results (5)     [100%] 5 of 5 ✔
ERROR ~ Error executing process > 'minimap_pipeline:makeReport (1)'

Caused by:
  Process `minimap_pipeline:makeReport (1)` terminated with an error exit status (1)

Command executed:

  workflow-glue report         "wf-metagenomics-report.html"         --workflow_name wf-metagenomics         --versions versions         --params params.json         --read_stats read_stats/*         --lineages lineages         --abundance_table "abundance_table_species.tsv"         --taxonomic_rank "S"         --pipeline "minimap2"         --abundance_threshold "0"        --n_taxa_barplot "9"        --align_stats alignment_stats

Command exit status:
  1

Command output:
  (empty)

Command error:
  [15:31:44 - matplotlib] Matplotlib created a temporary cache directory at /tmp/matplotlib-clbezbtj because the default path (/home/weishwu/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
  [15:31:45 - matplotlib.font_manager] generated new fontManager
  /home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/__init__.py:30: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    logger.warn(f"Could not load {name} due to missing module {e.name}")
  [15:31:46 - workflow_glue] Could not load abundance_tables due to missing module anytree
  [15:31:46 - workflow_glue] Starting entrypoint.
  [15:31:47 - workflow_glue.Report    ] Sample KP4.
  [15:31:47 - EChrtPlotr] Skipping axis label fixing
  [15:31:47 - workflow_glue.Report    ] rank phylum.
  [15:31:47 - workflow_glue.Report    ] rank class.
  [15:31:47 - workflow_glue.Report    ] rank order.
  [15:31:47 - workflow_glue.Report    ] rank family.
  [15:31:47 - workflow_glue.Report    ] rank genus.
  [15:31:47 - workflow_glue.Report    ] rank species.
  Traceback (most recent call last):
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow-glue", line 7, in <module>
      cli()
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/__init__.py", line 72, in cli
      args.func(args)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report.py", line 419, in main
      samples_references[s] = report_utils.load_alignment_data(
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 385, in load_alignment_data
      align_df = alignment_metrics(depth, stats)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 300, in alignment_metrics
      metrics = stats.apply(lambda row: coverage_dispersion(
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/frame.py", line 9568, in apply
      return op.apply().__finalize__(self, method="apply")
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/apply.py", line 764, in apply
      return self.apply_standard()
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/apply.py", line 891, in apply_standard
      results, res_index = self.apply_series_generator()
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/apply.py", line 907, in apply_series_generator
      results[i] = self.f(v)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 300, in <lambda>
      metrics = stats.apply(lambda row: coverage_dispersion(
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 280, in coverage_dispersion
      all_depth_pos = nreads_all_positions(depth_ref, ref_len)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 263, in nreads_all_positions
      all_depth_pos = pd.DataFrame(0, index=range(1, ref_len + 1), columns=["depth"])
  TypeError: can only concatenate str (not "int") to str

Work dir:
  /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/outputs/wf_metagenomics/KP4/work/96/6b7bccf947bde70d53445a2401772e

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details

executor >  local (14)
[6c/31c405] process > fastcat (1)                             [100%] 1 of 1 ✔
[25/079f7e] process > prepare_databases:download_unpack_ta... [100%] 1 of 1 ✔
[81/6bfd53] process > prepare_databases:check_reference_re... [100%] 1 of 1 ✔
[47/3dea9e] process > minimap_pipeline:run_common:getVersions [100%] 1 of 1 ✔
[83/157458] process > minimap_pipeline:run_common:getParams   [100%] 1 of 1 ✔
[90/dd8efd] process > minimap_pipeline:minimap (KP4)          [100%] 1 of 1 ✔
[e0/64a392] process > minimap_pipeline:getAlignmentStats (... [100%] 1 of 1 ✔
[58/34875d] process > minimap_pipeline:createAbundanceTables  [100%] 1 of 1 ✔
[96/6b7bcc] process > minimap_pipeline:makeReport (1)         [100%] 1 of 1, failed: 1 ✘
[ae/de7959] process > minimap_pipeline:output_results (5)     [100%] 5 of 5 ✔
ERROR ~ Error executing process > 'minimap_pipeline:makeReport (1)'

Caused by:
  Process `minimap_pipeline:makeReport (1)` terminated with an error exit status (1)

Command executed:

  workflow-glue report         "wf-metagenomics-report.html"         --workflow_name wf-metagenomics         --versions versions         --params params.json         --read_stats read_stats/*         --lineages lineages         --abundance_table "abundance_table_species.tsv"         --taxonomic_rank "S"         --pipeline "minimap2"         --abundance_threshold "0"        --n_taxa_barplot "9"        --align_stats alignment_stats

Command exit status:
  1

Command output:
  (empty)

Command error:
  [15:31:44 - matplotlib] Matplotlib created a temporary cache directory at /tmp/matplotlib-clbezbtj because the default path (/home/weishwu/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
  [15:31:45 - matplotlib.font_manager] generated new fontManager
  /home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/__init__.py:30: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
    logger.warn(f"Could not load {name} due to missing module {e.name}")
  [15:31:46 - workflow_glue] Could not load abundance_tables due to missing module anytree
  [15:31:46 - workflow_glue] Starting entrypoint.
  [15:31:47 - workflow_glue.Report    ] Sample KP4.
  [15:31:47 - EChrtPlotr] Skipping axis label fixing
  [15:31:47 - workflow_glue.Report    ] rank phylum.
  [15:31:47 - workflow_glue.Report    ] rank class.
  [15:31:47 - workflow_glue.Report    ] rank order.
  [15:31:47 - workflow_glue.Report    ] rank family.
  [15:31:47 - workflow_glue.Report    ] rank genus.
  [15:31:47 - workflow_glue.Report    ] rank species.
  Traceback (most recent call last):
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow-glue", line 7, in <module>
      cli()
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/__init__.py", line 72, in cli
      args.func(args)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report.py", line 419, in main
      samples_references[s] = report_utils.load_alignment_data(
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 385, in load_alignment_data
      align_df = alignment_metrics(depth, stats)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 300, in alignment_metrics
      metrics = stats.apply(lambda row: coverage_dispersion(
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/frame.py", line 9568, in apply
      return op.apply().__finalize__(self, method="apply")
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/apply.py", line 764, in apply
      return self.apply_standard()
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/apply.py", line 891, in apply_standard
      results, res_index = self.apply_series_generator()
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/pandas/core/apply.py", line 907, in apply_series_generator
      results[i] = self.f(v)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 300, in <lambda>
      metrics = stats.apply(lambda row: coverage_dispersion(
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 280, in coverage_dispersion
      all_depth_pos = nreads_all_positions(depth_ref, ref_len)
    File "/home/weishwu/.nextflow/assets/epi2me-labs/wf-metagenomics/bin/workflow_glue/report_utils/report_utils.py", line 263, in nreads_all_positions
      all_depth_pos = pd.DataFrame(0, index=range(1, ref_len + 1), columns=["depth"])
  TypeError: can only concatenate str (not "int") to str

Work dir:
  /nfs/mm-isilon/bioinfcore/ActiveProjects/Bachman_mikebach_ONT1_weishwu/outputs/wf_metagenomics/KP4/work/96/6b7bccf947bde70d53445a2401772e

Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`

 -- Check '.nextflow.log' file for details


### Application activity log entry

_No response_

### Were you able to successfully run the latest version of the workflow with the demo data?

yes

### Other demo data information

_No response_
weishwu commented 6 months ago

ref.zip

This contains the reference files I'm using:

This test run completed successfully:

nextflow run epi2me-labs/wf-metagenomics --fastq test_data/case04_exclude_host/reads.fastq.gz --exclude_host test_data/case04_exclude_host/host.fasta.gz
nggvs commented 6 months ago

Hi @weishwu , First of all, thank you for using the workflow! I'll take a look to try to reproduce your issue. The error you are getting may be related to the use of the flag --minimap2_by_reference. While I investigate this and take a look on the databases files, are you able to run the workflow with your data but without that flag successfully to make sure it is the use of this flag which is causing trouble (as I've seen from you report, you have run successfully the demo)?

Thank you very much!

weishwu commented 6 months ago

@nggvs Thanks! It worked after I removed --minimap2_by_reference. However, I added this flag in order to get the coverage on each individual sequence in my reference. My reference contains the wzi gene sequence for 741 Klebsiella pneumonia strains, and the goal of this analysis is to find out the abundance of each strain. The resolution of the pipeline can only get down to species. Is there a way to accommodate the pipeline to my needs? Since I got the BAM, I actually ran mosdepth to get the coverage on each of the 741 sequences in my reference. Do you think I can rely on this data for the abundance measurement? My major concern is that minimap2 may be not aligning the reads to the correct one if some of these sequences are very close to each other (though I only used the alignments with mapping quality >0). Thanks again.

nggvs commented 3 months ago

Hi @weishwu , Apologies for the late answer, to see how alignments are working, assuming you have a reference on what you expect to find in the sample, you can use the latest version of the EPI2ME Desktop Application (5.1.14). Running minimap2 with the option keep bam, will enable the IGV panel in the app so you can take a look on the alignments