search

epi2me-labs / wf-metagenomics

Metagenomic classification of long-read sequencing data
Other
60 stars 23 forks source link

Process requirement exceeds available memory #78

Closed joyleng closed 8 months ago

joyleng commented 10 months ago

Operating System

Windows 10

Other Linux

No response

Workflow Version

v1.0.0

Workflow Execution

EPI2ME Desktop application

EPI2ME Version

v5.1.8

CLI command run

No response

Workflow Execution - CLI Execution Profile

None

What happened?

Hi

I have had EPI2ME desktop working fine with version 5.1.6 which seems to have fixed a problem I had previously. However, when I updated to the most version (5.1.8) the metagenomics workflow fails pretty quickly. I am wanting to use the Kraken database however when EPI2ME tries to load the database it crashes. It seems that wsl is not able to access as much RAM as is needed, as it need 4GB but it is only using 3.7GB. I have checked the amount of RAM on my laptop 8GB RAM with about 7.8GB available. I have included the full log including the error below.

I have tried a couple of fixes to get wsl to use more of my laptop's RAM but nothing is working. There is some documentation on EPI2ME's website on this and I have added a .wslconfig file to ./user/username which contained: [wsl2] memory=5GB processors=4

However wsl doesn't seem to be picking this up.

Any help would be much appreciated.

Thanks Joy

Relevant log output

N E X T F L O W  ~  version 23.04.2
Launching `/mnt/c/Users/joyle/epi2melabs/workflows/epi2me-labs/wf-16s/main.nf` [gifted_banach] DSL2 - revision: 0f29a0d8f6
WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-16s v1.0.0
--------------------------------------------------------------------------------
Core Nextflow options
  runName               : gifted_banach
  containerEngine       : docker
  launchDir             : /mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HMGK5TFW3HZ981TXFF6DWKRB
  workDir               : /mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HMGK5TFW3HZ981TXFF6DWKRB/work
  projectDir            : /mnt/c/Users/joyle/epi2melabs/workflows/epi2me-labs/wf-16s
  userName              : epi2mewsl
  profile               : standard
  configFiles           : /mnt/c/Users/joyle/epi2melabs/workflows/epi2me-labs/wf-16s/nextflow.config
Input Options
  fastq                 : /mnt/d/Raw sequencing files/16S sequencing/MinION study2 Jan 2024/fastq_pass
  classifier            : kraken2
Reference Options
  store_dir             : /mnt/c/Users/joyle/epi2melabs/data
  database_sets         : [ncbi_16s_18s:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_16s_18s.fna, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_kraken2.tar.gz, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ref2taxid.targloci.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], ncbi_16s_18s_28s_ITS:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS.fna, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS_kraken2.tar.gz, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ref2taxid.ncbi_16s_18s_28s_ITS.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], SILVA_138_1:[database:null]]
  taxonomic_rank        : S
Kraken2 Options
  kraken2_memory_mapping: true
Output Options
  out_dir               : /mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HMGK5TFW3HZ981TXFF6DWKRB/output
Advanced Options
  min_len               : 100
  max_len               : 1000
!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-16s for your analysis please cite:
* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x
--------------------------------------------------------------------------------
This is epi2me-labs/wf-16s v1.0.0.
--------------------------------------------------------------------------------
Checking inputs.
Kraken2 pipeline.
Preparing databases.
Using default taxonomy database.
Unpacking kraken2 indexes
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
[skipping] Stored process > prepare_databases:download_unpack_taxonomy
[skipping] Stored process > prepare_databases:unpack_download_kraken2_database
[skipping] Stored process > prepare_databases:determine_bracken_length
[21/af9eba] Submitted process > fastcat (4)
[01/9061aa] Submitted process > kraken_pipeline:run_common:getVersions
[48/840089] Submitted process > fastcat (2)
[17/077df3] Submitted process > kraken_pipeline:run_common:getParams
ERROR ~ Consider to use --kraken2_memory_mapping to reduce the use of RAM memory.
 -- Check '/mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HMGK5TFW3HZ981TXFF6DWKRB/nextflow.log' file for details
ERROR ~ Error executing process > 'kraken_pipeline:run_kraken2 (1)'
Caused by:
  Process requirement exceeds available memory -- req: 4 GB; avail: 3.7 GB
Command executed:
  kraken2 --db ncbi_targeted_loci_kraken2_db seqs.fastq.gz    --threads 4     --report "barcode04.kraken2.report.txt"    --confidence 0 --memory-mapping > "barcode04.kraken2.assignments.tsv"
Command exit status:
  -
Command output:
  (empty)
Work dir:
  /mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HMGK5TFW3HZ981TXFF6DWKRB/work/dc/8b41b73ac505f78f67d67ced07758a
Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`
 -- Check '/mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HMGK5TFW3HZ981TXFF6DWKRB/nextflow.log' file for details
WARN: Killing running tasks (2)

Application activity log entry

No response

nggvs commented 10 months ago

Hi @joyleng , Apologies beforehand, this has been a reported issue and hopefully they will be a release soon to fix it.

joyleng commented 10 months ago

Hi Ok, thanks for letting me know. Good to know I am not the only one :)

nggvs commented 9 months ago

Hi, I'll let you know once it is solved! Thank you for the patience!

nggvs commented 9 months ago

Hi @joyleng , Apologies for the delay but the newest release 2.9.0 should solve this problem! Let me know if you find something else. Thank you for using the workflow!

joyleng commented 8 months ago

Hi

Sorry I haven't tested this sooner. I have just tried to run it with my sequencing files in the same folder/file structure as I used before when I could analyse samples but I don't think it is picking up my files anymore. I have pointed it to a folder which contains a folder called barcode1 (I just put one in there for test purposes) and this then contains the compressed fastq files. Has the folder structure required changed?

I have pasted the error below: N E X T F L O W ~ version 23.04.2 Launching /mnt/c/Users/joyle/epi2melabs/workflows/epi2me-labs/wf-16s/main.nf [sleepy_kalam] DSL2 - revision: 0f29a0d8f6 WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES ERROR ~ Validation of pipeline parameters failed! -- Check '/mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HQB500H4X3DMYX9MEQK52Y8X/nextflow.log' file for details ERROR ~ Missing required parameter: --fastq -- Check '/mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HQB500H4X3DMYX9MEQK52Y8X/nextflow.log' file for details ERROR ~ Missing required parameter: --classifier -- Check '/mnt/c/Users/joyle/epi2melabs/instances/wf-16s_01HQB500H4X3DMYX9MEQK52Y8X/nextflow.log' file for details

nggvs commented 8 months ago

Hi @joyleng , would you mind to open this as a different issue? It sounds like something different and could be helpful for others having it in a different one. Thank you very much in advance!

nggvs commented 8 months ago

Hi @joyleng , Is the initial issue already solved? If that is the answer, would you mind to close the issue? Thank you very much in advance

peradastra commented 7 months ago

I am still having this issue. 

(base) [hilaire@ad.bcm.edu@rpv-oitghp-p02 wf-meta]$ conda activate nextflow
(nextflow) [hilaire@ad.bcm.edu@rpv-oitghp-p02 wf-meta]$ bash wf-meta.sh 
N E X T F L O W  ~  version 23.10.1
Launching `https://github.com/epi2me-labs/wf-metagenomics` [tiny_woese] DSL2 - revision: 6636bc9044 [master]
WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-metagenomics v2.9.3-g6636bc9
--------------------------------------------------------------------------------
Core Nextflow options
  revision                   : master
  runName                    : tiny_woese
  containerEngine            : docker
  container                  : [withLabel:wfmetagenomics:ontresearch/wf-metagenomics:sha44a6dacff5f2001d917b774647bb4cbc1b53bc76, withLabel:wf_common:ontresearch/wf-common:sha645176f98b8780851f9c476a064d44c2ae76ddf6, withLabel:amr:ontresearch/abricate:sha2c763f19fac46035437854f1e2a5f05553542a78]
  launchDir                  : /home/ad.bcm.edu/hilaire/wf-meta
  workDir                    : /home/ad.bcm.edu/hilaire/wf-meta/work
  projectDir                 : /home/ad.bcm.edu/hilaire/.nextflow/assets/epi2me-labs/wf-metagenomics
  userName                   : hilaire@ad.bcm.edu
  profile                    : standard
  configFiles                : /home/ad.bcm.edu/hilaire/.nextflow/assets/epi2me-labs/wf-metagenomics/nextflow.config

Input Options
  fastq                      : seqid_fastqs

Sample Options
  sample_sheet               : 032824MC110414FAX26329.csv

Reference Options
  database                   : /mnt/scratch/k2_nt_20231129/
  database_sets              : [ncbi_16s_18s:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_16s_18s.fna, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_kraken2.tar.gz, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ref2taxid.targloci.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], ncbi_16s_18s_28s_ITS:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS.fna, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS_kraken2.tar.gz, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ref2taxid.ncbi_16s_18s_28s_ITS.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], SILVA_138_1:[database:null], Standard-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_standard_08gb_20231009.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip], PlusPF-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_pluspf_08gb_20230314.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip], PlusPFP-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_pluspfp_08gb_20230314.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip]]

Kraken2 Options
  kraken2_memory_mapping     : true
  include_kraken2_assignments: true

Advanced Options
  threads                    : 24

Miscellaneous Options
  disable_ping               : true

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-metagenomics for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-metagenomics v2.9.3-g6636bc9.
--------------------------------------------------------------------------------
Checking inputs.
Note: Reference/Database are custom.
Note: Memory available to the workflow must be slightly higher than size of the database custom index.
Note: Or consider to use the --kraken2_memory_mapping.
Note: Memory available to the workflow must be slightly higher than size of the database Standard-8 index (8GB) or consider to use --kraken2_memory_mapping
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
[-        ] process > validate_sample_sheet -
[-        ] process > validate_sample_sheet                           -
executor >  local (3)executor >  local (3)
executor >  local (3)
executor >  local (4)
executor >  local (4)
executor >  local (5)
executor >  local (5)
executor >  local (19)
executor >  local (19)
executor >  local (19)
executor >  local (19)
[65/99a592] process > validate_sample_sheet                           [100%] 1 of 1 ✔[fe/c79f0f] process > fastcat (6)                                     [  7%] 1 of 14
[skipped  ] process > prepare_databases:download_unpack_taxonomy      [100%] 1 of 1, stored: 1 ✔[skipped  ] process > prepare_databases:determine_bracken_length      [100%] 1 of 1, stored: 1 ✔[7b/29b507] process > kraken_pipeline:run_common:getVersions          [100%] 1 of 1 ✔[f8/40e426] process > kraken_pipeline:run_common:getParams            [100%] 1 of 1 ✔[-        ] process > kraken_pipeline:run_kraken2                     [  0%] 0 of 1executor >  local (19)[65/99a592] process > validate_sample_sheet                           [100%] 1 of 1 ✔[64/49c7f2] process > fastcat (10)                                    [100%] 1 of 1[skipped  ] process > prepare_databases:download_unpack_taxonomy      [100%] 1 of 1, stored: 1 ✔[skipped  ] process > prepare_databases:determine_bracken_length      [100%] 1 of 1, stored: 1 ✔[7b/29b507] process > kraken_pipeline:run_common:getVersions          [100%] 1 of 1 ✔[f8/40e426] process > kraken_pipeline:run_common:getParams            [100%] 1 of 1 ✔
[15/9d045a] process > kraken_pipeline:run_kraken2 (1123SEQID066-N029) [100%] 1 of 1, failed: 1
[-        ] process > kraken_pipeline:run_bracken                     -
[-        ] process > kraken_pipeline:createAbundanceTables           -
[-        ] process > kraken_pipeline:makeReport                      -
[-        ] process > kraken_pipeline:output_kraken2_read_assignments -
[5f/950773] process > kraken_pipeline:output_results (2)              [100%] 2 of 2
Note: Empty files or those files whose reads have been discarded after filtering based on read length and/or read quality will not appear in the report and will be excluded from subsequent analysis.
Kraken2 pipeline.
Preparing databases.
Using default taxonomy database.
Checking custom kraken2 database exists
Using the bracken dist file within your custom database directory.
[skipping] Stored process > prepare_databases:download_unpack_taxonomy
[skipping] Stored process > prepare_databases:determine_bracken_length
ERROR ~ Consider to use --kraken2_memory_mapping to reduce the use of RAM memory.

 -- Check '.nextflow.log' file for detailsERROR ~ Error executing process > 'kraken_pipeline:run_kraken2 (1123SEQID066-N029)'

Caused by:
  Process requirement exceeds available memory -- req: 712.6 GB; avail: 502.8 GB

Command executed:

  kraken2 --db k2_nt_20231129 seqs.fastq.gz         --threads 24         --report "1123SEQID066-N029.kraken2.report.txt"         --confidence 0 --memory-mapping > "1123SEQID066-N029.kraken2.assignments.tsv"

Command exit status:
  -

Command output:
  (empty)

Work dir:
  /home/ad.bcm.edu/hilaire/wf-meta/work/15/9d045aad227403758ff86e23ffc723

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

 -- Check '.nextflow.log' file for details
nggvs commented 5 months ago

Hi @peradastra , I have realized now that you have commented here. I apologize, is the problem solved or are you still experiencing it? If yes, would you mind to open a new issue?