epi2me-labs / wf-metagenomics

Metagenomic classification of long-read sequencing data
Other
55 stars 23 forks source link

wf-metagenomic: Minimap error 16S_18S NCBI database #64

Closed EsbergA closed 1 year ago

EsbergA commented 1 year ago

Operating System

macOS

Other Linux

No response

Workflow Version

wf-metagenomic v2.5.0

Workflow Execution

EPI2ME Desktop application

EPI2ME Version

No response

CLI command run

{ classifier: minimap2, batch_size: 0, analyse_unclassified: false, watch_path: false, database_set: ncbi_16s_18s, store_dir: store_dir, taxonomic_rank: S, port: 8080, host: localhost, external_kraken2: false, kraken2_memory_mapping: false, include_kraken2_assignments: false, kraken2_confidence: 0, minimap2exclude: false, split_prefix: false, keep_bam: false, amr: false, amr_db: resfinder, amr_minid: 80, amr_mincov: 80, abundance_threshold: 0, n_taxa_barplot: 8, out_dir: /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/output, min_len: 1200, threads: 4, server_threads: 4, kraken_clients: 2, disable_ping: false, help: false, version: false, fastq: /Users/andersesberg/epi2melabs/demo/epi2me-labs/wf-metagenomics/wf-metagenomics-demo/Lime9, min_read_qual: 12, max_len: 1700, wf: { agent: epi2melabs/5.1.2, epi2me_instance: 01HAVTWSBWFR1X79WYJS52TPA3, epi2me_user: guest_01HA9A2WRWMYV0YYDJJ20STRSX } }

Workflow Execution - CLI Execution Profile

standard (default)

What happened?

Run stopped with error in the minimap report step, all other step worked This happens with 2 out of 9 run. Tried to re-run but this does not help. Best regards Anders

Relevant log output

N E X T F L O W  ~  version 23.04.2
Launching `/Users/andersesberg/epi2melabs/workflows/epi2me-labs/wf-metagenomics/main.nf` [keen_wescoff] DSL2 - revision: b3b2bd30a2
WARN: NEXTFLOW RECURSION IS A PREVIEW FEATURE - SYNTAX AND FUNCTIONALITY CAN CHANGE IN FUTURE RELEASES
||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-metagenomics v2.5.0
--------------------------------------------------------------------------------
Core Nextflow options
  runName        : keen_wescoff
  containerEngine: docker
  launchDir      : /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3
  workDir        : /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work
  projectDir     : /Users/andersesberg/epi2melabs/workflows/epi2me-labs/wf-metagenomics
  userName       : andersesberg
  profile        : standard
  configFiles    : /Users/andersesberg/epi2melabs/workflows/epi2me-labs/wf-metagenomics/nextflow.config
Input Options
  fastq          : /Users/andersesberg/epi2melabs/demo/epi2me-labs/wf-metagenomics/wf-metagenomics-demo/Lime9
  classifier     : minimap2
Reference Options
  database_sets  : [ncbi_16s_18s:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_16s_18s.fna, refindex:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_16s_18s.fna.fai, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_kraken2.tar.gz, kmer_dist:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/database1000mers.kmer_distrib, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ref2taxid.targloci.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], ncbi_16s_18s_28s_ITS:[reference:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS.fna, refindex:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS.fna.fai, database:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ncbi_16s_18s_28s_ITS_kraken2.tar.gz, kmer_dist:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/database1000mers.kmer_distrib, ref2taxid:https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s_28s_ITS/ref2taxid.ncbi_16s_18s_28s_ITS.tsv, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip], SILVA_138_1:[database:null], PlusPF-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_pluspf_08gb_20230314.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip], PlusPFP-8:[database:https://genome-idx.s3.amazonaws.com/kraken/k2_pluspfp_08gb_20230314.tar.gz, taxonomy:https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/new_taxdump_2023-03-01.zip]]
Output Options
  out_dir        : /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/output
Advanced Options
  min_len        : 1200
  min_read_qual  : 12
  max_len        : 1700
  threads        : 4
  server_threads : 4
!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-metagenomics for your analysis please cite:
* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x
--------------------------------------------------------------------------------
This is epi2me-labs/wf-metagenomics v2.5.0.
--------------------------------------------------------------------------------
Checking inputs.
Checking fastq input.
[69/0f4d90] Submitted process > fastcat (18)
[a4/7a0663] Submitted process > minimap_pipeline:run_common:getParams
[47/6e111d] Submitted process > minimap_pipeline:run_common:getVersions
[52/1c6066] Submitted process > fastcat (20)
[e9/c54afd] Submitted process > fastcat (3)
[95/ab0af4] Submitted process > fastcat (10)
[44/86a6ef] Submitted process > minimap_pipeline:output (1)
Staging foreign file: https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump_archive/taxdmp_2023-01-01.zip
[55/38e1ca] Submitted process > fastcat (7)
Staging foreign file: https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_16s_18s.fna
[95/4b5f40] Submitted process > minimap_pipeline:output (2)
[1c/30ed09] Submitted process > fastcat (12)
[89/fa526b] Submitted process > fastcat (5)
[71/bb3b0f] Submitted process > minimap_pipeline:check_reference_ref2taxid
[91/49e8cf] Submitted process > fastcat (17)
[d9/d1fd28] Submitted process > fastcat (6)
[02/6088b3] Submitted process > fastcat (14)
[b4/d9eef9] Submitted process > minimap_pipeline:unpackTaxonomy
[5b/64f666] Submitted process > fastcat (4)
[e0/80ed22] Submitted process > fastcat (11)
[f8/c85484] Submitted process > fastcat (15)
[77/e6fef0] Submitted process > fastcat (1)
[96/52083d] Submitted process > fastcat (19)
[32/acf8e5] Submitted process > fastcat (16)
[0a/10a78b] Submitted process > fastcat (13)
[d1/bd6a9d] Submitted process > fastcat (33)
Staging foreign file: https://ont-exd-int-s3-euwst1-epi2me-labs.s3.amazonaws.com/wf-metagenomics/ncbi_16s_18s/ncbi_targeted_loci_16s_18s.fna.fai
[fd/6359a1] Submitted process > fastcat (2)
[be/16ed5f] Submitted process > fastcat (8)
[65/c6d28e] Submitted process > fastcat (9)
[0c/d00d32] Submitted process > fastcat (39)
[01/ee7215] Submitted process > fastcat (21)
[7e/85859a] Submitted process > fastcat (22)
[57/930937] Submitted process > fastcat (23)
[2a/b4283a] Submitted process > fastcat (24)
[e3/9509ac] Submitted process > fastcat (25)
[9e/003d10] Submitted process > fastcat (26)
[79/8f8f00] Submitted process > fastcat (27)
[f7/13cb3b] Submitted process > fastcat (28)
[85/1fe70c] Submitted process > fastcat (29)
[7f/282f83] Submitted process > fastcat (30)
[b0/2ea7ad] Submitted process > fastcat (31)
[59/6c3aeb] Submitted process > fastcat (32)
[ae/ebf283] Submitted process > fastcat (34)
[a0/53506c] Submitted process > fastcat (36)
[59/c64d28] Submitted process > fastcat (35)
[44/98f540] Submitted process > fastcat (38)
[fc/d26a02] Submitted process > fastcat (52)
[b2/79826d] Submitted process > fastcat (37)
[b6/fd7734] Submitted process > fastcat (56)
[e5/32a5d7] Submitted process > fastcat (54)
[e9/22481a] Submitted process > fastcat (59)
[b0/ee4d8e] Submitted process > fastcat (58)
[39/9d85d6] Submitted process > fastcat (40)
[12/d78d70] Submitted process > fastcat (41)
[1f/36214f] Submitted process > fastcat (42)
[9d/680421] Submitted process > fastcat (44)
[81/efbaf3] Submitted process > fastcat (45)
[3e/a960f0] Submitted process > fastcat (64)
[e9/3d1e12] Submitted process > fastcat (60)
[3f/ab5e4f] Submitted process > fastcat (46)
[b5/bfd157] Submitted process > fastcat (70)
[35/25aeb0] Submitted process > fastcat (43)
[e0/00a9a6] Submitted process > fastcat (47)
[59/07c980] Submitted process > fastcat (48)
[f2/70b008] Submitted process > fastcat (49)
[05/105a91] Submitted process > fastcat (51)
[cb/9d25ba] Submitted process > fastcat (50)
[80/032fbd] Submitted process > fastcat (53)
[c2/a248c6] Submitted process > fastcat (55)
[f1/68038a] Submitted process > fastcat (57)
[db/147792] Submitted process > fastcat (78)
[3c/4fd65b] Submitted process > fastcat (80)
[18/2ebe4c] Submitted process > fastcat (63)
[1c/240629] Submitted process > fastcat (65)
[4f/438320] Submitted process > fastcat (84)
[41/f537db] Submitted process > fastcat (62)
[00/6d0de2] Submitted process > fastcat (61)
[f0/f18862] Submitted process > fastcat (66)
[a2/6e2f4c] Submitted process > fastcat (67)
[88/356699] Submitted process > fastcat (89)
[6b/a80423] Submitted process > fastcat (68)
[77/27e63a] Submitted process > fastcat (69)
[c8/4ecd7d] Submitted process > fastcat (71)
[4b/bf3e4b] Submitted process > fastcat (72)
[47/d72ace] Submitted process > fastcat (93)
[2c/5b9be0] Submitted process > fastcat (73)
[cf/79cf94] Submitted process > fastcat (75)
[d3/dd942b] Submitted process > fastcat (74)
[c6/2343fe] Submitted process > fastcat (76)
[ea/8de8b5] Submitted process > fastcat (79)
[88/d2b6d4] Submitted process > fastcat (96)
[36/99ec06] Submitted process > fastcat (77)
[ab/fdec37] Submitted process > fastcat (81)
[8c/f0f417] Submitted process > fastcat (83)
[d8/4c051b] Submitted process > fastcat (85)
[06/3ccc6f] Submitted process > fastcat (82)
[e9/b042d0] Submitted process > fastcat (87)
[3a/100fca] Submitted process > fastcat (86)
[8c/4eef89] Submitted process > fastcat (90)
[f6/181acc] Submitted process > fastcat (88)
[61/7a6f6a] Submitted process > fastcat (91)
[e6/5d72a8] Submitted process > fastcat (92)
[fa/8d54e9] Submitted process > fastcat (94)
[3d/24936c] Submitted process > fastcat (95)
[04/35fb62] Submitted process > minimap_pipeline:minimap (13)
[cc/1d81b6] Submitted process > minimap_pipeline:minimap (14)
[35/1bbb10] Submitted process > minimap_pipeline:minimap (11)
[a3/44ff71] Submitted process > minimap_pipeline:minimap (9)
[a6/7eae0e] Submitted process > minimap_pipeline:minimap (6)
[ed/d319fe] Submitted process > minimap_pipeline:minimap (3)
[fc/a59304] Submitted process > minimap_pipeline:minimap (10)
[6f/1b11ca] Submitted process > minimap_pipeline:minimap (8)
[fe/f18f64] Submitted process > minimap_pipeline:minimap (12)
[df/d54fce] Submitted process > minimap_pipeline:minimap (5)
[5e/3af0c9] Submitted process > minimap_pipeline:minimap (7)
[54/318dd8] Submitted process > minimap_pipeline:minimap (1)
[d7/16de22] Submitted process > minimap_pipeline:minimap (15)
[fb/0984dd] Submitted process > minimap_pipeline:minimap (2)
[74/090417] Submitted process > minimap_pipeline:minimap (4)
[53/b58446] Submitted process > minimap_pipeline:minimap (16)
[26/0ce04b] Submitted process > minimap_pipeline:minimap (17)
[95/de9b56] Submitted process > minimap_pipeline:minimap (18)
[f1/4ec8e3] Submitted process > minimap_pipeline:minimap (19)
[50/c004a4] Submitted process > minimap_pipeline:minimap (20)
[17/3b2368] Submitted process > minimap_pipeline:minimap (21)
[4c/34da8c] Submitted process > minimap_pipeline:minimap (22)
[86/c59145] Submitted process > minimap_pipeline:minimap (23)
[d7/979d0d] Submitted process > minimap_pipeline:minimap (24)
[57/d5ac25] Submitted process > minimap_pipeline:minimap (25)
[83/17c454] Submitted process > minimap_pipeline:minimap (26)
[22/d37929] Submitted process > minimap_pipeline:minimap (27)
[24/5bdff7] Submitted process > minimap_pipeline:minimap (28)
[07/e0f652] Submitted process > minimap_pipeline:minimap (29)
[02/6442d3] Submitted process > minimap_pipeline:minimap (30)
[c4/db61ed] Submitted process > minimap_pipeline:minimap (31)
[28/3ef3d5] Submitted process > minimap_pipeline:minimap (32)
[09/3b20e5] Submitted process > minimap_pipeline:minimap (33)
[80/335f8c] Submitted process > minimap_pipeline:minimap (34)
[c1/871f1a] Submitted process > minimap_pipeline:minimap (35)
[23/b5c28f] Submitted process > minimap_pipeline:minimap (36)
[03/81fcaa] Submitted process > minimap_pipeline:minimap (37)
[51/ac6374] Submitted process > minimap_pipeline:minimap (38)
[a5/41870c] Submitted process > minimap_pipeline:minimap (39)
[13/cb4d0b] Submitted process > minimap_pipeline:minimap (40)
[aa/7c8b2a] Submitted process > minimap_pipeline:minimap (41)
[6e/d61562] Submitted process > minimap_pipeline:minimap (42)
[86/bc7359] Submitted process > minimap_pipeline:minimap (43)
[1e/a073e6] Submitted process > minimap_pipeline:minimap (44)
[fa/a4c140] Submitted process > minimap_pipeline:minimap (45)
[94/316d8b] Submitted process > minimap_pipeline:minimap (46)
[b0/db4e0b] Submitted process > minimap_pipeline:minimap (47)
[a2/a5b1d5] Submitted process > minimap_pipeline:minimap (48)
[ec/20f985] Submitted process > minimap_pipeline:minimap (49)
[00/bb40c8] Submitted process > minimap_pipeline:minimap (50)
[49/1c039d] Submitted process > minimap_pipeline:minimap (51)
[1c/008344] Submitted process > minimap_pipeline:minimap (52)
[ef/d40e44] Submitted process > minimap_pipeline:minimap (53)
[15/4f3566] Submitted process > minimap_pipeline:minimap (54)
[35/4aa2e8] Submitted process > minimap_pipeline:minimap (55)
[6b/53d86e] Submitted process > minimap_pipeline:minimap (56)
[c6/3e33e1] Submitted process > minimap_pipeline:minimap (57)
[4d/ea11a5] Submitted process > minimap_pipeline:minimap (58)
[03/74ceda] Submitted process > minimap_pipeline:minimap (59)
[64/336afd] Submitted process > minimap_pipeline:minimap (60)
[20/8cd16f] Submitted process > minimap_pipeline:minimap (61)
[c1/102d4f] Submitted process > minimap_pipeline:minimap (62)
[7a/dd2494] Submitted process > minimap_pipeline:minimap (63)
[e9/b41290] Submitted process > minimap_pipeline:minimap (64)
[48/56dcf5] Submitted process > minimap_pipeline:minimap (65)
[98/6ac374] Submitted process > minimap_pipeline:minimap (66)
[9d/dba732] Submitted process > minimap_pipeline:minimap (67)
[01/11fb60] Submitted process > minimap_pipeline:minimap (68)
[90/eb5c0a] Submitted process > minimap_pipeline:minimap (69)
[c4/6ed7bc] Submitted process > minimap_pipeline:minimap (70)
[ed/667dda] Submitted process > minimap_pipeline:minimap (71)
[55/da0d71] Submitted process > minimap_pipeline:minimap (72)
[1c/92450d] Submitted process > minimap_pipeline:minimap (73)
[05/d9a217] Submitted process > minimap_pipeline:minimap (74)
[1d/9b33fa] Submitted process > minimap_pipeline:minimap (75)
[68/04ae4f] Submitted process > minimap_pipeline:minimap (76)
[8b/beeeb0] Submitted process > minimap_pipeline:minimap (77)
[6a/6073a6] Submitted process > minimap_pipeline:minimap (78)
[33/3cea8f] Submitted process > minimap_pipeline:minimap (79)
[d6/6a64da] Submitted process > minimap_pipeline:minimap (80)
[61/f6db3b] Submitted process > minimap_pipeline:minimap (81)
[e5/afca48] Submitted process > minimap_pipeline:minimap (82)
[f9/456e66] Submitted process > minimap_pipeline:minimap (83)
[b1/5f580a] Submitted process > minimap_pipeline:minimap (84)
[9f/77faf4] Submitted process > minimap_pipeline:minimap (85)
[34/ef8532] Submitted process > minimap_pipeline:minimap (86)
[80/50a76f] Submitted process > minimap_pipeline:minimap (87)
[6b/58f1dd] Submitted process > minimap_pipeline:minimap (88)
[fc/dd7e57] Submitted process > minimap_pipeline:minimap (89)
[eb/b6bb54] Submitted process > minimap_pipeline:minimap (90)
[fd/9048bb] Submitted process > minimap_pipeline:minimap (91)
[94/3b17ce] Submitted process > minimap_pipeline:minimap (92)
[eb/ef9803] Submitted process > minimap_pipeline:minimap (93)
[76/3458ea] Submitted process > minimap_pipeline:minimap (94)
[fd/95f5fb] Submitted process > minimap_pipeline:minimap (95)
[73/353747] Submitted process > minimap_pipeline:minimap (96)
[9a/b9812e] Submitted process > minimap_pipeline:makeReport (1)
ERROR ~ Error executing process > 'minimap_pipeline:makeReport (1)'
Caused by:
  Process `minimap_pipeline:makeReport (1)` terminated with an error exit status (1)
Command executed:
  workflow-glue report         "wf-metagenomics-report.html"         --workflow_name wf-metagenomics         --versions versions         --params params.json         --stats per-read-stats.tsv         --lineages lineages         --taxonomic_rank "S"         --pipeline "minimap"         --abundance_threshold "0"        --n_taxa_barplot "8"
Command exit status:
  1
Command output:
  (empty)
Command error:
  [22:54:38 - matplotlib.font_manager] generated new fontManager
  [22:54:39 - workflow_glue] Starting entrypoint.
  Traceback (most recent call last):
    File "/Users/andersesberg/epi2melabs/workflows/epi2me-labs/wf-metagenomics/bin/workflow-glue", line 7, in <module>
      cli()
    File "/Users/andersesberg/epi2melabs/workflows/epi2me-labs/wf-metagenomics/bin/workflow_glue/__init__.py", line 72, in cli
      args.func(args)
    File "/Users/andersesberg/epi2melabs/workflows/epi2me-labs/wf-metagenomics/bin/workflow_glue/report.py", line 179, in main
      plt = ezc.sunburst(
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/plots/sunburst.py", line 62, in sunburst
      if data[0].get('value'):
  IndexError: list index out of range
Work dir:
  /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/9a/b9812e3a3d4426f3e6a1e5644776c1
Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line
 -- Check '/Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/nextflow.log' file for details

Application activity log entry

No response

nggvs commented 1 year ago

Hi! Thank you for using the workflow! It may be something related with your data. Could you send me the next files so that I can investigate what is happening.

In the EPI2ME Desktop application go to "Track Analyses" (in the dashboard), choose the one that has been "stopped with Error" and click on "Open folder", then move to the directory:work/9a/b9812e3a3d4426f3e6a1e5644776c1. It should be a folder called 'lineages'

Could you send me the content of that folder? You can zip it to send it through GitHub

EsbergA commented 1 year ago

Thank you for a quick answer. Attached is the Lineages / Anders

Från: Natalia Garcia Garcia @.> Datum: fredag, 22 september 2023 09:54 Till: epi2me-labs/wf-metagenomics @.> Kopia: Anders Esberg @.>, Author @.> Ämne: Re: [epi2me-labs/wf-metagenomics] wf-metagenomic: Minimap error 16S_18S NCBI database (Issue #64)

Hi! Thank you for using the workflow! It may be something related with your data. Could you send me the next files so that I can investigate what is happening.

In the EPI2ME Desktop application go to "Track Analyses" (in the dashboard), choose the one that has been "stopped with Error" and click on "Open folder", then move to the directory: work/9a/b9812e3a3d4426f3e6a1e5644776c1. It should be a folder called 'lineages'

Could you send me the content of that folder? You can zip it to send it through GitHub

— Reply to this email directly, view it on GitHubhttps://github.com/epi2me-labs/wf-metagenomics/issues/64#issuecomment-1730970845, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCRRRSDNPVJ77N37VZYAMWLX3U73ZANCNFSM6AAAAAA5CT6LYM. You are receiving this because you authored the thread.Message ID: @.***>

nggvs commented 1 year ago

Apologies, but I don't see any attached file

EsbergA commented 1 year ago

lineages.zip

EsbergA commented 1 year ago

Hi again Were you able to identify the issue? I tried the wf-16S work-flow with the same issue. / A

nggvs commented 1 year ago

Hi, Apologies but I haven't found it yet. I suspect that one (or more) of the files contained in the lineages folder are empty. They should contain the different lineages identified and the counts. However, I could check it because they were symbolic links. I will add something to identify the barcode which is failing but meanwhile, could you check it or zip the files:

barcode90.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/61/f6db3bb23b39d0118054f2e3d597a3/barcode90.json barcode86.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/ed/d319fefded353a7b4c3c2c68a552bc/barcode86.json barcode73.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/5e/3af0c97cd3cd74e7736ca415bae1fd/barcode73.json barcode69.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/28/3ef3d5bae9bc0e3724023711618ce3/barcode69.json barcode53.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/eb/ef980386f0918682353ca750031ba0/barcode53.json barcode45.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/53/b5844636bf290b0f24b7d30b61db90/barcode45.json barcode28.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/50/c004a42aa1716f862726524ed9f539/barcode28.json barcode24.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/e9/b4129010390ddfb4048793b2a2f553/barcode24.json barcode12.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/90/eb5c0a408ee1944d894a911bffd4b0/barcode12.json barcode04.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fa/a4c14097955e2c964f2005435779df/barcode04.json barcode96.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/ed/667ddad9c665021ef637561b7a9925/barcode96.json barcode91.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fc/dd7e57e71f7a6a755ea7969f42ed4e/barcode91.json barcode87.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fb/0984dd6268294e3492ad890cd0c87c/barcode87.json barcode81.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/95/de9b56de36ec7c71e73ba8bc1679c5/barcode81.json barcode80.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/cc/1d81b6cc144e5353121e34e31eacc6/barcode80.json barcode79.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/55/da0d714f266c503e2ea65507763324/barcode79.json barcode78.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/01/11fb60e0648a6d187796f607c37c97/barcode78.json barcode75.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/04/35fb629e0ad841b0a8175bf3956484/barcode75.json barcode72.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/35/1bbb1018e17913aa5182ecf677d9da/barcode72.json barcode68.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6b/53d86edad6642d82f9d11d5978f894/barcode68.json barcode65.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6b/58f1dd5a4d04f26209ca1feeddfa27/barcode65.json barcode64.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c1/102d4f2001f2615ccd25726ec743cf/barcode64.json barcode63.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/f9/456e66f5185dcfc1043f2487b0cdbd/barcode63.json barcode59.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/00/bb40c87f282c624f257ce78c225e27/barcode59.json barcode58.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/80/335f8c02e68f4c0f5c87fd70bcc2a9/barcode58.json barcode55.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/e5/afca48eccebfdce98a31781f23c073/barcode55.json barcode54.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/94/3b17ce4f35aade888e119d794dd3d3/barcode54.json barcode52.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/d6/6a64dabb1705f19836f0adf07caa14/barcode52.json barcode49.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/1e/a073e64ff5e84cf4212f29df924b51/barcode49.json barcode48.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/68/04ae4f091ba7efdbafc91dcb7f9ebf/barcode48.json barcode44.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6f/1b11caacf5c1c012a6f2b3c7ede527/barcode44.json barcode43.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fc/a59304b0523610bda45404bb0888af/barcode43.json barcode42.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/a3/44ff7111f9b13d09e3f2672fcfb877/barcode42.json barcode39.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/80/50a76f4b0f43d25dcb1d153582159d/barcode39.json barcode38.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6a/6073a698c2fa32c604fc6726cbc9e5/barcode38.json barcode34.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/07/e0f652fe2ce3569d6a4d72ac38fed7/barcode34.json barcode33.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/02/6442d3c56ef1bf43eb86a8e9ea7a77/barcode33.json barcode32.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/a2/a5b1d51e2ac6a479fef9b8ea0bd457/barcode32.json barcode29.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/22/d37929247a34470fe7ab8996d76f11/barcode29.json barcode25.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6e/d61562bdffe172e768c9ad14bd928e/barcode25.json barcode22.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/7a/dd2494ad8ff9f3bb067ac5ec9cde96/barcode22.json barcode18.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/d7/979d0dcaa6cde93fb101cbd688bea9/barcode18.json barcode15.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/98/6ac37421617e573f3624da0913e3aa/barcode15.json barcode14.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/51/ac63747d3db7fcd056e5cc64344857/barcode14.json barcode13.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/4d/ea11a58589f22f6c591d55dc289b01/barcode13.json barcode09.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/73/353747b6458d7f06ca5c27819d5566/barcode09.json barcode08.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/eb/b6bb545e69af6a540aed01b7b5117f/barcode08.json barcode05.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c4/db61ed388e6675af591ea0eb8f8bde/barcode05.json barcode03.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/4c/34da8c5bbb8bf7f06f2ebff466565c/barcode03.json barcode02.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/09/3b20e56dd162ebfceb6fb4111737c6/barcode02.json barcode95.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/b0/db4e0be15f9f12eae4e114a60d38f2/barcode95.json barcode94.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/a5/41870c1373d06adac4fe79e8c245e7/barcode94.json barcode93.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c1/871f1adcda8e02aaf59c70bcda4fa9/barcode93.json barcode92.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/49/1c039d286b05b35f1f7e9c0bee9b15/barcode92.json barcode89.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fe/f18f6418b3465e3d9926c73dea1d8f/barcode89.json barcode88.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/74/09041775532d8a7e4ae1d7f8317fed/barcode88.json barcode85.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/64/336afd252de3e5ad033d2c0fc76504/barcode85.json barcode84.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/33/3cea8f10d477a39f05a519b90f0cf4/barcode84.json barcode83.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/ef/d40e44e424ec1a00b8f33f0887a90c/barcode83.json barcode82.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/86/bc73592172d400713cf32df29262ed/barcode82.json barcode77.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/05/d9a217b9960a440c1cf3dc78cde96e/barcode77.json barcode76.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/20/8cd16f77811e37997782a4d224b099/barcode76.json barcode74.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/54/318dd8e66a93d59f8fb08174b59772/barcode74.json barcode71.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/aa/7c8b2a2c6109c7a1ee051e3c301d42/barcode71.json barcode70.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/8b/beeeb087bfc1b90e1d8b703407bba6/barcode70.json barcode67.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/23/b5c28fa34510153a49e124979c20c1/barcode67.json barcode66.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/35/4aa2e83fd1b51dbd50b953d0ed59a4/barcode66.json barcode62.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fd/9048bb5b2cffa7ec95482f51ac952f/barcode62.json barcode61.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/15/4f3566e4e2539055cea255f13d07f4/barcode61.json barcode60.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/13/cb4d0b361491ca273cae20465b6daa/barcode60.json barcode57.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/03/74ceda9c9d49cd1675ef922aecc2af/barcode57.json barcode56.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/03/81fcaaf736df8626501cce8e09f17d/barcode56.json barcode51.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/f1/4ec8e304ca588b1344b625bba9009c/barcode51.json barcode50.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c6/3e33e1da0815594571b51973c4a79b/barcode50.json barcode47.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c4/6ed7bc78b383aad6f65b790ac04b55/barcode47.json barcode46.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/1c/92450d07e308c1f391e9739386d566/barcode46.json barcode41.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/1d/9b33fa86ec50c78ac737fde362a3d1/barcode41.json barcode40.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/1c/008344ec69632ca9cef084453551eb/barcode40.json barcode37.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/9f/77faf44b43053827de11490d922458/barcode37.json barcode36.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fd/95f5fbab00748048c873217646777e/barcode36.json barcode35.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/94/316d8b31bfaa1728ad12ca0b133d0c/barcode35.json barcode31.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/b1/5f580a8bf1fa2facff28b7ba4ac3d3/barcode31.json barcode30.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/48/56dcf55f01d32ef5c0d0f289c61414/barcode30.json barcode27.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/57/d5ac254a439dff833fc5829d56012a/barcode27.json barcode26.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/a6/7eae0e2215e3fff28c91bb22e1f87c/barcode26.json barcode23.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/ec/20f985fbe1800c24a1491bca68fe8e/barcode23.json barcode21.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/26/0ce04b1ef19e87767bf1c42b673ca0/barcode21.json barcode20.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/86/c591458aafc31cb1b44f32fbd7b921/barcode20.json barcode19.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/d7/16de22801cc813a18d5bf5d680b77a/barcode19.json barcode17.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/17/3b23681fbbaf2be8fc7b0d315bef0b/barcode17.json barcode16.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/24/5bdff7663b09c2a61e90c5b7f6014d/barcode16.json barcode11.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/83/17c4544b0b5fc6d22a785cb34269c3/barcode11.json barcode10.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/df/d54fced364361daea4b88f7cc0fc08/barcode10.json barcode06.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/9d/dba732dd965c6ab9bedfc4268be0e8/barcode06.json barcode01.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/34/ef85323b2264f0cd197e44a185bfc4/barcode01.json barcode07.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/76/3458ea0f4fae5d1d6451b08945a07d/barcode07.json

EsbergA commented 1 year ago

Ok, this could make sense, as some folders may be totally empty, I will look through the folders. I run 96 barcoded samples at the time including negative and positive samples and some were not used. /A

Skickat från Outlook för iOShttps://aka.ms/o0ukef


Från: Natalia Garcia Garcia @.> Skickat: Monday, September 25, 2023 10:19:31 AM Till: epi2me-labs/wf-metagenomics @.> Kopia: Anders Esberg @.>; Author @.> Ämne: Re: [epi2me-labs/wf-metagenomics] wf-metagenomic: Minimap error 16S_18S NCBI database (Issue #64)

Hi, Apologies but I haven't found it yet. I suspect that one of the files contained in the lineages folder is empty. However, I could check it because they were symbolic links. I will add something to identify the barcode which is failing but meanwhile, could you check it or zip the files:

barcode90.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/61/f6db3bb23b39d0118054f2e3d597a3/barcode90.json barcode86.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/ed/d319fefded353a7b4c3c2c68a552bc/barcode86.json barcode73.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/5e/3af0c97cd3cd74e7736ca415bae1fd/barcode73.json barcode69.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/28/3ef3d5bae9bc0e3724023711618ce3/barcode69.json barcode53.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/eb/ef980386f0918682353ca750031ba0/barcode53.json barcode45.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/53/b5844636bf290b0f24b7d30b61db90/barcode45.json barcode28.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/50/c004a42aa1716f862726524ed9f539/barcode28.json barcode24.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/e9/b4129010390ddfb4048793b2a2f553/barcode24.json barcode12.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/90/eb5c0a408ee1944d894a911bffd4b0/barcode12.json barcode04.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fa/a4c14097955e2c964f2005435779df/barcode04.json barcode96.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/ed/667ddad9c665021ef637561b7a9925/barcode96.json barcode91.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fc/dd7e57e71f7a6a755ea7969f42ed4e/barcode91.json barcode87.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fb/0984dd6268294e3492ad890cd0c87c/barcode87.json barcode81.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/95/de9b56de36ec7c71e73ba8bc1679c5/barcode81.json barcode80.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/cc/1d81b6cc144e5353121e34e31eacc6/barcode80.json barcode79.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/55/da0d714f266c503e2ea65507763324/barcode79.json barcode78.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/01/11fb60e0648a6d187796f607c37c97/barcode78.json barcode75.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/04/35fb629e0ad841b0a8175bf3956484/barcode75.json barcode72.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/35/1bbb1018e17913aa5182ecf677d9da/barcode72.json barcode68.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6b/53d86edad6642d82f9d11d5978f894/barcode68.json barcode65.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6b/58f1dd5a4d04f26209ca1feeddfa27/barcode65.json barcode64.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c1/102d4f2001f2615ccd25726ec743cf/barcode64.json barcode63.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/f9/456e66f5185dcfc1043f2487b0cdbd/barcode63.json barcode59.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/00/bb40c87f282c624f257ce78c225e27/barcode59.json barcode58.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/80/335f8c02e68f4c0f5c87fd70bcc2a9/barcode58.json barcode55.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/e5/afca48eccebfdce98a31781f23c073/barcode55.json barcode54.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/94/3b17ce4f35aade888e119d794dd3d3/barcode54.json barcode52.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/d6/6a64dabb1705f19836f0adf07caa14/barcode52.json barcode49.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/1e/a073e64ff5e84cf4212f29df924b51/barcode49.json barcode48.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/68/04ae4f091ba7efdbafc91dcb7f9ebf/barcode48.json barcode44.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6f/1b11caacf5c1c012a6f2b3c7ede527/barcode44.json barcode43.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fc/a59304b0523610bda45404bb0888af/barcode43.json barcode42.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/a3/44ff7111f9b13d09e3f2672fcfb877/barcode42.json barcode39.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/80/50a76f4b0f43d25dcb1d153582159d/barcode39.json barcode38.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6a/6073a698c2fa32c604fc6726cbc9e5/barcode38.json barcode34.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/07/e0f652fe2ce3569d6a4d72ac38fed7/barcode34.json barcode33.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/02/6442d3c56ef1bf43eb86a8e9ea7a77/barcode33.json barcode32.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/a2/a5b1d51e2ac6a479fef9b8ea0bd457/barcode32.json barcode29.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/22/d37929247a34470fe7ab8996d76f11/barcode29.json barcode25.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/6e/d61562bdffe172e768c9ad14bd928e/barcode25.json barcode22.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/7a/dd2494ad8ff9f3bb067ac5ec9cde96/barcode22.json barcode18.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/d7/979d0dcaa6cde93fb101cbd688bea9/barcode18.json barcode15.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/98/6ac37421617e573f3624da0913e3aa/barcode15.json barcode14.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/51/ac63747d3db7fcd056e5cc64344857/barcode14.json barcode13.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/4d/ea11a58589f22f6c591d55dc289b01/barcode13.json barcode09.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/73/353747b6458d7f06ca5c27819d5566/barcode09.json barcode08.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/eb/b6bb545e69af6a540aed01b7b5117f/barcode08.json barcode05.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c4/db61ed388e6675af591ea0eb8f8bde/barcode05.json barcode03.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/4c/34da8c5bbb8bf7f06f2ebff466565c/barcode03.json barcode02.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/09/3b20e56dd162ebfceb6fb4111737c6/barcode02.json barcode95.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/b0/db4e0be15f9f12eae4e114a60d38f2/barcode95.json barcode94.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/a5/41870c1373d06adac4fe79e8c245e7/barcode94.json barcode93.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c1/871f1adcda8e02aaf59c70bcda4fa9/barcode93.json barcode92.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/49/1c039d286b05b35f1f7e9c0bee9b15/barcode92.json barcode89.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fe/f18f6418b3465e3d9926c73dea1d8f/barcode89.json barcode88.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/74/09041775532d8a7e4ae1d7f8317fed/barcode88.json barcode85.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/64/336afd252de3e5ad033d2c0fc76504/barcode85.json barcode84.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/33/3cea8f10d477a39f05a519b90f0cf4/barcode84.json barcode83.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/ef/d40e44e424ec1a00b8f33f0887a90c/barcode83.json barcode82.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/86/bc73592172d400713cf32df29262ed/barcode82.json barcode77.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/05/d9a217b9960a440c1cf3dc78cde96e/barcode77.json barcode76.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/20/8cd16f77811e37997782a4d224b099/barcode76.json barcode74.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/54/318dd8e66a93d59f8fb08174b59772/barcode74.json barcode71.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/aa/7c8b2a2c6109c7a1ee051e3c301d42/barcode71.json barcode70.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/8b/beeeb087bfc1b90e1d8b703407bba6/barcode70.json barcode67.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/23/b5c28fa34510153a49e124979c20c1/barcode67.json barcode66.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/35/4aa2e83fd1b51dbd50b953d0ed59a4/barcode66.json barcode62.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fd/9048bb5b2cffa7ec95482f51ac952f/barcode62.json barcode61.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/15/4f3566e4e2539055cea255f13d07f4/barcode61.json barcode60.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/13/cb4d0b361491ca273cae20465b6daa/barcode60.json barcode57.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/03/74ceda9c9d49cd1675ef922aecc2af/barcode57.json barcode56.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/03/81fcaaf736df8626501cce8e09f17d/barcode56.json barcode51.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/f1/4ec8e304ca588b1344b625bba9009c/barcode51.json barcode50.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c6/3e33e1da0815594571b51973c4a79b/barcode50.json barcode47.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/c4/6ed7bc78b383aad6f65b790ac04b55/barcode47.json barcode46.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/1c/92450d07e308c1f391e9739386d566/barcode46.json barcode41.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/1d/9b33fa86ec50c78ac737fde362a3d1/barcode41.json barcode40.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/1c/008344ec69632ca9cef084453551eb/barcode40.json barcode37.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/9f/77faf44b43053827de11490d922458/barcode37.json barcode36.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/fd/95f5fbab00748048c873217646777e/barcode36.json barcode35.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/94/316d8b31bfaa1728ad12ca0b133d0c/barcode35.json barcode31.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/b1/5f580a8bf1fa2facff28b7ba4ac3d3/barcode31.json barcode30.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/48/56dcf55f01d32ef5c0d0f289c61414/barcode30.json barcode27.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/57/d5ac254a439dff833fc5829d56012a/barcode27.json barcode26.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/a6/7eae0e2215e3fff28c91bb22e1f87c/barcode26.json barcode23.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/ec/20f985fbe1800c24a1491bca68fe8e/barcode23.json barcode21.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/26/0ce04b1ef19e87767bf1c42b673ca0/barcode21.json barcode20.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/86/c591458aafc31cb1b44f32fbd7b921/barcode20.json barcode19.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/d7/16de22801cc813a18d5bf5d680b77a/barcode19.json barcode17.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/17/3b23681fbbaf2be8fc7b0d315bef0b/barcode17.json barcode16.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/24/5bdff7663b09c2a61e90c5b7f6014d/barcode16.json barcode11.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/83/17c4544b0b5fc6d22a785cb34269c3/barcode11.json barcode10.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/df/d54fced364361daea4b88f7cc0fc08/barcode10.json barcode06.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/9d/dba732dd965c6ab9bedfc4268be0e8/barcode06.json barcode01.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/34/ef85323b2264f0cd197e44a185bfc4/barcode01.json barcode07.json -> /Users/andersesberg/epi2melabs/instances/wf-metagenomics_01HAVTWSBWFR1X79WYJS52TPA3/work/76/3458ea0f4fae5d1d6451b08945a07d/barcode07.json

— Reply to this email directly, view it on GitHubhttps://github.com/epi2me-labs/wf-metagenomics/issues/64#issuecomment-1733151742, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCRRRSDQ7ZFUBGW26V5HXY3X4E5BHANCNFSM6AAAAAA5CT6LYM. You are receiving this because you authored the thread.Message ID: @.***>

marsfro commented 1 year ago

I had the same error. I just removed --threads from input and process started, but it's very interesting - working all threads - Miracles!

nggvs commented 1 year ago

Hi @EsbergA ! Please let me know if that solves the problem. However, it shouldn't break like that if just some barcodes are empty, so something else may be happening.

Hi @marsfro ! Thank you for using the workflow! Would you mind to send the output of the error, also are you running the workflow in the app or in the command line? In the last case, could you copy the command that you're using (also, to clarify what do you mean by removing 'threads' from the input)?

EsbergA commented 1 year ago

Removing all folders with only one tar.gz file solved the issue. This correlated to empty or very few reads assigned to the specific barcode. Thank you for the guidance

Skickat från Outlook för iOShttps://aka.ms/o0ukef


Från: Natalia Garcia Garcia @.> Skickat: Tuesday, September 26, 2023 10:34:31 AM Till: epi2me-labs/wf-metagenomics @.> Kopia: Anders Esberg @.>; Mention @.> Ämne: Re: [epi2me-labs/wf-metagenomics] wf-metagenomic: Minimap error 16S_18S NCBI database (Issue #64)

Hi @EsbergAhttps://github.com/EsbergA ! Please let me know if that solves the problem. However, it shouldn't break like that if just some barcodes are empty, so something else may be happening.

Hi @marsfrohttps://github.com/marsfro ! Thank you for using the workflow! Would you mind to send the output of the error, also are you running the workflow in the app or in the command line? In the last case, could you copy the command that you're using (also, to clarify what do you mean by removing 'threads' from the input)?

— Reply to this email directly, view it on GitHubhttps://github.com/epi2me-labs/wf-metagenomics/issues/64#issuecomment-1735079876, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BCRRRSFKHSAR4LYCU6GI7B3X4KHRPANCNFSM6AAAAAA5CT6LYM. You are receiving this because you were mentioned.Message ID: @.***>

marsfro commented 1 year ago

@nggvs Thank you for your pipeline!! I'm already tired of rushing between kraken2 and centrifuge

nextflow run epi2me-labs/wf-metagenomics --fastq /path/fastq_folders --classifier minimap2 --threads 8 - it was first run
nextflow run epi2me-labs/wf-metagenomics --fastq /path/fastq_folders --classifier minimap2 - second, without threads nextflow_2nd_log.txt nextflow_1st_log.txt

marsfro commented 1 year ago

Caused by: Process minimap_pipeline:minimap (12) terminated with an error exit status (1)

Command executed:

minimap2 -t "8" -ax map-ont "ncbi_targeted_loci_16s_18s.fna" "seqs.fastq.gz" | samtools view -h -F 2304 - | workflow-glue format_minimap2 - -o "9o.minimap2.assignments.tsv" -r "ref2taxid.targloci.tsv" | samtools sort -o "9o.bam" - samtools index "9o.bam" awk -F '\t' '{print $3}' "9o.minimap2.assignments.tsv" > taxids.tmp taxonkit --data-dir "taxonomy_dir" lineage -R taxids.tmp | workflow-glue aggregate_lineages -p "9o.minimap2" file1=cat *.json echo "{"'"9o"'": "$file1"}" >> temp cp "temp" "9o.json"

Command exit status: 1

Command output: (empty)

Command error: [ERROR] failed to open file 'ncbi_targeted_loci_16s_18s.fna': No such file or directory [main_samview] fail to read the header from "-".

nggvs commented 1 year ago

Hi @EsbergA !

Thank you for your comment. I guess it is a typo but just in case why do you mean by tar.gz files? The workflow takes fastq or gzip files, but not tar.gz. Besides, if you consider that your issue has been solved, please consider closing the issue and feel free to open a new one if you need it !

Hi @marsfro ! Thank you for using the pipeline! It seems that it is a different error. Could you open a new issue with it (it is useful to have different problems in different issues so that anyone in the future can check them, a way to organise them). The error that you have posted is from the first command using --threads?