search

epi2me-labs / wf-16s

Other
22 stars 5 forks source link

Error making html report #11

Closed CGD-Helix closed 7 months ago

CGD-Helix commented 7 months ago

Operating System

Ubuntu 22.04

Other Linux

No response

Workflow Version

v1.1.2

Workflow Execution

Command line

EPI2ME Version

No response

CLI command run

nextflow run ~/epi2melabs/workflows/epi2me-labs/wf-16s/ --fastq ~/demultiplexed_reads --classifier minimap2 --out_dir ~/16S_exp/EPI2Me_wf-16S

Workflow Execution - CLI Execution Profile

standard (default)

What happened?

When executing the wf-16s pipeline, all processes complete as normal but the final process of making the .html report has failed to execute and resulted in an error with exit status 1. The output can be seen in the log output section of this issue.

Any help on this matter would be appreciated.

Thank you.

Relevant log output

--------------------------------------------------------------------------------
This is epi2me-labs/wf-16s v1.1.2.
--------------------------------------------------------------------------------
Checking inputs.
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
executor >  local (83)
[a6/58e373] process > fastcat (37)                                   [100%] 38 of 38 ✔
[skipped  ] process > prepare_databases:download_unpack_taxonomy     [100%] 1 of 1, stored: 1 ✔
[skipped  ] process > prepare_databases:download_reference_ref2taxid [100%] 1 of 1, stored: 1 ✔
[c4/65fffd] process > minimap_pipeline:run_common:getVersions        [100%] 1 of 1 ✔
[5f/a97230] process > minimap_pipeline:run_common:getParams          [100%] 1 of 1 ✔
[84/b58bad] process > minimap_pipeline:minimap (barcode05)           [100%] 38 of 38 ✔
[4c/9f642a] process > minimap_pipeline:createAbundanceTables         [100%] 1 of 1 ✔
[3a/b74dd3] process > minimap_pipeline:makeReport (1)                [  0%] 0 of 1
[c4/677bba] process > minimap_pipeline:output_results (3)            [100%] 3 of 3
ERROR ~ Error executing process > 'minimap_pipeline:makeReport (1)'

Caused by:
  Process `minimap_pipeline:makeReport (1)` terminated with an error exit status (1)

Command executed:

  workflow-glue report         "wf-16s-report.html"         --workflow_name wf-16s         --versions versions         --params params.json         --read_stats read_stats/*         --lineages lineages         --abundance_table "abundance_table_genus.tsv"         --taxonomic_rank "G"         --pipeline "minimap2"         --abundance_threshold "1"        --n_taxa_barplot "9"

Command exit status:
  1

Command output:
  (empty)

Command error:
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
executor >  local (83)
[a6/58e373] process > fastcat (37)                                   [100%] 38 of 38 ✔
[skipped  ] process > prepare_databases:download_unpack_taxonomy     [100%] 1 of 1, stored: 1 ✔
[skipped  ] process > prepare_databases:download_reference_ref2taxid [100%] 1 of 1, stored: 1 ✔
[c4/65fffd] process > minimap_pipeline:run_common:getVersions        [100%] 1 of 1 ✔
[5f/a97230] process > minimap_pipeline:run_common:getParams          [100%] 1 of 1 ✔
[84/b58bad] process > minimap_pipeline:minimap (barcode05)           [100%] 38 of 38 ✔
[4c/9f642a] process > minimap_pipeline:createAbundanceTables         [100%] 1 of 1 ✔
[3a/b74dd3] process > minimap_pipeline:makeReport (1)                [100%] 1 of 1, failed: 1 ✘
[c4/677bba] process > minimap_pipeline:output_results (3)            [100%] 3 of 3 ✔
ERROR ~ Error executing process > 'minimap_pipeline:makeReport (1)'

Caused by:
  Process `minimap_pipeline:makeReport (1)` terminated with an error exit status (1)

Command executed:

  workflow-glue report         "wf-16s-report.html"         --workflow_name wf-16s         --versions versions         --params params.json         --read_stats read_stats/*         --lineages lineages         --abundance_table "abundance_table_genus.tsv"         --taxonomic_rank "G"         --pipeline "minimap2"         --abundance_threshold "1"        --n_taxa_barplot "9"

Command exit status:
  1

Command output:
  (empty)

Command error:
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
executor >  local (83)
[a6/58e373] process > fastcat (37)                                   [100%] 38 of 38 ✔
[skipped  ] process > prepare_databases:download_unpack_taxonomy     [100%] 1 of 1, stored: 1 ✔
[skipped  ] process > prepare_databases:download_reference_ref2taxid [100%] 1 of 1, stored: 1 ✔
[c4/65fffd] process > minimap_pipeline:run_common:getVersions        [100%] 1 of 1 ✔
[5f/a97230] process > minimap_pipeline:run_common:getParams          [100%] 1 of 1 ✔
[84/b58bad] process > minimap_pipeline:minimap (barcode05)           [100%] 38 of 38 ✔
[4c/9f642a] process > minimap_pipeline:createAbundanceTables         [100%] 1 of 1 ✔
[3a/b74dd3] process > minimap_pipeline:makeReport (1)                [100%] 1 of 1, failed: 1 ✘
[c4/677bba] process > minimap_pipeline:output_results (3)            [100%] 3 of 3 ✔
ERROR ~ Error executing process > 'minimap_pipeline:makeReport (1)'

Caused by:
  Process `minimap_pipeline:makeReport (1)` terminated with an error exit status (1)

Command executed:

  workflow-glue report         "wf-16s-report.html"         --workflow_name wf-16s         --versions versions         --params params.json         --read_stats read_stats/*         --lineages lineages         --abundance_table "abundance_table_genus.tsv"         --taxonomic_rank "G"         --pipeline "minimap2"         --abundance_threshold "1"        --n_taxa_barplot "9"

Command exit status:
  1

Command output:
  (empty)

Command error:
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  /home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py:213: UserWarning: Ignoring `palette` because no `hue` variable has been assigned.
    plt = ezc.lineplot(data=df, x=xlab, y=ylab, hue=None)
  [08:29:54 - workflow_glue.Report    ] Sample barcode08.
  [08:29:54 - EChrtPlotr] Skipping axis label fixing
  [08:29:54 - workflow_glue.Report    ] Sample barcode06.
  [08:29:54 - EChrtPlotr] Skipping axis label fixing
  [08:29:54 - workflow_glue.Report    ] Sample HW-16S-BP15.
  [08:29:54 - EChrtPlotr] Skipping axis label fixing
  [08:29:54 - workflow_glue.Report    ] Sample HW-16S-BP20.
  Traceback (most recent call last):
    File "/home/user/epi2melabs/workflows/epi2me-labs/wf-16s/bin/workflow-glue", line 7, in <module>
      cli()
    File "/home/user/epi2melabs/workflows/epi2me-labs/wf-16s/wf-metagenomics/bin/workflow_glue/__init__.py", line 72, in cli
      args.func(args)
    File "/home/user/epi2melabs/workflows/epi2me-labs/wf-16s/wf-metagenomics/bin/workflow_glue/report.py", line 178, in main
      plt = ezc.sunburst(
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/plots/sunburst.py", line 62, in sunburst
      if data[0].get('value'):
  IndexError: list index out of range

Work dir:
  /home/user/work/3a/b74dd3a1d87865d66a011e87f005d8

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

 -- Check '.nextflow.log' file for details

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

No response

CGD-Helix commented 7 months ago

Upon review, I believe the issue may be due to certain barcodes having a very minimal number of classified reads. These are misclassifications and possibly they are interfering with the report generation. When removing these barcode folders, the report is generated.

nggvs commented 7 months ago

Hi @CGD-Helix , I was about to suggest to remove empty barcodes. Nevertheless, we latest release, this problem should be solved without doing that. Let me know if works for you. Thank you very much for using the workflow!

CGD-Helix commented 7 months ago

Hi @nggvs - OK great, thank you!