workflow failing in makeAlignmentReport: ValueError: cannot convert float NaN to integer

[sfcs]

Operating System

Windows 10

Other Linux

Rocky Linux release 8.7 (Green Obsidian)

Workflow Version

v1.10.1-g79ddc44

Workflow Execution

EPI2ME Desktop application

EPI2ME Version

No response

CLI command run

nextflow run epi2me-labs/wf-human-variation \ -profile biowulf \ --bam 'U2OS_ONT_WGS_hg38.sorted.bam' \ --basecaller_cfg 'dna_r10.4.1_e8.2_400bps_hac@v4.2.0' \ --ref '/data/clatterbucksosf/hg38/genome.fa' \ --sample_name 'U2OS_hg38_WGS' \ --bam_min_coverage 0 \ --mod \ --snp \ --cnv \ --str \ --sv \ -resume

Workflow Execution - CLI Execution Profile

None

What happened?

--The directory /home/clatterbucksosf/.config/matplotlib does exist and is writable, but presumably isn't bound for some reason? I am not sure whether this is actually causing the issue.

ERROR ~ Error executing process > 'makeAlignmentReport'

Caused by: Process makeAlignmentReport terminated with an error exit status (1)

Command executed:

workflow-glue report_al \ --name U2OS_hg38_WGS.wf-human-alignment-report.html \ --stats_dir readstats/ \ --reference_fai ref.fasta.fai \ --flagstat_dir flagstats/ \ --depths_dir depths/ \ --versions versions.txt \ --window_size 25000 \ --params params.json

Command exit status: 1

Command output: (empty)

Command error: [14:53:36 - matplotlib] Matplotlib created a temporary cache directory at /tmp/matplotlib-z510drlq because the default path (/home/clatterbucksosf/.config/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing. [14:53:36 - matplotlib.font_manager] generated new fontManager [14:53:38 - workflow_glue] Starting entrypoint. /home/epi2melabs/conda/lib/python3.8/site-packages/numpy/lib/nanfunctions.py:1215: RuntimeWarning: Mean of empty slice return np.nanmean(a, axis, out=out, keepdims=keepdims) Traceback (most recent call last): File "/data/clatterbucksosf/nxf/assets/epi2me-labs/wf-human-variation/bin/workflow-glue", line 7, in cli() File "/gs10/users/clatterbucksosf/nxf/assets/epi2me-labs/wf-human-variation/bin/workflow_glue/init.py", line 72, in cli args.func(args) File "/gs10/users/clatterbucksosf/nxf/assets/epi2me-labs/wf-human-variation/bin/workflow_glue/report_al.py", line 71, in main SeqSummary(f"{args.stats_dir}/") File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 67, in init draw_all_plots(df_all, theme) File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 135, in draw_all_plots EZChart(read_quality_plot(seq_summary), theme) File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 182, in read_quality_plot median_q = int(df.mean_quality.median()) ValueError: cannot convert float NaN to integer

Work dir: /gpfs/gsfs10/users/clatterbucksosf/U2OS_WGS_121323/work/e2/640baf2e1c88cae4cb0393aea240e2

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named .command.sh

-- Check '.nextflow.log' file for details

Relevant log output

Core Nextflow options
  revision        : master
  runName         : thirsty_franklin
  containerEngine : singularity
  container       : [withLabel:wf_human_snp:ontresearch/wf-human-variation-snp:sha14d7c4b75950fbffae7b2ea452dba21b263b84a1, withLabel:wf_human_sv:ont
research/wf-human-variation-sv:shabc3ac908a14705f248cdf49f218956ec33e93ef9, withLabel:wf_human_mod:ontresearch/modkit:shaeedb131a939d3eea2f9bd4dbecec805c0fa20bdb
, withLabel:wf_cnv:ontresearch/wf-cnv:sha428cb19e51370020ccf29ec2af4eead44c6a17c2, withLabel:wf_human_str:ontresearch/wf-human-variation-str:sha28799bc3058fa256c
01c1f07c87f04e4ade1fcc1, withLabel:snpeff_annotation:ontresearch/snpeff:sha4f289afaf754c7a3e0b9ffb6c0b5be0f89a5cf04, withLabel:wf_common:ontresearch/wf-common:sh
a1c5febff9f75143710826498b093d9769a5edbb9, default:ontresearch/wf-human-variation:sha26be394971f6444d657b0be6745a341beb0b2843]
  launchDir       : /gpfs/gsfs10/users/clatterbucksosf/U2OS_WGS_121323
  workDir         : /gpfs/gsfs10/users/clatterbucksosf/U2OS_WGS_121323/work
  projectDir      : /data/clatterbucksosf/nxf/assets/epi2me-labs/wf-human-variation
  userName        : clatterbucksosf
  profile         : biowulf
  configFiles     : /data/clatterbucksosf/nxf/assets/epi2me-labs/wf-human-variation/nextflow.config, /gpfs/gsfs10/users/clatterbucksosf/U2OS_WGS_1213
23/nextflow.config

Workflow Options
  sv              : true
  snp             : true
  cnv             : true
  str             : true
  mod             : true

Main options
  sample_name     : U2OS_hg38_WGS
  bam             : U2OS_ONT_WGS_hg38.sorted.bam
  ref             : /data/clatterbucksosf/hg38/genome.fa
  basecaller_cfg  : dna_r10.4.1_e8.2_400bps_hac@v4.2.0
  bam_min_coverage: 0

: COMPLETED; exit: 1; error: -; workDir: /gpfs/gsfs10/users/clatterbucksosf/U2OS_WGS_121323/work/e2/640baf2e1c88cae4cb0393aea240e2 started: 1706298805969; exited
: 2024-01-26T19:54:54.130897313Z; ]
Jan-26 14:54:55.955 [Task monitor] DEBUG nextflow.processor.TaskProcessor - Handling unexpected condition for
  task: name=makeAlignmentReport; work-dir=/gpfs/gsfs10/users/clatterbucksosf/U2OS_WGS_121323/work/e2/640baf2e1c88cae4cb0393aea240e2
  error [nextflow.exception.ProcessFailedException]: Process `makeAlignmentReport` terminated with an error exit status (1)
Jan-26 14:54:55.994 [Task monitor] ERROR nextflow.processor.TaskProcessor - Error executing process > 'makeAlignmentReport'

Caused by:
  Process `makeAlignmentReport` terminated with an error exit status (1)

Command executed:

  workflow-glue report_al \
      --name U2OS_hg38_WGS.wf-human-alignment-report.html \
      --stats_dir readstats/ \
      --reference_fai ref.fasta.fai \
      --flagstat_dir flagstats/ \
      --depths_dir depths/ \
      --versions versions.txt \
      --window_size 25000 \
      --params params.json

Command exit status:
  1

Command output:
  (empty)

Command error:
  [14:53:36 - matplotlib] Matplotlib created a temporary cache directory at /tmp/matplotlib-z510drlq because the default path (/home/clatterbucksosf/.config/matp
lotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the
 import of Matplotlib and to better support multiprocessing.
  [14:53:36 - matplotlib.font_manager] generated new fontManager
  [14:53:38 - workflow_glue] Starting entrypoint.
  /home/epi2melabs/conda/lib/python3.8/site-packages/numpy/lib/nanfunctions.py:1215: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  Traceback (most recent call last):
    File "/data/clatterbucksosf/nxf/assets/epi2me-labs/wf-human-variation/bin/workflow-glue", line 7, in <module>
      cli()
    File "/gs10/users/clatterbucksosf/nxf/assets/epi2me-labs/wf-human-variation/bin/workflow_glue/__init__.py", line 72, in cli
      args.func(args)
    File "/gs10/users/clatterbucksosf/nxf/assets/epi2me-labs/wf-human-variation/bin/workflow_glue/report_al.py", line 71, in main
      SeqSummary(f"{args.stats_dir}/")
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 67, in __init__
      draw_all_plots(df_all, theme)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 135, in draw_all_plots
      EZChart(read_quality_plot(seq_summary), theme)
    File "/home/epi2melabs/conda/lib/python3.8/site-packages/ezcharts/components/fastcat.py", line 182, in read_quality_plot
      median_q = int(df.mean_quality.median())
  ValueError: cannot convert float NaN to integer

Work dir:
  /gpfs/gsfs10/users/clatterbucksosf/U2OS_WGS_121323/work/e2/640baf2e1c88cae4cb0393aea240e2

Application activity log entry

No response

Were you able to successfully run the latest version of the workflow with the demo data?

yes

Other demo data information

(I have started downloading the demo data but have not run it yet.)

[RenzoTale88]

Hi @sfcs can you please check that the files in /gpfs/gsfs10/users/clatterbucksosf/U2OS_WGS_121323/work/e2/640baf2e1c88cae4cb0393aea240e2/readstats/ are not empty? Could you please share the content of one of them?

[sfcs]

Ah, there is not a readstats directory at all. What have I done wrong here?

[RenzoTale88]

@sfcs apologies for my very late reply. We need to dig this out going backwards. Can you share the .nextflow.log or the execution/report.html that you can find within output directory?

[sfcs]

The demo data ran perfectly, so I am thinking it must be something wrong with my bam file. I was trying to run an aligned bam, but before we get into the weeds too much with troubleshooting I am trying to run the unaligned data. That's in process right now. I will let you know if it bombs out in the same way, and provide those files.

(We just got our sequencer and they guys are learning the ropes running it as much as I am learning the ropes with the processing, so none of this is super urgent! Literally just playing with some cell line data.)

[sfcs]

@RenzoTale88 Did not have this problem running on the unaligned bam, so just going to go with that. Thanks!

[RenzoTale88]

@sfcs thanks for confirming that the issue is upstream from the workflow. Closing.