epi2me-labs / wf-single-cell

Other
64 stars 31 forks source link

Error in add_tag #66

Closed irzamsarfraz closed 6 months ago

irzamsarfraz commented 7 months ago

Operating System

Linux

Other Linux

No response

Workflow Version

v0.3.0

Workflow Execution

EPI2ME Desktop application

EPI2ME Version

No response

CLI command run

nextflow run epi2me-labs/wf-single-cell \
    -w ${OUTPUT}/workspace \
    -profile singularity \
    --fastq /restricted/projectnb/camplab/projects/20231027_Cam_Oxford_Nanopore/data/raw/231027Cam/pseudoID-1514/20231102_1500_1C_PAQ13978_6b7e4e9c/fastq_pass/unclassified_merged/merged.fastq \
    --kit_name 5prime \
    --kit_version v1 \
    --expected_cells 100000 \
    --ref_genome_dir refdata-gex-GRCh38-2020-A/ \
    --out_dir ${OUTPUT} \
    --plot_umaps \
    --max_threads 16 \
    --matrix_min_genes 10 \
    --resources_mm2_max_threads 16

Workflow Execution - CLI Execution Profile

None

What happened?

Error from add_tag as shown below:

executor > local (266) [ef/624b7c] process > fastcat (1) [100%] 1 of 1 ✔ [60/5e5ef9] process > parse_kit_metadata (1) [100%] 1 of 1 ✔ [63/530d21] process > pipeline:getVersions [100%] 1 of 1 ✔ [a7/d1508f] process > pipeline:getParams [100%] 1 of 1 ✔ [2e/133276] process > pipeline:summariseCatChunkR... [100%] 1 of 1 ✔ [ed/e88185] process > pipeline:stranding:callada... [100%] 16 of 16 ✔ [2c/8176d0] process > pipeline:stranding:combine... [100%] 1 of 1 ✔ [b5/6487b1] process > pipeline:stranding:summariz... [100%] 1 of 1 ✔ [9c/e7f591] process > pipeline:align:call_paftools [100%] 1 of 1 ✔ [b2/0b70da] process > pipeline:align:get_chrom_sizes [100%] 1 of 1 ✔ [64/57e693] process > pipeline:align:align_to_ref... [100%] 1 of 1 ✔ [ea/2a2bfc] process > pipeline:process_bams:split... [100%] 1 of 1 ✔ [9e/9708b4] process > pipeline:process_bams:get_c... [100%] 1 of 1 ✔ [b1/de63f2] process > pipeline:process_bams:extra... [100%] 40 of 40 ✔ [ae/0a6279] process > pipeline:process_bams:combi... [100%] 1 of 1 ✔ [24/86aeb3] process > pipeline:process_bams:gener... [100%] 1 of 1 ✔ [e9/55815c] process > pipeline:process_bams:assig... [ 90%] 36 of 40 [06/fa9188] process > pipeline:process_bams:strin... [100%] 40 of 40 ✔ [b3/afe532] process > pipeline:process_bams:align... [100%] 40 of 40 ✔ [65/df09da] process > pipeline:process_bams:assig... [ 77%] 28 of 36 [77/cd29c6] process > pipeline:process_bams:clust... [ 96%] 27 of 28 [f7/291f0d] process > pipeline:process_bams:tag_b... [ 33%] 9 of 27, failed: 1 [- ] process > pipeline:process_bams:combi... - [- ] process > pipeline:process_bams:combi... - [- ] process > pipeline:process_bams:umi_g... - [- ] process > pipeline:process_bams:const... - [- ] process > pipeline:process_bams:proce... - [- ] process > pipeline:processbams:umap... - [- ] process > pipeline:processbams:pack... - [- ] process > pipeline:prepare_report_data - [- ] process > pipeline:makeReport - [- ] process > output - [- ] process > output_report - Error executing process > 'pipeline:process_bams:tag_bams (8)'

Caused by: Process pipeline:process_bams:tag_bams (8) terminated with an error exit status (1)

Command executed:

workflow-glue tag_bam --in_bam align.bam --tags tags.tsv --out_bam merged.GL000009.2.tagged.bam --chrom GL000009.2

samtools index merged.GL000009.2.tagged.bam

Command exit status: 1

Command output: (empty)

Command error: /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() /home/epi2melabs/conda/lib/python3.8/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() [16:46:51 - workflow_glue] Starting entrypoint. Traceback (most recent call last): File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue", line 7, in cli() File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/init.py", line 72, in cli args.func(args) File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 88, in main add_tags(args.tags, args.in_bam, args.out_bam, args.chrom, args.flip) File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 69, in add_tags align.set_tag('UY', row['UY'], value_type="Z") File "pysam/libcalignedsegment.pyx", line 2316, in pysam.libcalignedsegment.AlignedSegment.set_tag File "pysam/libcalignedsegment.pyx", line 2400, in pysam.libcalignedsegment.AlignedSegment.set_tag File "pysam/libcutils.pyx", line 134, in pysam.libcutils.force_bytes TypeError: Argument must be string, bytes or unicode.

Relevant log output

==========================================================
Start date : Wed Dec  6 14:30:03 EST 2023
Job name : wf_singlecell
Job ID : 2844958  undefined
==========================================================
N E X T F L O W  ~  version 21.10.6
Launching `epi2me-labs/wf-single-cell` [nostalgic_goldstine] - revision: 2aba2e898b [master]

||||||||||   _____ ____ ___ ____  __  __ _____      _       _
||||||||||  | ____|  _ \_ _|___ \|  \/  | ____|    | | __ _| |__  ___
|||||       |  _| | |_) | |  __) | |\/| |  _| _____| |/ _` | '_ \/ __|
|||||       | |___|  __/| | / __/| |  | | |__|_____| | (_| | |_) \__ \
||||||||||  |_____|_|  |___|_____|_|  |_|_____|    |_|\__,_|_.__/|___/
||||||||||  wf-single-cell v0.3.0-g2aba2e8
--------------------------------------------------------------------------------
Core Nextflow options
  revision                 : master
  runName                  : nostalgic_goldstine
  containerEngine          : singularity
  launchDir                : /restricted/projectnb/camplab/home/isarfraz/Nanopore/20231027_Oxford_Nanpore/wf-single-cell
  workDir                  : /restricted/projectnb/camplab/home/isarfraz/Nanopore/20231027_Oxford_Nanpore/wf-single-cell/output/workspace
  projectDir               : /usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell
  userName                 : isarfraz
  profile                  : singularity
  configFiles              : /usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/nextflow.config

Input Options
  fastq                    : /restricted/projectnb/camplab/projects/20231027_Cam_Oxford_Nanopore/data/raw/231027Cam/pseudoID-1514/20231102_1500_1C_PAQ13978_6b7e4e9c/fastq_pass/unclassified_merged/merged.fastq
  ref_genome_dir           : refdata-gex-GRCh38-2020-A/
  kit_name                 : 5prime
  kit_version              : v1
  expected_cells           : 100000

Output Options
  plot_umaps               : true

Advanced options
  max_threads              : 16
  matrix_min_genes         : 10
  resources_mm2_max_threads: 16

!! Only displaying parameters that differ from the pipeline defaults !!
--------------------------------------------------------------------------------
If you use epi2me-labs/wf-single-cell for your analysis please cite:

* The nf-core framework
  https://doi.org/10.1038/s41587-020-0439-x

--------------------------------------------------------------------------------
This is epi2me-labs/wf-single-cell v0.3.0-g2aba2e8.
--------------------------------------------------------------------------------
WARN: Nextflow version 21.10.6 does not match workflow required version: >=23.04.2 -- Execution will continue, but things may break!
Searching input for [.fastq, .fastq.gz, .fq, .fq.gz] files.
[-        ] process > fastcat                        -
[-        ] process > parse_kit_metadata             -
[-        ] process > pipeline:getVersions           -
[-        ] process > pipeline:getParams             -
[-        ] process > pipeline:summariseCatChunkR... -
[-        ] process > pipeline:stranding:call_ada... -
[-        ] process > pipeline:stranding:combine_... -
[-        ] process > pipeline:stranding:summariz... -
[-        ] process > pipeline:align:call_paftools   -
[-        ] process > pipeline:align:get_chrom_sizes -
[-        ] process > pipeline:align:align_to_ref    -
[-        ] process > pipeline:process_bams:split... -
[-        ] process > pipeline:process_bams:get_c... -

executor >  local (264)
[ef/624b7c] process > fastcat (1)                    [100%] 1 of 1 ✔
[60/5e5ef9] process > parse_kit_metadata (1)         [100%] 1 of 1 ✔
[63/530d21] process > pipeline:getVersions           [100%] 1 of 1 ✔
[a7/d1508f] process > pipeline:getParams             [100%] 1 of 1 ✔
[2e/133276] process > pipeline:summariseCatChunkR... [100%] 1 of 1 ✔
[ed/e88185] process > pipeline:stranding:call_ada... [100%] 16 of 16 ✔
[2c/8176d0] process > pipeline:stranding:combine_... [100%] 1 of 1 ✔
[b5/6487b1] process > pipeline:stranding:summariz... [100%] 1 of 1 ✔
[9c/e7f591] process > pipeline:align:call_paftools   [100%] 1 of 1 ✔
[b2/0b70da] process > pipeline:align:get_chrom_sizes [100%] 1 of 1 ✔
[64/57e693] process > pipeline:align:align_to_ref... [100%] 1 of 1 ✔
[ea/2a2bfc] process > pipeline:process_bams:split... [100%] 1 of 1 ✔
[9e/9708b4] process > pipeline:process_bams:get_c... [100%] 1 of 1 ✔
[b1/de63f2] process > pipeline:process_bams:extra... [100%] 40 of 40 ✔
[ae/0a6279] process > pipeline:process_bams:combi... [100%] 1 of 1 ✔
[24/86aeb3] process > pipeline:process_bams:gener... [100%] 1 of 1 ✔
[e9/55815c] process > pipeline:process_bams:assig... [ 90%] 36 of 40
[06/fa9188] process > pipeline:process_bams:strin... [100%] 40 of 40 ✔
[b3/afe532] process > pipeline:process_bams:align... [100%] 40 of 40 ✔
[65/df09da] process > pipeline:process_bams:assig... [ 77%] 28 of 36
[77/cd29c6] process > pipeline:process_bams:clust... [ 96%] 27 of 28
[44/a90e91] process > pipeline:process_bams:tag_b... [ 25%] 7 of 27
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -

executor >  local (264)
[ef/624b7c] process > fastcat (1)                    [100%] 1 of 1 ✔
[60/5e5ef9] process > parse_kit_metadata (1)         [100%] 1 of 1 ✔
[63/530d21] process > pipeline:getVersions           [100%] 1 of 1 ✔
[a7/d1508f] process > pipeline:getParams             [100%] 1 of 1 ✔
[2e/133276] process > pipeline:summariseCatChunkR... [100%] 1 of 1 ✔
[ed/e88185] process > pipeline:stranding:call_ada... [100%] 16 of 16 ✔
[2c/8176d0] process > pipeline:stranding:combine_... [100%] 1 of 1 ✔
[b5/6487b1] process > pipeline:stranding:summariz... [100%] 1 of 1 ✔
[9c/e7f591] process > pipeline:align:call_paftools   [100%] 1 of 1 ✔
[b2/0b70da] process > pipeline:align:get_chrom_sizes [100%] 1 of 1 ✔
[64/57e693] process > pipeline:align:align_to_ref... [100%] 1 of 1 ✔
[ea/2a2bfc] process > pipeline:process_bams:split... [100%] 1 of 1 ✔
[9e/9708b4] process > pipeline:process_bams:get_c... [100%] 1 of 1 ✔
[b1/de63f2] process > pipeline:process_bams:extra... [100%] 40 of 40 ✔
[ae/0a6279] process > pipeline:process_bams:combi... [100%] 1 of 1 ✔
[24/86aeb3] process > pipeline:process_bams:gener... [100%] 1 of 1 ✔
[e9/55815c] process > pipeline:process_bams:assig... [ 90%] 36 of 40
[06/fa9188] process > pipeline:process_bams:strin... [100%] 40 of 40 ✔
[b3/afe532] process > pipeline:process_bams:align... [100%] 40 of 40 ✔
[65/df09da] process > pipeline:process_bams:assig... [ 77%] 28 of 36
[77/cd29c6] process > pipeline:process_bams:clust... [ 96%] 27 of 28
[44/a90e91] process > pipeline:process_bams:tag_b... [ 25%] 7 of 27
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -

executor >  local (266)
[ef/624b7c] process > fastcat (1)                    [100%] 1 of 1 ✔
[60/5e5ef9] process > parse_kit_metadata (1)         [100%] 1 of 1 ✔
[63/530d21] process > pipeline:getVersions           [100%] 1 of 1 ✔
[a7/d1508f] process > pipeline:getParams             [100%] 1 of 1 ✔
[2e/133276] process > pipeline:summariseCatChunkR... [100%] 1 of 1 ✔
[ed/e88185] process > pipeline:stranding:call_ada... [100%] 16 of 16 ✔
[2c/8176d0] process > pipeline:stranding:combine_... [100%] 1 of 1 ✔
[b5/6487b1] process > pipeline:stranding:summariz... [100%] 1 of 1 ✔
[9c/e7f591] process > pipeline:align:call_paftools   [100%] 1 of 1 ✔
[b2/0b70da] process > pipeline:align:get_chrom_sizes [100%] 1 of 1 ✔
[64/57e693] process > pipeline:align:align_to_ref... [100%] 1 of 1 ✔
[ea/2a2bfc] process > pipeline:process_bams:split... [100%] 1 of 1 ✔
[9e/9708b4] process > pipeline:process_bams:get_c... [100%] 1 of 1 ✔
[b1/de63f2] process > pipeline:process_bams:extra... [100%] 40 of 40 ✔
[ae/0a6279] process > pipeline:process_bams:combi... [100%] 1 of 1 ✔
[24/86aeb3] process > pipeline:process_bams:gener... [100%] 1 of 1 ✔
[e9/55815c] process > pipeline:process_bams:assig... [ 90%] 36 of 40
[06/fa9188] process > pipeline:process_bams:strin... [100%] 40 of 40 ✔
[b3/afe532] process > pipeline:process_bams:align... [100%] 40 of 40 ✔
[65/df09da] process > pipeline:process_bams:assig... [ 77%] 28 of 36
[77/cd29c6] process > pipeline:process_bams:clust... [ 96%] 27 of 28
[4c/7760df] process > pipeline:process_bams:tag_b... [ 29%] 8 of 27
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -
Error executing process > 'pipeline:process_bams:tag_bams (8)'

Caused by:
  Process `pipeline:process_bams:tag_bams (8)` terminated with an error exit status (1)

Command executed:

  workflow-glue tag_bam         --in_bam align.bam         --tags tags.tsv         --out_bam merged.GL000009.2.tagged.bam         --chrom GL000009.2         

  samtools index merged.GL000009.2.tagged.bam

Command exit status:
  1

Command output:
  (empty)

Command error:
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  [16:46:51 - workflow_glue] Starting entrypoint.
  Traceback (most recent call last):
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue", line 7, in <module>
      cli()
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/__init__.py", line 72, in cli
      args.func(args)
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 88, in main
      add_tags(args.tags, args.in_bam, args.out_bam, args.chrom, args.flip)
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 69, in add_tags
      align.set_tag('UY', row['UY'], value_type="Z")
    File "pysam/libcalignedsegment.pyx", line 2316, in pysam.libcalignedsegment.AlignedSegment.set_tag
    File "pysam/libcalignedsegment.pyx", line 2400, in pysam.libcalignedsegment.AlignedSegment.set_tag
    File "pysam/libcutils.pyx", line 134, in pysam.libcutils.force_bytes
  TypeError: Argument must be string, bytes or unicode.

Work dir:
  /restricted/projectnb/camplab/home/isarfraz/Nanopore/20231027_Oxford_Nanpore/wf-single-cell/output/workspace/f7/291f0d72d126c6cfc44b03671d74e2

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

executor >  local (266)
[ef/624b7c] process > fastcat (1)                    [100%] 1 of 1 ✔
[60/5e5ef9] process > parse_kit_metadata (1)         [100%] 1 of 1 ✔
[63/530d21] process > pipeline:getVersions           [100%] 1 of 1 ✔
[a7/d1508f] process > pipeline:getParams             [100%] 1 of 1 ✔
[2e/133276] process > pipeline:summariseCatChunkR... [100%] 1 of 1 ✔
[ed/e88185] process > pipeline:stranding:call_ada... [100%] 16 of 16 ✔
[2c/8176d0] process > pipeline:stranding:combine_... [100%] 1 of 1 ✔
[b5/6487b1] process > pipeline:stranding:summariz... [100%] 1 of 1 ✔
[9c/e7f591] process > pipeline:align:call_paftools   [100%] 1 of 1 ✔
[b2/0b70da] process > pipeline:align:get_chrom_sizes [100%] 1 of 1 ✔
[64/57e693] process > pipeline:align:align_to_ref... [100%] 1 of 1 ✔
[ea/2a2bfc] process > pipeline:process_bams:split... [100%] 1 of 1 ✔
[9e/9708b4] process > pipeline:process_bams:get_c... [100%] 1 of 1 ✔
[b1/de63f2] process > pipeline:process_bams:extra... [100%] 40 of 40 ✔
[ae/0a6279] process > pipeline:process_bams:combi... [100%] 1 of 1 ✔
[24/86aeb3] process > pipeline:process_bams:gener... [100%] 1 of 1 ✔
[e9/55815c] process > pipeline:process_bams:assig... [ 90%] 36 of 40
[06/fa9188] process > pipeline:process_bams:strin... [100%] 40 of 40 ✔
[b3/afe532] process > pipeline:process_bams:align... [100%] 40 of 40 ✔
[65/df09da] process > pipeline:process_bams:assig... [ 77%] 28 of 36
[77/cd29c6] process > pipeline:process_bams:clust... [ 96%] 27 of 28
[f7/291f0d] process > pipeline:process_bams:tag_b... [ 33%] 9 of 27, failed: 1
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -
Error executing process > 'pipeline:process_bams:tag_bams (8)'

Caused by:
  Process `pipeline:process_bams:tag_bams (8)` terminated with an error exit status (1)

Command executed:

  workflow-glue tag_bam         --in_bam align.bam         --tags tags.tsv         --out_bam merged.GL000009.2.tagged.bam         --chrom GL000009.2         

  samtools index merged.GL000009.2.tagged.bam

Command exit status:
  1

Command output:
  (empty)

Command error:
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  [16:46:51 - workflow_glue] Starting entrypoint.
  Traceback (most recent call last):
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue", line 7, in <module>
      cli()
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/__init__.py", line 72, in cli
      args.func(args)
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 88, in main
      add_tags(args.tags, args.in_bam, args.out_bam, args.chrom, args.flip)
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 69, in add_tags
      align.set_tag('UY', row['UY'], value_type="Z")
    File "pysam/libcalignedsegment.pyx", line 2316, in pysam.libcalignedsegment.AlignedSegment.set_tag
    File "pysam/libcalignedsegment.pyx", line 2400, in pysam.libcalignedsegment.AlignedSegment.set_tag
    File "pysam/libcutils.pyx", line 134, in pysam.libcutils.force_bytes
  TypeError: Argument must be string, bytes or unicode.

Work dir:
  /restricted/projectnb/camplab/home/isarfraz/Nanopore/20231027_Oxford_Nanpore/wf-single-cell/output/workspace/f7/291f0d72d126c6cfc44b03671d74e2

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

executor >  local (266)
[ef/624b7c] process > fastcat (1)                    [100%] 1 of 1 ✔
[60/5e5ef9] process > parse_kit_metadata (1)         [100%] 1 of 1 ✔
[63/530d21] process > pipeline:getVersions           [100%] 1 of 1 ✔
[a7/d1508f] process > pipeline:getParams             [100%] 1 of 1 ✔
[2e/133276] process > pipeline:summariseCatChunkR... [100%] 1 of 1 ✔
[ed/e88185] process > pipeline:stranding:call_ada... [100%] 16 of 16 ✔
[2c/8176d0] process > pipeline:stranding:combine_... [100%] 1 of 1 ✔
[b5/6487b1] process > pipeline:stranding:summariz... [100%] 1 of 1 ✔
[9c/e7f591] process > pipeline:align:call_paftools   [100%] 1 of 1 ✔
[b2/0b70da] process > pipeline:align:get_chrom_sizes [100%] 1 of 1 ✔
[64/57e693] process > pipeline:align:align_to_ref... [100%] 1 of 1 ✔
[ea/2a2bfc] process > pipeline:process_bams:split... [100%] 1 of 1 ✔
[9e/9708b4] process > pipeline:process_bams:get_c... [100%] 1 of 1 ✔
[b1/de63f2] process > pipeline:process_bams:extra... [100%] 40 of 40 ✔
[ae/0a6279] process > pipeline:process_bams:combi... [100%] 1 of 1 ✔
[24/86aeb3] process > pipeline:process_bams:gener... [100%] 1 of 1 ✔
[e9/55815c] process > pipeline:process_bams:assig... [ 90%] 36 of 40
[06/fa9188] process > pipeline:process_bams:strin... [100%] 40 of 40 ✔
[b3/afe532] process > pipeline:process_bams:align... [100%] 40 of 40 ✔
[65/df09da] process > pipeline:process_bams:assig... [ 77%] 28 of 36
[77/cd29c6] process > pipeline:process_bams:clust... [ 96%] 27 of 28
[f7/291f0d] process > pipeline:process_bams:tag_b... [ 33%] 9 of 27, failed: 1
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:combi... -
[-        ] process > pipeline:process_bams:umi_g... -
[-        ] process > pipeline:process_bams:const... -
[-        ] process > pipeline:process_bams:proce... -
[-        ] process > pipeline:process_bams:umap_... -
[-        ] process > pipeline:process_bams:pack_... -
[-        ] process > pipeline:prepare_report_data   -
[-        ] process > pipeline:makeReport            -
[-        ] process > output                         -
[-        ] process > output_report                  -
Error executing process > 'pipeline:process_bams:tag_bams (8)'

Caused by:
  Process `pipeline:process_bams:tag_bams (8)` terminated with an error exit status (1)

Command executed:

  workflow-glue tag_bam         --in_bam align.bam         --tags tags.tsv         --out_bam merged.GL000009.2.tagged.bam         --chrom GL000009.2         

  samtools index merged.GL000009.2.tagged.bam

Command exit status:
  1

Command output:
  (empty)

Command error:
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  /home/epi2melabs/conda/lib/python3.8/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details.
    @numba.jit()
  [16:46:51 - workflow_glue] Starting entrypoint.
  Traceback (most recent call last):
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue", line 7, in <module>
      cli()
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/__init__.py", line 72, in cli
      args.func(args)
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 88, in main
      add_tags(args.tags, args.in_bam, args.out_bam, args.chrom, args.flip)
    File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 69, in add_tags
      align.set_tag('UY', row['UY'], value_type="Z")
    File "pysam/libcalignedsegment.pyx", line 2316, in pysam.libcalignedsegment.AlignedSegment.set_tag
    File "pysam/libcalignedsegment.pyx", line 2400, in pysam.libcalignedsegment.AlignedSegment.set_tag
    File "pysam/libcutils.pyx", line 134, in pysam.libcutils.force_bytes
  TypeError: Argument must be string, bytes or unicode.

Work dir:
  /restricted/projectnb/camplab/home/isarfraz/Nanopore/20231027_Oxford_Nanpore/wf-single-cell/output/workspace/f7/291f0d72d126c6cfc44b03671d74e2

Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`

WARN: Killing pending tasks (16)

Application activity log entry

No response

nrhorner commented 7 months ago

Hi @irzamsarfraz

Sorry that you are having problems with the workflow. This a known issue and a fix will be released very soon.

blindsey101112 commented 6 months ago

Hi @nrhorner,

Thanks for looking into this. We have just had the same error. Do you know roughly when the fix might be released?

nrhorner commented 6 months ago

Hi @blindsey101112

The fix is on our pre-release branch. I'd be grateful if you could test it out for me.

nextflow run epi2me-labs/wf-single-cell -r prerelease ...

blindsey101112 commented 6 months ago

Great, thank you very much. I've just set it running now and will let you know how it goes.

On Tue, 16 Jan 2024 at 09:58, Neil Horner @.***> wrote:

Hi @blindsey101112 https://github.com/blindsey101112

The fix is on our pre-release branch. I'd be grateful if you could test it out for me.

nextflow run epi2me-labs/wf-single-cell -r prerelease ...

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-single-cell/issues/66#issuecomment-1893414514, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZICVRREIW5YM42FIBCZOIDYOZFMPAVCNFSM6AAAAABAKJJ3TCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJTGQYTINJRGQ . You are receiving this because you were mentioned.Message ID: @.***>

blindsey101112 commented 6 months ago

It worked fine with the pre-release. Thanks very much

On Tue, 16 Jan 2024 at 10:25, Benjamin Lindsey @.***> wrote:

Great, thank you very much. I've just set it running now and will let you know how it goes.

On Tue, 16 Jan 2024 at 09:58, Neil Horner @.***> wrote:

Hi @blindsey101112 https://github.com/blindsey101112

The fix is on our pre-release branch. I'd be grateful if you could test it out for me.

nextflow run epi2me-labs/wf-single-cell -r prerelease ...

— Reply to this email directly, view it on GitHub https://github.com/epi2me-labs/wf-single-cell/issues/66#issuecomment-1893414514, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZICVRREIW5YM42FIBCZOIDYOZFMPAVCNFSM6AAAAABAKJJ3TCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJTGQYTINJRGQ . You are receiving this because you were mentioned.Message ID: @.***>

nrhorner commented 6 months ago

Great. Thanks for letting us know. I'll close this ticket now

irzamsarfraz commented 6 months ago

@nrhorner This seems to work with one of the smaller samples but with another larger sample, I seem to get error again for some reason.

Nextflow command: nextflow run epi2me-labs/wf-single-cell -r prerelease -w results_1415/workspace -profile singularity --fastq /restricted/projectnb/camplab/projects/20231027_Cam_Oxford_Nanopore/data/raw/231027Cam/PseudoID-1415/20231101_1447_2C_PAQ97447_72c9ab2d/fastq_pass/all_merged/all_merged.fastq --kit_name 5prime --kit_version v1 --expected_cells 100000 --ref_genome_dir refdata-gex-GRCh38-2020-A/ --out_dir results_1415 --max_threads 22 --matrix_min_genes 10 --resources_mm2_max_threads 22 --matrix_min_genes 1 --matrix_min_cells 1 --barcode_max_ed 2 --barcode_min_ed_diff 1 --plot_umaps False --umap_n_repeats 0

Error: Error executing process > 'pipeline:process_bams:tag_bams (16)'

Caused by: Process pipeline:process_bams:tag_bams (16) terminated with an error exit status (1)

Command executed:

workflow-glue tag_bam --in_bam align.bam --tags tags.tsv --out_bam all_merged.chrY.tagged.bam --chrom chrY

samtools index all_merged.chrY.tagged.bam

Command exit status: 1

Command output: (empty)

Command error: /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1063: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1071: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() /home/epi2melabs/conda/lib/python3.8/site-packages/umap/distances.py:1086: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() /home/epi2melabs/conda/lib/python3.8/site-packages/umap/umap_.py:660: NumbaDeprecationWarning: The 'nopython' keyword argument was not supplied to the 'numba.jit' decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See https://numba.readthedocs.io/en/stable/reference/deprecation.html#deprecation-of-object-mode-fall-back-behaviour-when-using-jit for details. @numba.jit() [03:44:37 - workflow_glue] Starting entrypoint. Traceback (most recent call last): File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow-glue", line 7, in cli() File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/init.py", line 72, in cli args.func(args) File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 107, in main add_tags(args.tags, args.in_bam, args.out_bam, args.chrom, args.flip) File "/usr2/collab/isarfraz/.nextflow/assets/epi2me-labs/wf-single-cell/bin/workflow_glue/tag_bam.py", line 80, in add_tags align.set_tag('CR', row['CR'], value_type="Z") File "pysam/libcalignedsegment.pyx", line 2316, in pysam.libcalignedsegment.AlignedSegment.set_tag File "pysam/libcalignedsegment.pyx", line 2400, in pysam.libcalignedsegment.AlignedSegment.set_tag File "pysam/libcutils.pyx", line 134, in pysam.libcutils.force_bytes TypeError: Argument must be string, bytes or unicode.

Work dir: /restricted/projectnb/camplab/home/isarfraz/Nanopore/20231027_Oxford_Nanpore/wf-single-cell/results_1415/workspace/72/d91b6b46cf4680d8cf981ddd800546

Tip: you can replicate the issue by changing to the process work dir and entering the command bash .command.run

Thank you for looking into this!