ValueError: cannot infer dimensions from zero sized index arrays

[OneHitKO]

Hello, thanks for developing a really nice tool!

I've ran cellbender remove-background with no issues on multiome data while keeping both Gene Expression and Peaks features. However, I'm currently rerunning it on the same dataset with --exclude-feature-types Peaks.

When excluding peaks, I receive the following error after cellbender creates the posterior.h5 file:

cellbender:remove-background: Computing target noise counts per gene for MCKP estimator
cellbender:remove-background: Using MCKP noise targets computed for FPR 0.01
cellbender:remove-background: Computing denoised counts using mckp estimator
cellbender:remove-background: Working on chunk (1/35)
cellbender:remove-background:     [1.94 mins per chunk]
cellbender:remove-background: Working on chunk (2/35)
cellbender:remove-background: Working on chunk (3/35)
cellbender:remove-background: Working on chunk (4/35)
cellbender:remove-background: Working on chunk (5/35)
cellbender:remove-background: Working on chunk (6/35)
cellbender:remove-background: Working on chunk (7/35)
cellbender:remove-background: Working on chunk (8/35)
cellbender:remove-background: Working on chunk (9/35)
Traceback (most recent call last):
  File "/g/saka/Kristy/software/envs/cellbender/bin/cellbender", line 8, in <module>
    sys.exit(main())
  File "/home/kou/CellBender/cellbender/base_cli.py", line 118, in main
    cli_dict[args.tool].run(args)
  File "/home/kou/CellBender/cellbender/remove_background/cli.py", line 185, in run
    return main(args)
  File "/home/kou/CellBender/cellbender/remove_background/cli.py", line 219, in main
    posterior = run_remove_background(args)
  File "/home/kou/CellBender/cellbender/remove_background/run.py", line 152, in run_remove_background
    file_name=file_name,
  File "/home/kou/CellBender/cellbender/remove_background/run.py", line 300, in compute_output_denoised_counts_reports_metrics
    use_multiple_processes=args.use_multiprocessing_estimation,
  File "/home/kou/CellBender/cellbender/remove_background/posterior.py", line 291, in compute_denoised_counts
    **kwargs,
  File "/home/kou/CellBender/cellbender/remove_background/estimation.py", line 523, in estimate_noise
    noise_log_prob_coo=_subset_coo(noise_log_prob_coo, logic),
  File "/home/kou/CellBender/cellbender/remove_background/estimation.py", line 873, in _subset_coo
    return sp.coo_matrix((coo.data[logic], (coo.row[logic], coo.col[logic])))
  File "/g/saka/Kristy/software/envs/cellbender/lib/python3.7/site-packages/scipy/sparse/coo.py", line 147, in __init__
    raise ValueError('cannot infer dimensions from zero '
ValueError: cannot infer dimensions from zero sized index arrays

Here is the exact command:

cellbender remove-background --input cellranger/LN0025/outs/raw_feature_bc_matrix.h5 --output cellbender/LN0025/out_gex.h5 --expected-cells 10000 --exclude-feature-types Peaks --cuda

Any feedback how I can overcome this error, which was not generated before? I'm using CellBender 0.3.1 (Workflow hash e9eeea4787)

Below is the UMI counts knee plot in case it is helpful. Thanks very much for the help!

[Wang-Yongqi]

Hi I've run into the same problem with two different datasets, the problem seems to be also with --exclude-feature-types Peaks. It always reports error after the 9th chunk here.

[Wang-Yongqi]

Also another line of clue, this issue seems to be Win specific. I successfully ran through on a Mac machine with exactly the same code. Hope it helps.

[OneHitKO]

Interesting, my code fails in Linux, but I can try on MacOS. Thanks!

[Baboon61]

cellbender 0.3.1 was failing for me as well at the same chunk on Linux with --exclude-feature-types Peaks flag. Downgrading to cellbender 0.3.0 did not raise the issue.