search

broadinstitute / CellBender

CellBender is a software package for eliminating technical artifacts from high-throughput single-cell RNA sequencing (scRNA-seq) data.
https://cellbender.rtfd.io
BSD 3-Clause "New" or "Revised" License
271 stars 50 forks source link

Index Error in priors.py #343

Open Ratoncito opened 3 months ago

Ratoncito commented 3 months ago

Ran ~ 70 .h5 files through cellbender sbatch array script and all but one file completed successfully. One file gave the following error: 

cellbender:remove-background: Command:
cellbender remove-background --cuda --input ../../data/D22-5737_raw_feature_bc_matrix.h5 --output D22-5737_background_removed_raw_feature_bc_matrix.h5
cellbender:remove-background: CellBender 0.3.0
cellbender:remove-background: (Workflow hash 6c01e194fa)
cellbender:remove-background: 2024-03-26 11:10:27
cellbender:remove-background: Running remove-background
cellbender:remove-background: Loading data from ../../data/D22-5737_raw_feature_bc_matrix.h5
cellbender:remove-background: CellRanger v3 format
cellbender:remove-background: Features in dataset: 32285 Gene Expression
cellbender:remove-background: Trimming features for inference.
cellbender:remove-background: 7855 features have nonzero counts.
Traceback (most recent call last):
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/bin/cellbender", line 8, in <module>
    sys.exit(main())
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/base_cli.py", line 118, in main
    cli_dict[args.tool].run(args)
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/cli.py", line 185, in run
    return main(args)
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/cli.py", line 230, in main
    posterior = run_remove_background(args)
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/run.py", line 84, in run_remove_background
    dataset_obj = get_dataset_obj(args=args)
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/data/dataset.py", line 538, in get_dataset_obj
    fpr=args.fpr,
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/data/dataset.py", line 116, in __init__
    self.priors = get_priors(umi_counts=counts, low_count_threshold=low_count_threshold)
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/data/priors.py", line 337, in get_priors
    low_count_threshold=low_count_threshold,
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/data/priors.py", line 237, in get_cell_count_empty_count
    cell_count_low_limit=cell_count_low_limit,
  File "/net/vast-storage.ib.cluster/scratch/vast/kellislab/shared/software/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/data/priors.py", line 98, in _peak_density_given_cutoff
    noncell_counts = np.sort(noncell_counts)[::-1][inds]
IndexError: index 0 is out of bounds for axis 0 with size 0

Turns out this sample is just really really poor quality and (probably) none of the cells passed the min requirements for number of features or counts. Maybe the cellbender dev team could include an error catch or message at one of the above lines of code to help future users figure this out more quickly. Or even run a quick QC on the samples before they are run.

image

Or I could be wrong and this is not the reason it failed. Not 100% sure but thought I would share my experience with the community.

sjfleming commented 2 months ago

Thanks for sharing @Ratoncito , and yes I think you are right! I should fix this so that it gives a real error message and not a cryptic IndexError.

albert239825 commented 4 weeks ago

Hi, I am also getting this issue, however, I have a pretty big dataset and don't think the lack of quality cells is the issue. I have run it on the the original dataset as well as the dataset after QC and arrived at the same issue. https://github.com/broadinstitute/CellBender/issues/190#issuecomment-2153366445