IndexError: index 0 is out of bounds for dimension 1 with size 0

[YingYa]

Hi, I am running into IndexError and ValueError errors for some samples after traning.

I run the following command with us.gcr.io/broad-dsde-methods/cellbender:20230403: cellbender remove-background --input test2.RawMatrix.h5ad --output test2.RawMatrix.remove_background.round01.h5 --cuda --posterior-batch-size 64 --z-dim 20 --z-layers 100 --total-droplets-included 80000 --empty-drop-training-fraction 0.4 --training-fraction 0.8 --learning-rate 0.00005

And get this error message:

cellbender:remove-background: Posterior not currently included in checkpoint. cellbender:remove-background: Computing posterior noise count probabilities in mini-batches... cellbender:remove-background: Saved posterior as ./test2.RawMatrix.remove_background.round01_posterior.npz cellbender:remove-background: Added posterior object to checkpoint file. cellbender:remove-background: Unable to save all plots. cellbender:remove-background: Traceback (most recent call last): File "/software/cellbender/cellbender/remove_background/run.py", line 183, in save_output_plots fig = plot_summary(loss=inferred_model.loss, umi_counts=counts, p=p, z=z) File "/software/cellbender/cellbender/remove_background/run.py", line 245, in plot_summary plt.plot(z_pca[:, 0], z_pca[:, 1], IndexError: index 0 is out of bounds for dimension 1 with size 0

cellbender:remove-background: Saved cell barcodes in ./test2.RawMatrix.remove_background.round01_cell_barcodes.csv cellbender:remove-background: Computing target noise counts per gene for MCKP estimator Traceback (most recent call last): File "/opt/conda/bin/cellbender", line 33, in sys.exit(load_entry_point('cellbender', 'console_scripts', 'cellbender')()) File "/software/cellbender/cellbender/base_cli.py", line 112, in main cli_dict[args.tool].run(args) File "/software/cellbender/cellbender/remove_background/cli.py", line 181, in run main(args) File "/software/cellbender/cellbender/remove_background/cli.py", line 226, in main run_remove_background(args) File "/software/cellbender/cellbender/remove_background/run.py", line 132, in run_remove_background compute_output_denoised_counts_reports_metrics( File "/software/cellbender/cellbender/remove_background/run.py", line 299, in compute_output_denoised_counts_reports_metrics noise_target_fun_per_cell = compute_mean_target_removal_as_function( File "/software/cellbender/cellbender/remove_background/posterior.py", line 1625, in compute_mean_target_removal_as_function mean_noise_csr = estimator.estimate_noise( File "/software/cellbender/cellbender/remove_background/estimation.py", line 134, in estimate_noise result = apply_function_dense_chunks(noise_log_prob_coo=noise_log_prob_coo, File "/software/cellbender/cellbender/remove_background/estimation.py", line 410, in apply_function_dense_chunks for coo, row in chunked_iterator(coo=noise_log_prob_coo): File "/software/cellbender/cellbender/remove_background/estimation.py", line 366, in chunked_iterator for i in range(0, coo.row.max(), batch_size): File "/opt/conda/lib/python3.8/site-packages/numpy/core/_methods.py", line 40, in _amax return umr_maximum(a, axis, None, out, keepdims, initial, where) ValueError: zero-size array to reduction operation maximum which has no identity

The UMI curve was posted here:

[sjfleming]

So this dataset looks like it might be more problematic. What is your belief about the number of cells in this experiment? I cannot see by eye where the cells are and where they empty droplets are. Unless you're doing something very unusual, I'd say this might be a QC failure.

Now, that being said, you still should not be getting these kinds of errors from the code, so I will fix that. Thanks for reporting!

[YingYa]

Hi @sjfleming, I am analyzing some single nuclear sequencing experiments, the expected cell for our experiment is about 10k cells, the targeting 100k cells is very very overloaded.

Yes, this sample looks like QC failure. I am trying to filter artifacts that this sample may be kept for the following analysis.

[sjfleming]

Hi @YingYa , I have made an attempt to fix this bug, but now the code will give you a RuntimeError that says "No cells found" instead of this confusing IndexError and ValueError.

The problem here is that CellBender did not find any cells, and I do not think it will find cells even after the minor code fixes I've recently added, since I suspect there might not be any cells present in this experiment. It looks like a sample where it's possible that all the nuclei accidentally lysed or something, and all you've got left is ambient RNA.

I do think that this is a QC failure, and unfortunately CellBender cannot really do much with these kinds of samples.

[YingYa]

Hi @sjfleming, thanks for your help. I will check this sample again, and maybe a new experiment of this sample for comparing.