Index error in the normalization step

[ahalxq]

Hi,

I got an error "IndexError: Unknown indexer True of type <class 'numpy.bool_'>" with the following traceback: Traceback (most recent call last): File "3DVI/scripts/3DVI.py", line 373, in res = [normalize(bandM, cellInfo, chromSelect, bandDist, nLatent, batchFlag, gpuFlag) for bandM, chromSelect, bandDist in bandMiter] File "3DVI/scripts/3DVI.py", line 373, in res = [normalize(bandM, cellInfo, chromSelect, bandDist, nLatent, batchFlag, gpuFlag) for bandM, chromSelect, bandDist in bandMiter] File "3DVI/scripts/3DVI.py", line 157, in normalize scvi.data.setup_anndata(adata) File ".../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/scvi/data/_anndata.py", line 191, in setup_anndata local_l_mean_key, local_l_var_key = _setup_library_size(adata, batch_key, layer) File ".../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/scvi/data/_anndata.py", line 807, in _setup_library_size _compute_library_size_batch( File ".../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/scvi/data/_utils.py", line 73, in _compute_library_size_batch data = adata[idx_batch].X File ".../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/anndata/_core/anndata.py", line 1108, in getitem oidx, vidx = self._normalize_indices(index) File ".../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/anndata/_core/anndata.py", line 1089, in _normalize_indices return _normalize_indices(index, self.obs_names, self.var_names) File ".../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/anndata/_core/index.py", line 35, in _normalize_indices ax0 = _normalize_index(ax0, names0) File ".../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/anndata/_core/index.py", line 107, in _normalize_index raise IndexError(f"Unknown indexer {indexer!r} of type {type(indexer)}")

I have no idea what could cause the error. The command I use is: python 3DVI/scripts/3DVI.py \ -b "${bandMax}" \ -c "${chrom}" \ -r "${resolution}" \ -i "${inputDir}" \ -o "${outputDir}" \ -g "${genomeSize}" \ -n 100 -p "${cpuN}" Could you please help me with it? Thanks a lot!

[yezhengSTAT]

Hello, Were you able to run the demo data using the commands in step 3? I also did not see these errors before.

[ahalxq]

Hi Ye,

Thanks a lot for your timely response! Yes, I am able to run the demo data with no error. The problem I am facing now is trying to use 3DVI+ for our own single-cell HiC data. If possible, could you please help me with it?

The input file looks like this: chr10 70000 chr10 70000 69 chr10 70000 chr10 80000 3 chr10 70000 chr10 90000 1 chr10 70000 chr10 100000 4 chr10 70000 chr10 110000 4

Before running 3DVI+, I concatenated the data files for all chromosomes together, and, since there is only 1 dataset, I did not input the cell summary. Could this be the problem?

Here is the program output before running into this error: Maximum genomic distance: 10 Chromosomes to be processed: chr20 Resolution: 1000000 Path to the input scHi-C data: ../data/Astro/ Path to output directory: ../output/Astro_contact_matrix/ Cell summary file: None Genome size file: ...//hg19.chrom.sizes Remove batch effect or not: False Dimension of latent space: 100 Use GPU or not: False Number of CPUs for parallel computing: 1 PCA number: 10 UMAP plot: True t-SNE plot: True Caculate total number of bin per chromosome. Prepare cell summary file. Read in scHi-C data. Convert interactions into band matrix. 100%|████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2.22it/s] Concat cells into cell x locus-pair matrix. 3DVI normalization. INFO No batch_key inputted, assuming all cells are same batch
.../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/scvi/data/_anndata.py:709: UserWarning: Category 0 in adata.obs['_scvi_batch'] has fewer than 3 cells. SCVI may not train properly. warnings.warn( .../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/scvi/data/_anndata.py:716: UserWarning: Is adata.obs['{}'] continuous? SCVI doesn't support continuous obs yet. warnings.warn( INFO No label_key inputted, assuming all cells have same label
.../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/scvi/data/_anndata.py:709: UserWarning: Category 0 in adata.obs['_scvi_labels'] has fewer than 3 cells. SCVI may not train properly. warnings.warn( INFO Using data from adata.X
INFO Computing library size prior per batch

Thank you so much!

[yezhengSTAT]

Hello, Glad to hear that the demoData is working!

"I concatenated the data files for all chromosomes together, and, since there is only 1 dataset, I did not input the cell summary." Do you mean you concatenate all the cells into one file?

".../anaconda3/envs/schic-3dvi/lib/python3.8/site-packages/scvi/data/_anndata.py:709: UserWarning: Category 0 in adata.obs['_scvi_batch'] has fewer than 3 cells. SCVI may not train properly." Seems that there are fewer than 3 cells are provided?

Please keep all the interactions within each cell in an independent file. You can refer to the demoData folder for reference.

Best, Ye