Issue with finding variable genes

[SathiyaNManivannan]

Hello,

I am using anaconda3 on a Windows10 machine with Spyder interface to run Stream. When I try to use my data to identify Variable genes, I get the following warning and then no variable genes are selected.

.\anaconda3\lib\site-packages\statsmodels\nonparametric\smoothers_lowess.py:165: RuntimeWarning: invalid value encountered in true_divide res = _lowess(y, x, frac=frac, it=it, delta=delta) .\anaconda3\lib\site-packages\statsmodels\nonparametric\smoothers_lowess.py:165: RuntimeWarning: invalid value encountered in greater_equal res = _lowess(y, x, frac=frac, it=it, delta=delta) .\anaconda3\lib\site-packages\stream\core.py:523: RuntimeWarning: invalid value encountered in less mat_sign[np.where(residuals<0)[0]] = -1 .\anaconda3\lib\site-packages\stream\core.py:528: RuntimeWarning: invalid value encountered in less_equal id_non_var_genes = np.where(residuals<=cutoff)[0] 0 variable genes are selected

I believe this may be due to Zero or NA values in the adata file. The error is consistent across different 'loess_frac' values. I am not sure how to address this issue.

PCA analysis works fine with this data. But, I wish to see if Variable gene-based analysis gives better results.

Thanks,

Sincerely,

Sathiya

[huidongchen]

Hi Sathiya,

Have you normalized and log-transformed your expression matrix before running st.select_variable_genes?

[SathiyaNManivannan]

Yes! Please see below the script up to which I do not seem to get an error.

import os os.environ["R_USER"]= "SATHIYA" import stream as st adata = st.read(file_name= "./outs/filtered_gene_bc_matrices/mouse_GRCm38.93.gtf/matrix.mtx", file_format='mtx') st.set_workdir(adata,"./Stream") st.add_cell_labels(adata) st.add_cell_colors(adata) st.add_metadata(adata,file_name="./meta_data_for_STREAM.txt") st.normalize_per_cell(adata) st.log_transform(adata) st.remove_mt_genes(adata) st.filter_cells(adata)

error is found after

st.select_variable_genes(adata)

[huidongchen]

hmmm, tbh this error is quite new to me. Would you mind sharing a minimum example data to reproduce the error? I'm happy to look into that. It's hard for me to tell based on the given error info.

[huidongchen]

I just recalled another previous case. #50

Do you think this might help solve your issue?

[SathiyaNManivannan]

Hi,

First and foremost, I really appreciate your prompt help with this issue. But, trying #50 does not seem to help. Trying to convert the array to a dense array throws an error. "import numpy as np

adata.X=np.array(adata.X.todense()) Traceback (most recent call last):

File "", line 1, in adata.X=np.array(adata.X.todense())

AttributeError: 'numpy.ndarray' object has no attribute 'todense'"

Please find attached example data.

example_data.txt

I am getting the exact same error when I try to find variable genes.

[huidongchen]

Hi Sathiya,

I tried to run STREAM on your example data. I actually don't see any errors. Attached is the notebook i'm using.

test.ipynb.zip