Error running scDblFinder

[malonzm1]

Hi,

When running scDBlFinder I run into the error: Error in h(simpleError(msg, call)) : error in evaluating the argument 'x' in selecting a method for function 'colSums': Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 102

The code is the following:

from glob import glob
import os
import scanpy as sc
import anndata2ri
import logging

import rpy2.rinterface_lib.callbacks as rcb
import rpy2.robjects as ro

rcb.logger.setLevel(logging.ERROR)
ro.pandas2ri.activate()
anndata2ri.activate()

%load_ext rpy2.ipython
%%R
library(Seurat)
library(scater)
library(scDblFinder)
library(BiocParallel)

gse = 'GSE134649'
infolder = '/scratch/cs/pan-autoimmune/data/counts/scRNAseq/%s/table'%gse
folders = glob('%s/GSM*/'%infolder)
gsms = []
for folder in folders:
    gsm = folder.split('/')[-2]
    gsms.append(gsm)
adatas1 = [sc.read_10x_mtx('%s/%s'%(infolder,gsm),cache=True, var_names='gene_symbols') for gsm in gsms]

%%R -i data_mat -o doublet_score -o doublet_class

set.seed(123)
sce = scDblFinder(
    SingleCellExperiment(
        list(counts=data_mat),
    ) 
)

and the error trace is:

Error in h(simpleError(msg, call)) : 
  error in evaluating the argument 'x' in selecting a method for function 'colSums': Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 102
---------------------------------------------------------------------------
RRuntimeError                             Traceback (most recent call last)
File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/ipython/rmagic.py:385, in RMagics.eval(self, code)
    383 try:
    384     # Need the newline in case the last line in code is a comment.
--> 385     value, visible = ro.r("withVisible({%s\n})" % code)
    386 except (ri.embedded.RRuntimeError, ValueError) as exception:
    387     # Otherwise next return seems to have copy of error.

File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/robjects/__init__.py:459, in R.__call__(self, string)
    458 p = rinterface.parse(string)
--> 459 res = self.eval(p)
    460 return conversion.get_conversion().rpy2py(res)

File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/robjects/functions.py:208, in SignatureTranslatedFunction.__call__(self, *args, **kwargs)
    207         kwargs[r_k] = v
--> 208 return (super(SignatureTranslatedFunction, self)
    209         .__call__(*args, **kwargs))

File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/robjects/functions.py:131, in Function.__call__(self, *args, **kwargs)
    130         new_kwargs[k] = cv.py2rpy(v)
--> 131 res = super(Function, self).__call__(*new_args, **new_kwargs)
    132 res = cv.rpy2py(res)

File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/rinterface_lib/conversion.py:45, in _cdata_res_to_rinterface.<locals>._(*args, **kwargs)
     44 def _(*args, **kwargs):
---> 45     cdata = function(*args, **kwargs)
     46     # TODO: test cdata is of the expected CType

File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/rinterface.py:817, in SexpClosure.__call__(self, *args, **kwargs)
    816     if error_occured[0]:
--> 817         raise embedded.RRuntimeError(_rinterface._geterrmessage())
    818 return res

RRuntimeError: Error in h(simpleError(msg, call)) : 
  error in evaluating the argument 'x' in selecting a method for function 'colSums': Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 102

During handling of the above exception, another exception occurred:

RInterpreterError                         Traceback (most recent call last)
Cell In[8], line 1
----> 1 get_ipython().run_cell_magic('R', '-i data_mat -o doublet_score -o doublet_class', '\nset.seed(123)\nsce = scDblFinder(\n    SingleCellExperiment(\n        list(counts=data_mat),\n    ) \n)\ndoublet_score = sce$scDblFinder.score\ndoublet_class = sce$scDblFinder.class\n')

File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/IPython/core/interactiveshell.py:2493, in InteractiveShell.run_cell_magic(self, magic_name, line, cell)
   2491 with self.builtin_trap:
   2492     args = (magic_arg_s, cell)
-> 2493     result = fn(*args, **kwargs)
   2495 # The code below prevents the output from being displayed
   2496 # when using magics with decorator @output_can_be_silenced
   2497 # when the last Python token in the expression is a ';'.
   2498 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False):

File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/ipython/rmagic.py:943, in RMagics.R(self, line, cell, local_ns)
    941     if not e.stdout.endswith(e.err):
    942         print(e.err)
--> 943     raise e
    944 finally:
    945     if self.device in DEVICES_STATIC:

File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/ipython/rmagic.py:923, in RMagics.R(self, line, cell, local_ns)
    921         return_output = False
    922 else:
--> 923     text_result, result, visible = self.eval(code)
    924     text_output += text_result
    925     if visible:

File /scratch/work/malonzm1/.conda_envs/R_for_scater/lib/python3.9/site-packages/rpy2/ipython/rmagic.py:389, in RMagics.eval(self, code)
    386 except (ri.embedded.RRuntimeError, ValueError) as exception:
    387     # Otherwise next return seems to have copy of error.
    388     warning_or_other_msg = self.flush()
--> 389     raise RInterpreterError(code, str(exception),
    390                             warning_or_other_msg)
    391 text_output = self.flush()
    392 return text_output, value, visible[0]

RInterpreterError: Failed to parse and evaluate line '\nset.seed(123)\nsce = scDblFinder(\n    SingleCellExperiment(\n        list(counts=data_mat),\n    ) \n)\ndoublet_score = sce$scDblFinder.score\ndoublet_class = sce$scDblFinder.class\n'.
R error message: "Error in h(simpleError(msg, call)) : \n  error in evaluating the argument 'x' in selecting a method for function 'colSums': Cholmod error 'problem too large' at file ../Core/cholmod_dense.c, line 102"

Please advise.

Thanks and good day.

[plger]

Hi, I don't really understand how exactly your code passes the data to R, but 1) my guess would be that somewhere in there your matrix is converted to dense format and that gets too large. 2) it looks like you have multiple samples merged together and then pass the whole thing to scDblFinder without telling it about the samples. If that's the case, that's very bad. You should either pass the samples one by one, or provide the samples argument.

[plger]

Will close this issue unless you have an update.

[malonzm1]

Hi,

Thanks. I filtered the data before running scDblFinder and the error stopped.