search

theislab / cellrank

CellRank: dynamics from multi-view single-cell data
https://cellrank.org
BSD 3-Clause "New" or "Revised" License
347 stars 46 forks source link

Ipython kernal dies when calculating Schur decomposition #754

Closed lijxug closed 3 years ago

lijxug commented 3 years ago

When running cellrank, specifically on the step 'g.compute_schur(n_components=20)' or when 'cr.tl.terminal_states' hits the step 'Computing Schur decomposition', my jupyter lab always warns me that the kernal have died and restarted.

I'd like to paste the error messages here but there is none in the notebook. Could you help me troubleshooting?

Versions:

cellrank==1.5.0 scanpy==1.8.1 anndata==0.7.6 numpy==1.20.3 numba==0.54.0 scipy==1.7.1 pandas==1.3.2 pygpcca==1.0.2 scikit-learn==0.24.2 statsmodels==0.12.2 python-igraph==0.9.6 scvelo==0.2.4 pygam==0.8.0 matplotlib==3.3.4 seaborn==0.11.1

michalk8 commented 3 years ago

Did you install petsc4py/slepc4py as e.g. pip install 'cellrank[krylov]'? If not, the crash can be due to the fact that if PETSc is not installed, the full Schur decomposition is calculated using scipy, for which the transition matrix is densified (this can crash the Python process, esp. for large number of cells). In any case, I'd check the logs in terminal from jupyter and/or also increase the verbosity as:

import cellrank
cr.settings.verbosity = 4
michalk8 commented 3 years ago

Do you still experience the issue @lijxug ?

lijxug commented 3 years ago

Thanks for the reply @michalk8 .

It turns out that my petsc4py wasn't installed properly. This error message pops up during the installation:

Using cached petsc4py-3.13.0.tar.gz (1.9 MB)
ERROR: Command errored out with exit status 1:
command: /data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mdggd082/petsc4py_2bbdf70aa70342bbb903da643ba75998/setup.py'"'"'; file='"'"'/tmp/pip-install-m dggd082/petsc4py_2bbdf70aa70342bbb903da643ba75998/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.cl ose();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-2wpdvqzp
cwd: /tmp/pip-install-mdggd082/petsc4py_2bbdf70aa70342bbb903da643ba75998/
Complete output (52 lines):
running egg_info
creating /tmp/pip-pip-egg-info-2wpdvqzp/petsc4py.egg-info
writing /tmp/pip-pip-egg-info-2wpdvqzp/petsc4py.egg-info/PKG-INFO
writing dependency_links to /tmp/pip-pip-egg-info-2wpdvqzp/petsc4py.egg-info/dependency_links.txt
writing requirements to /tmp/pip-pip-egg-info-2wpdvqzp/petsc4py.egg-info/requires.txt
writing top-level names to /tmp/pip-pip-egg-info-2wpdvqzp/petsc4py.egg-info/top_level.txt
writing manifest file '/tmp/pip-pip-egg-info-2wpdvqzp/petsc4py.egg-info/SOURCES.txt' Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-mdggd082/petsc4py_2bbdf70aa70342bbb903da643ba75998/setup.py", line 270, in
main()
File "/tmp/pip-install-mdggd082/petsc4py_2bbdf70aa70342bbb903da643ba75998/setup.py", line 267, in main
run_setup()
File "/tmp/pip-install-mdggd082/petsc4py_2bbdf70aa70342bbb903da643ba75998/setup.py", line 119, in run_setup
setup(packages = ['petsc4py',
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/site-packages/setuptools/init.py", line 153, in setup
return distutils.core.setup(**attrs)
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/distutils/core.py", line 148, in setup
dist.run_commands()
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/distutils/dist.py", line 966, in run_commands
self.run_command(cmd)
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/distutils/dist.py", line 985, in run_command
cmd_obj.run()
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 299, in run
self.find_sources()
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 306, in find_sources mm.run()
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 541, in run
self.add_defaults()
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 577, in add_defaults
sdist.add_defaults(self)
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/distutils/command/sdist.py", line 226, in add_defaults
self._add_defaults_python()
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/site-packages/setuptools/command/sdist.py", line 109, in _add_defaults_python
build_py = self.get_finalized_command('build_py')
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/distutils/cmd.py", line 299, in get_finalized_command
cmd_obj.ensure_finalized()
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/distutils/cmd.py", line 107, in ensure_finalized
self.finalize_options()
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/site-packages/setuptools/command/build_py.py", line 36, in finalize_options
orig.build_py.finalize_options(self)
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/distutils/command/build_py.py", line 43, in finalize_options
self.set_undefined_options('build',
File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/distutils/cmd.py", line 287, in set_undefined_options
src_cmd_obj.ensure_finalized() File "/data1/jason/tools/softwares/anaconda3/envs/singlecell_python3.8/lib/python3.8/distutils/cmd.py", line 107, in ensure_finalized self.finalize_options() File "/tmp/pip-install-mdggd082/petsc4py_2bbdf70aa70342bbb903da643ba75998/conf/baseconf.py", line 398, in finalize_options self.petsc_dir = config.get_petsc_dir(self.petsc_dir) File "/tmp/pip-install-mdggd082/petsc4py_2bbdf70aa70342bbb903da643ba75998/conf/baseconf.py", line 336, in get_petsc_dir petsc_dir = petsc.get_petsc_dir() AttributeError: module 'petsc' has no attribute 'get_petsc_dir'

WARNING: Discarding https://files.pythonhosted.org/packages/7c/e7/5b089013c5188ee5f619ad64749fc3e6355943950dfcf421c327d66ee2ac/petsc4py-3.13.0.tar.gz#sha256=ace21f71102e752fefac6c81b65207edba89a6974b3c58c71599c6c358640f39 (from https://pypi.org/simple/petsc4py/). Comman d errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

I am still struggling to fix this. Although I understand that this might be a totally different issue, I'd still like your opinion on this if you have seen this before.

michalk8 commented 3 years ago

The error is most likely because petsc4py must be installed after petsc and it isn't. For Linux, you can try the steps mentioned in: https://cellrank.readthedocs.io/en/stable/installation.html#dependencies Alternatively, you can also try using the conda package, see: https://cellrank.readthedocs.io/en/stable/installation.html#bioconda If none of the above works, please take a look at PETSc forums here or here - there are similar solutions that also might be helpful.

Marius1311 commented 3 years ago

Any progress on this @lijxug?

lijxug commented 3 years ago

Sorry for the delay. Yes, I uninstalled the packages mentioned by @michalk8 and then reinstalled them. Now cellrank is working as expected.

Marius Lange @.***> 于2021年10月6日周三 上午12:29写道：

Any progress on this @lijxug https://github.com/lijxug?