Pyscenic ctx taking a long time

willrosenow commented 3 years ago

I have been running pyscenic ctx for over 5 days. My unfiltered loom file contains ~7k cells. How long does this step typically take? I am running it from a slurm script on my HPC using the following script:

pyscenic ctx files/adj.csv \ cisTarget_databases/mm10refseq-r80500bp_up_and_100bp_down_tss.mc9nr.feather \ --annotations_fname cisTarget_databases/motifs-v9-nr.mgi-m0.001-o0.0.tbl.txt \ --expression_mtx_fname files/ob_unfiltered.loom \ --output files/ob_pyscenic.csv \ --mask_dropouts \ --num_workers 39

In the output, it gets to the Calculating regulons step, but stops there.

2021-02-08 08:30:12,939 - pyscenic.cli.pyscenic - INFO - Calculating regulons.

Do you have any recommendations on making this script run faster?

Thanks, Will

cflerin commented 3 years ago

There's not really enough information here to help. At minimum you should provide the following from the bug report template:

pySCENIC version: [e.g. 0.10.0, use pyscenic -h at the command line]
Installation method: [e.g. Conda, Pip, Docker, Singularity]
Run environment: [e.g. Jupyter notebook, CLI, Nextflow, HPC script]
OS: [e.g. Ubuntu]
Package versions: [obtain using pip freeze, conda list, or skip this if using Docker/Singularity]:
```
...
```

willrosenow commented 3 years ago

Here are the details:

pySCENIC version: 0.10.4

Installation method: Pip

Run environment: HPC slurm script

OS: CentOS

Package versions: adjustText==0.7.3 alabaster==0.7.12 anaconda-client==1.7.2 anaconda-navigator==1.9.7 anaconda-project==0.8.3 anndata==0.7.5 arboreto==0.1.5 asn1crypto==1.0.1 astroid==2.3.1 astropy==3.2.2 atomicwrites==1.3.0 attrs==19.2.0 Babel==2.7.0 backcall==0.1.0 backports.os==0.1.1 backports.shutil-get-terminal-size==1.0.0 beautifulsoup4==4.8.0 biothings-client==0.2.3 bitarray==1.0.1 bkcharts==0.2 blaze==0.11.3 bleach==3.1.0 bokeh==1.3.4 boltons==20.1.0 boto==2.49.0 Bottleneck==1.2.1 cellbrowser==0.7.15 cellex==1.2.1 certifi==2019.9.11 cffi==1.12.3 chardet==3.0.4 Click==7.0 cloudpickle==1.2.2 clyent==1.2.2 colorama==0.4.1 conda==4.8.0 conda-build==3.18.11 conda-package-handling==1.6.0 conda-verify==3.1.1 contextlib2==0.6.0 cryptography==2.7 cycler==0.10.0 Cython==0.29.13 cytoolz==0.10.0 dask==1.0.0 datashape==0.5.4 decorator==4.4.0 defusedxml==0.6.0 descartes==1.1.0 dill==0.3.1.1 distributed==1.28.1 docutils==0.15.2 entrypoints==0.3 et-xmlfile==1.0.1 fa2==0.3.5 fastcache==1.1.0 fenics-dijitso==2019.1.0 fenics-ffc==2019.1.0.post0 fenics-fiat==2019.1.0 fenics-ufl==2019.1.0 filelock==3.0.12 Flask==1.1.1 Flask-Cors==3.0.8 frozendict==1.2 fsspec==0.5.2 future==0.18.2 get-version==2.1 gevent==1.4.0 glob2==0.7 gmpy2==2.0.8 greenlet==0.4.15 h5py==2.10.0 HeapDict==1.0.1 html5lib==1.0.1 idna==2.8 imageio==2.6.0 imagesize==1.1.0 importlib-metadata==0.23 interlap==0.2.6 ipykernel==5.1.2 ipyparallel==6.2.4 ipython==7.8.0 ipython-genutils==0.2.0 ipywidgets==7.5.1 isort==4.3.21 itsdangerous==1.1.0 jdcal==1.4.1 jedi==0.15.1 jeepney==0.4.1 Jinja2==2.10.3 joblib==0.13.2 json5==0.8.5 jsonschema==3.0.2 jupyter==1.0.0 jupyter-client==5.3.3 jupyter-console==6.0.0 jupyter-core==4.5.0 jupyterlab==1.1.4 jupyterlab-launcher==0.13.1 jupyterlab-server==1.0.6 jupytext==1.3.2 keyring==18.0.0 kiwisolver==1.1.0 lazy-object-proxy==1.4.2 legacy-api-wrap==1.2 leidenalg==0.8.3 libarchive-c==2.8 lief==0.9.0 llvmlite==0.29.0 locket==0.2.0 loompy==3.0.6 lxml==4.4.1 MarkupSafe==1.1.1 matplotlib==3.2.1 mccabe==0.6.1 metakernel==0.24.3 mistune==0.8.4 mizani==0.7.2 mkl-fft==1.0.14 mkl-random==1.1.0 mkl-service==2.3.0 mock==3.0.5 more-itertools==7.2.0 mpmath==1.1.0 msgpack==0.6.1 MulticoreTSNE==0.1 multipledispatch==0.6.0 multiprocessing-on-dill==3.5.0a4 mygene==3.1.0 natsort==7.0.1 navigator-updater==0.2.1 nb-conda-kernels==2.2.2 nbconvert==5.6.0 nbformat==4.4.0 networkx==2.3 nltk==3.4.5 nose==1.3.7 notebook==6.0.1 numba==0.45.1 numexpr==2.7.0 numpy==1.17.2 numpy-groupies==0+unknown numpydoc==0.9.1 odo==0.5.1 olefile==0.46 opencv-python==4.1.2.30 openpyxl==3.0.0 packaging==19.2 palettable==3.3.0 pandas==0.25.3 pandocfilters==1.4.2 parso==0.5.1 partd==1.0.0 path.py==12.0.1 pathlib2==2.3.5 patsy==0.5.1 pbr==5.4.2 pep8==1.7.1 pexpect==4.7.0 pickleshare==0.7.5 Pillow==6.2.0 pkgconfig==1.5.1 pkginfo==1.5.0.1 plotnine==0.7.1 pluggy==0.13.0 ply==3.11 portalocker==1.5.2 prometheus-client==0.7.1 prompt-toolkit==2.0.10 psutil==5.6.3 ptyprocess==0.6.0 py==1.8.0 pyarrow==0.16.0 pycodestyle==2.5.0 pycosat==0.6.3 pycparser==2.19 pycrypto==2.6.1 pycurl==7.43.0.3 pyflakes==2.1.1 Pygments==2.4.2 pylint==2.4.2 pyodbc==4.0.27 pyOpenSSL==19.0.0 pyparsing==2.4.2 pyrsistent==0.15.4 pysam==0.16.0.1 pyscenic==0.10.4 PySocks==1.7.1 pytest==5.2.1 pytest-arraydiff==0.3 pytest-astropy==0.5.0 pytest-doctestplus==0.4.0 pytest-openfiles==0.4.0 pytest-remotedata==0.3.2 python-dateutil==2.8.0 python-igraph==0.8.3 pytz==2019.3 PyWavelets==1.0.3 PyYAML==5.1.2 pyzmq==18.1.0 QtAwesome==0.6.0 qtconsole==4.5.5 QtPy==1.9.0 requests==2.22.0 rope==0.14.0 ruamel-yaml==0.15.46 scanpy==1.7.0rc1 scikit-image==0.15.0 scikit-learn==0.21.3 scipy==1.6.0 scvelo==0.2.2 seaborn==0.9.0 SecretStorage==3.1.1 Send2Trash==1.5.0 setuptools-scm==3.5.0 simplegeneric==0.8.1 sinfo==0.3.1 singledispatch==3.4.0.3 six==1.12.0 snowballstemmer==2.0.0 sortedcollections==1.1.2 sortedcontainers==2.1.0 soupsieve==1.9.3 Sphinx==2.2.0 sphinxcontrib-applehelp==1.0.1 sphinxcontrib-devhelp==1.0.1 sphinxcontrib-htmlhelp==1.0.2 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.2 sphinxcontrib-serializinghtml==1.1.3 sphinxcontrib-websupport==1.1.2 spyder==3.3.6 spyder-kernels==0.5.2 SQLAlchemy==1.3.9 statsmodels==0.12.1 stdlib-list==0.8.0 sympy==1.4 tables==3.6.1 tbb==2020.0.133 tblib==1.4.0 terminado==0.8.2 testfixtures==6.10.0 testpath==0.4.2 texttable==1.6.3 toolz==0.10.0 tornado==6.0.3 tqdm==4.36.1 traitlets==4.3.3 typing==3.6.4 umap-learn==0.4.2 unicodecsv==0.14.1 urllib3==1.24.2 wcwidth==0.1.7 webencodings==0.5.1 Werkzeug==0.16.0 widgetsnbextension==3.5.1 wrapt==1.11.2 wurlitzer==1.0.3 xarray==0.14.1 xlrd==1.2.0 XlsxWriter==1.2.1 xlwt==1.3.0 zict==1.0.0 zipp==0.6.0

cflerin commented 3 years ago

Thanks for the info. I would recommend:

Update to pySCENIC 0.11.0 (there are package updates that could resolve this issue)
If the new version still doesn't help, you could try the multiprocessing version instead of using dask. You can do this by using the --mode "custom_multiprocessing" option in the CLI command.

(similar to #142)

willrosenow commented 3 years ago

Thank you @cflerin !

I'll try both of those suggestions.

aertslab / pySCENIC

Pyscenic ctx taking a long time #265