aertslab / pycisTopic

pycisTopic is a Python module to simultaneously identify cell states and cis-regulatory topics from single cell epigenomics data.
Other
56 stars 10 forks source link

Numba ConstantInferenceError in find_diff_features [BUG] #94

Open josephineyates opened 11 months ago

josephineyates commented 11 months ago

First, I'd like thank you very much for the great work and package you created! I am trying to run the different steps in the Multiome tutorial (https://scenicplus.readthedocs.io/en/latest/pbmc_multiome_tutorial.html) and am getting stuck at the DAR step.

When I run:

from pycisTopic.diff_features import *
imputed_acc_obj = impute_accessibility(cistopic_obj, selected_cells=None, selected_regions=None, scale_factor=10**6)
normalized_imputed_acc_obj = normalize_scores(imputed_acc_obj, scale_factor=10**4)
variable_regions = find_highly_variable_features(normalized_imputed_acc_obj, plot = False)
markers_dict = find_diff_features(cistopic_obj, imputed_acc_obj, variable='highlevel_annotation', var_features=variable_regions, split_pattern = '-')

I get the following error:

2023-09-19 15:30:00,761 cisTopic     INFO     Imputing region accessibility
2023-09-19 15:30:00,762 cisTopic     INFO     Impute region accessibility for regions 0-20000
2023-09-19 15:30:00,902 cisTopic     INFO     Impute region accessibility for regions 20000-40000
2023-09-19 15:30:01,010 cisTopic     INFO     Impute region accessibility for regions 40000-60000
2023-09-19 15:30:01,132 cisTopic     INFO     Impute region accessibility for regions 60000-80000
2023-09-19 15:30:01,258 cisTopic     INFO     Impute region accessibility for regions 80000-100000
2023-09-19 15:30:01,382 cisTopic     INFO     Impute region accessibility for regions 100000-120000
2023-09-19 15:30:01,464 cisTopic     INFO     Done!
2023-09-19 15:30:01,465 cisTopic     INFO     Normalizing imputed data
2023-09-19 15:30:04,534 cisTopic     INFO     Done!
2023-09-19 15:30:04,535 cisTopic     INFO     Calculating mean
2023-09-19 15:30:04,704 cisTopic     INFO     Calculating variance
2023-09-19 15:30:08,025 cisTopic     INFO     Done!
2023-09-19 15:30:08,495 cisTopic     INFO     Subsetting data for Carcinoma (653 of 1479)
---------------------------------------------------------------------------
ConstantInferenceError                    Traceback (most recent call last)
Cell In[29], line 5
      3 normalized_imputed_acc_obj = normalize_scores(imputed_acc_obj, scale_factor=10**4)
      4 variable_regions = find_highly_variable_features(normalized_imputed_acc_obj, plot = False)
----> 5 markers_dict = find_diff_features(cistopic_obj, imputed_acc_obj, variable='highlevel_annotation', var_features=variable_regions, split_pattern = '-')

File ~/opt/anaconda3/envs/spatial/lib/python3.9/site-packages/pycisTopic/diff_features.py:800, in find_diff_features(cistopic_obj, imputed_features_obj, variable, var_features, contrasts, adjpval_thr, log2fc_thr, split_pattern, n_cpu, **kwargs)
    798     ray.shutdown()
    799 else:
--> 800     markers_list = [
    801         markers(
    802             subset_imputed_features_obj,
    803             barcode_groups[i],
    804             contrasts_names[i],
    805             adjpval_thr=adjpval_thr,
    806             log2fc_thr=log2fc_thr,
    807             n_cpu=1,
    808         )
    809         for i in range(len(contrasts))
    810     ]
    812 markers_dict = {
    813     contrasts_name: marker
    814     for contrasts_name, marker in zip(contrasts_names, markers_list)
    815 }
    817 return markers_dict

File ~/opt/anaconda3/envs/spatial/lib/python3.9/site-packages/pycisTopic/diff_features.py:801, in <listcomp>(.0)
    798     ray.shutdown()
    799 else:
    800     markers_list = [
--> 801         markers(
    802             subset_imputed_features_obj,
    803             barcode_groups[i],
    804             contrasts_names[i],
    805             adjpval_thr=adjpval_thr,
    806             log2fc_thr=log2fc_thr,
    807             n_cpu=1,
    808         )
    809         for i in range(len(contrasts))
    810     ]
    812 markers_dict = {
    813     contrasts_name: marker
    814     for contrasts_name, marker in zip(contrasts_names, markers_list)
    815 }
    817 return markers_dict

File ~/opt/anaconda3/envs/spatial/lib/python3.9/site-packages/pycisTopic/diff_features.py:884, in markers(input_mat, barcode_group, contrast_name, adjpval_thr, log2fc_thr, n_cpu)
    882     bg_mat = mat[:, bg_cells_index].toarray()
    883 else:
--> 884     fg_mat = subset_array_second_axis(arr=mat, col_indices=fg_cells_index)
    885     bg_mat = subset_array_second_axis(arr=mat, col_indices=bg_cells_index)
    887 log.info(f"Computing p-value for {contrast_name}")

File ~/opt/anaconda3/envs/spatial/lib/python3.9/site-packages/numba/core/dispatcher.py:480, in _DispatcherBase._compile_for_args(self, *args, **kws)
    476     error_rewrite(e, 'interpreter')
    477 except errors.ConstantInferenceError as e:
    478     # this is from trying to infer something as constant when it isn't
    479     # or isn't supported as a constant
--> 480     error_rewrite(e, 'constant_inference')
    481 except Exception as e:
    482     if config.SHOW_HELP:

File ~/opt/anaconda3/envs/spatial/lib/python3.9/site-packages/numba/core/dispatcher.py:409, in _DispatcherBase._compile_for_args.<locals>.error_rewrite(e, issue_type)
    407     raise e
    408 else:
--> 409     raise e.with_traceback(None)

ConstantInferenceError: Failed in nopython mode pipeline (step: nopython rewrites)
Constant inference not possible for: $102build_string.16 + $100format_value.14

File "../../../../opt/anaconda3/envs/spatial/lib/python3.9/site-packages/pycisTopic/diff_features.py", line 1042:
def subset_array_second_axis(arr, col_indices):
    <source elided>
    if np.min(col_indices) < -arr.shape[1]:
        raise IndexError(f"index {np.min(col_indices)} is out of bounds for axis 1 with size {arr.shape[1]}")
        ^

<Figure size 640x480 with 0 Axes>

Any idea why this might be happening? I tried updating Numba to the latest version but this then breaks the UMAP visualization.

Here is the detail of the environment I am currently using.

# packages in environment at /Users/josephineyates/opt/anaconda3/envs/spatial:
#
# Name                    Version                   Build  Channel
adjusttext                0.8                      pypi_0    pypi
aiohttp                   3.8.4                    pypi_0    pypi
aiohttp-cors              0.7.0                    pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
alabaster                 0.7.13                   pypi_0    pypi
anndata                   0.8.0                    pypi_0    pypi
annoy                     1.17.3                   pypi_0    pypi
ansiwrap                  0.8.4                    pypi_0    pypi
anyio                     3.6.2                    pypi_0    pypi
appnope                   0.1.3                    pypi_0    pypi
arboreto                  0.1.6                    pypi_0    pypi
argon2-cffi               21.3.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow                     1.2.3                    pypi_0    pypi
asciitree                 0.3.3                    pypi_0    pypi
astor                     0.8.1                    pypi_0    pypi
asttokens                 2.2.1                    pypi_0    pypi
async-timeout             4.0.2                    pypi_0    pypi
attrs                     22.2.0                   pypi_0    pypi
autograd                  1.6.2                    pypi_0    pypi
autograd-gamma            0.5.0                    pypi_0    pypi
babel                     2.12.1                   pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
bbknn                     1.6.0                    pypi_0    pypi
beautifulsoup4            4.11.2                   pypi_0    pypi
blas                      1.0                         mkl  
bleach                    6.0.0                    pypi_0    pypi
blessed                   1.20.0                   pypi_0    pypi
blosc2                    2.0.0                    pypi_0    pypi
bokeh                     3.2.0                    pypi_0    pypi
boltons                   23.0.0                   pypi_0    pypi
brotlipy                  0.7.0           py39h9ed2024_1003  
bs4                       0.0.1                    pypi_0    pypi
bzip2                     1.0.8                h1de35cc_0  
ca-certificates           2023.01.10           hecd8cb5_0  
cachetools                5.3.1                    pypi_0    pypi
catheat                   0.2.0                    pypi_0    pypi
cattrs                    23.1.2                   pypi_0    pypi
certifi                   2022.12.7        py39hecd8cb5_0  
cffi                      1.15.1           py39h6c40b1e_3  
chardet                   5.2.0                    pypi_0    pypi
charset-normalizer        3.1.0                    pypi_0    pypi
click                     8.1.3                    pypi_0    pypi
cloudpickle               2.2.1                    pypi_0    pypi
colorful                  0.5.5                    pypi_0    pypi
comm                      0.1.2                    pypi_0    pypi
contourpy                 1.0.7                    pypi_0    pypi
cryptography              39.0.1           py39hf6deb26_0  
ctxcore                   0.2.0                    pypi_0    pypi
cycler                    0.11.0                   pypi_0    pypi
cykhash                   2.0.1                    pypi_0    pypi
cython                    0.29.36                  pypi_0    pypi
cytoolz                   0.12.1                   pypi_0    pypi
dask                      2023.6.1                 pypi_0    pypi
dask-image                2022.9.0                 pypi_0    pypi
dataclasses-json          0.6.0                    pypi_0    pypi
debugpy                   1.6.6                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
decoupler                 1.3.4                    pypi_0    pypi
defusedxml                0.7.1                    pypi_0    pypi
dill                      0.3.6                    pypi_0    pypi
distlib                   0.3.7                    pypi_0    pypi
distributed               2023.6.1                 pypi_0    pypi
docrep                    0.3.2                    pypi_0    pypi
docutils                  0.18.1                   pypi_0    pypi
entrypoints               0.4                      pypi_0    pypi
et-xmlfile                1.1.0                    pypi_0    pypi
exceptiongroup            1.1.3                    pypi_0    pypi
executing                 1.2.0                    pypi_0    pypi
fasteners                 0.18                     pypi_0    pypi
fastjsonschema            2.16.3                   pypi_0    pypi
fbpca                     1.0                      pypi_0    pypi
ffmpeg                    4.3                  h0a44026_0    pytorch
filelock                  3.9.0            py39hecd8cb5_0  
flit-core                 3.6.0              pyhd3eb1b0_0  
fonttools                 4.39.0                   pypi_0    pypi
formulaic                 0.6.4                    pypi_0    pypi
fqdn                      1.5.1                    pypi_0    pypi
freetype                  2.12.1               hd8bbffd_0  
frozendict                2.3.8                    pypi_0    pypi
frozenlist                1.3.3                    pypi_0    pypi
fsspec                    2023.3.0                 pypi_0    pypi
future                    0.18.3                   pypi_0    pypi
gensim                    4.3.2                    pypi_0    pypi
geoparse                  2.0.3                    pypi_0    pypi
geosketch                 1.2                      pypi_0    pypi
gettext                   0.21.0               h7535e17_0  
giflib                    5.2.1                h6c40b1e_3  
globre                    0.1.5                    pypi_0    pypi
gmp                       6.2.1                he9d5cce_3  
gmpy2                     2.1.2            py39hd5de756_0  
gnutls                    3.6.15               hed9c0bf_0  
google-api-core           2.11.1                   pypi_0    pypi
google-auth               2.23.0                   pypi_0    pypi
googleapis-common-protos  1.60.0                   pypi_0    pypi
gpustat                   1.1.1                    pypi_0    pypi
grpcio                    1.58.0                   pypi_0    pypi
gseapy                    1.0.4                    pypi_0    pypi
gtfparse                  2.0.1                    pypi_0    pypi
h5py                      3.8.0                    pypi_0    pypi
harmonypy                 0.0.9                    pypi_0    pypi
hmmlearn                  0.3.0                    pypi_0    pypi
icu                       58.2                 h0a44026_3  
idna                      3.4              py39hecd8cb5_0  
igraph                    0.10.4                   pypi_0    pypi
imagecodecs               2023.7.4                 pypi_0    pypi
imageio                   2.26.0                   pypi_0    pypi
imagesize                 1.4.1                    pypi_0    pypi
importlib-metadata        6.0.0                    pypi_0    pypi
importlib-resources       5.12.0                   pypi_0    pypi
infercnvpy                0.4.1                    pypi_0    pypi
inflect                   6.0.2                    pypi_0    pypi
intel-openmp              2021.4.0          hecd8cb5_3538  
interface-meta            1.3.0                    pypi_0    pypi
interlap                  0.2.7                    pypi_0    pypi
intervaltree              3.1.0                    pypi_0    pypi
ipykernel                 6.21.3                   pypi_0    pypi
ipympl                    0.9.3                    pypi_0    pypi
ipython                   8.11.0                   pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                8.0.4                    pypi_0    pypi
isoduration               20.11.0                  pypi_0    pypi
jedi                      0.18.2                   pypi_0    pypi
jinja2                    3.1.2            py39hecd8cb5_0  
joblib                    1.2.0                    pypi_0    pypi
jpeg                      9e                   h6c40b1e_1  
jsonpointer               2.3                      pypi_0    pypi
jsonschema                4.17.3                   pypi_0    pypi
jupyter                   1.0.0                    pypi_0    pypi
jupyter-client            8.0.3                    pypi_0    pypi
jupyter-console           6.6.3                    pypi_0    pypi
jupyter-core              5.2.0                    pypi_0    pypi
jupyter-events            0.6.3                    pypi_0    pypi
jupyter-server            2.4.0                    pypi_0    pypi
jupyter-server-terminals  0.4.4                    pypi_0    pypi
jupyterlab-pygments       0.2.2                    pypi_0    pypi
jupyterlab-widgets        3.0.5                    pypi_0    pypi
kiwisolver                1.4.4                    pypi_0    pypi
lame                      3.100                h1de35cc_0  
lazy-loader               0.1                      pypi_0    pypi
lcms2                     2.12                 hf1fd2bf_0  
lda                       2.0.0                    pypi_0    pypi
leidenalg                 0.9.1                    pypi_0    pypi
lerc                      3.0                  he9d5cce_0  
levenshtein               0.21.1                   pypi_0    pypi
libcxx                    14.0.6               h9765a3e_0  
libdeflate                1.17                 hb664fd8_0  
libffi                    3.4.2                hecd8cb5_6  
libiconv                  1.16                 hca72f7f_2  
libidn2                   2.3.2                h9ed2024_0  
libpng                    1.6.39               h6c40b1e_0  
libtasn1                  4.16.0               h9ed2024_0  
libtiff                   4.5.0                hcec6c5f_2  
libunistring              0.9.10               h9ed2024_0  
libwebp                   1.2.4                hf6ce154_1  
libwebp-base              1.2.4                h6c40b1e_1  
libxml2                   2.9.14               hbf8cd5e_0  
lifelines                 0.27.7                   pypi_0    pypi
llvm-openmp               14.0.6               h0dcd299_0  
llvmlite                  0.38.1                   pypi_0    pypi
locket                    1.0.0                    pypi_0    pypi
loompy                    3.0.7                    pypi_0    pypi
loomxpy                   0.4.2                    pypi_0    pypi
louvain                   0.8.0                    pypi_0    pypi
lxml                      4.9.2                    pypi_0    pypi
lz4                       4.3.2                    pypi_0    pypi
lz4-c                     1.9.4                hcec6c5f_0  
maca-python               1.0.1                    pypi_0    pypi
macs2                     2.2.9.1                  pypi_0    pypi
macs3                     3.0.0b3                  pypi_0    pypi
markupsafe                2.1.2                    pypi_0    pypi
marshmallow               3.20.1                   pypi_0    pypi
matplotlib                3.7.1                    pypi_0    pypi
matplotlib-inline         0.1.6                    pypi_0    pypi
matplotlib-scalebar       0.8.1                    pypi_0    pypi
mistune                   2.0.5                    pypi_0    pypi
mkl                       2021.4.0           hecd8cb5_637  
mkl-service               2.4.0            py39h9ed2024_0  
mkl_fft                   1.3.1            py39h4ab4a9b_0  
mkl_random                1.2.2            py39hb2f4e1b_0  
mpc                       1.1.0                h6ef4df4_1  
mpfr                      4.0.2                h9066e36_1  
mpmath                    1.2.1            py39hecd8cb5_0  
msgpack                   1.0.5                    pypi_0    pypi
multidict                 6.0.4                    pypi_0    pypi
multiprocessing-on-dill   3.5.0a4                  pypi_0    pypi
mypy-extensions           1.0.0                    pypi_0    pypi
natsort                   8.3.1                    pypi_0    pypi
nbclassic                 0.5.3                    pypi_0    pypi
nbclient                  0.7.2                    pypi_0    pypi
nbconvert                 7.2.10                   pypi_0    pypi
nbformat                  5.7.3                    pypi_0    pypi
nbsphinx                  0.9.3                    pypi_0    pypi
nbsphinx-link             1.3.0                    pypi_0    pypi
ncls                      0.0.68                   pypi_0    pypi
ncurses                   6.4                  hcec6c5f_0  
nest-asyncio              1.5.6                    pypi_0    pypi
nettle                    3.7.3                h230ac6f_1  
networkx                  3.0                      pypi_0    pypi
notebook                  6.5.3                    pypi_0    pypi
notebook-shim             0.2.2                    pypi_0    pypi
numba                     0.55.2                   pypi_0    pypi
numcodecs                 0.11.0                   pypi_0    pypi
numexpr                   2.8.4                    pypi_0    pypi
numpy                     1.22.4                   pypi_0    pypi
numpy-groupies            0.9.22                   pypi_0    pypi
numpydoc                  1.5.0                    pypi_0    pypi
nvidia-ml-py              12.535.108               pypi_0    pypi
omnipath                  1.0.6                    pypi_0    pypi
opencensus                0.11.3                   pypi_0    pypi
opencensus-context        0.1.3                    pypi_0    pypi
opencv-python             4.7.0.72                 pypi_0    pypi
openh264                  2.1.1                h8346a28_0  
openpyxl                  3.1.2                    pypi_0    pypi
openssl                   1.1.1t               hca72f7f_0  
packaging                 23.0                     pypi_0    pypi
pandas                    1.5.0                    pypi_0    pypi
pandocfilters             1.5.0                    pypi_0    pypi
papermill                 2.4.0                    pypi_0    pypi
parso                     0.8.3                    pypi_0    pypi
partd                     1.3.0                    pypi_0    pypi
patsy                     0.5.3                    pypi_0    pypi
pbr                       3.1.1                    pypi_0    pypi
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    9.4.0                    pypi_0    pypi
pims                      0.6.1                    pypi_0    pypi
pip                       23.2.1                   pypi_0    pypi
platformdirs              3.1.1                    pypi_0    pypi
polars                    0.19.3                   pypi_0    pypi
prometheus-client         0.16.0                   pypi_0    pypi
prompt-toolkit            3.0.38                   pypi_0    pypi
protobuf                  4.24.3                   pypi_0    pypi
psutil                    5.9.4                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.2                    pypi_0    pypi
py-cpuinfo                9.0.0                    pypi_0    pypi
py-spy                    0.3.14                   pypi_0    pypi
pyarrow                   12.0.1                   pypi_0    pypi
pyasn1                    0.5.0                    pypi_0    pypi
pyasn1-modules            0.3.0                    pypi_0    pypi
pybedtools                0.9.1                    pypi_0    pypi
pybigwig                  0.3.22                   pypi_0    pypi
pybiomart                 0.2.0                    pypi_0    pypi
pycistopic                1.0.3.dev18+ge563fb6          pypi_0    pypi
pycparser                 2.21               pyhd3eb1b0_0  
pydantic                  1.10.6                   pypi_0    pypi
pyfasta                   0.5.2                    pypi_0    pypi
pygments                  2.14.0                   pypi_0    pypi
pynndescent               0.5.8                    pypi_0    pypi
pyometiff                 0.0.13                   pypi_0    pypi
pyopenssl                 23.0.0           py39hecd8cb5_0  
pyparsing                 3.0.9                    pypi_0    pypi
pyranges                  0.0.127                  pypi_0    pypi
pyreadr                   0.4.7                    pypi_0    pypi
pyrsistent                0.19.3                   pypi_0    pypi
pysam                     0.21.0                   pypi_0    pypi
pyscenic                  0.12.1+6.g31d51a1          pypi_0    pypi
pysocks                   1.7.1            py39hecd8cb5_0  
python                    3.9.16               h218abb5_2  
python-dateutil           2.8.2                    pypi_0    pypi
python-igraph             0.10.4                   pypi_0    pypi
python-json-logger        2.0.7                    pypi_0    pypi
python-levenshtein        0.21.1                   pypi_0    pypi
pytoml                    0.1.21                   pypi_0    pypi
pytz                      2022.7.1                 pypi_0    pypi
pywavelets                1.4.1                    pypi_0    pypi
pyyaml                    6.0                      pypi_0    pypi
pyzmq                     25.0.1                   pypi_0    pypi
qtconsole                 5.4.1                    pypi_0    pypi
qtpy                      2.3.0                    pypi_0    pypi
rapidfuzz                 3.3.0                    pypi_0    pypi
ray                       2.7.0                    pypi_0    pypi
readline                  8.2                  hca72f7f_0  
requests                  2.28.2                   pypi_0    pypi
requests-cache            1.1.0                    pypi_0    pypi
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rsa                       4.9                      pypi_0    pypi
scanorama                 1.7.3                    pypi_0    pypi
scanpy                    1.9.3                    pypi_0    pypi
scikit-image              0.20.0                   pypi_0    pypi
scikit-learn              1.2.2                    pypi_0    pypi
scipy                     1.9.1                    pypi_0    pypi
scrublet                  0.2.3                    pypi_0    pypi
seaborn                   0.11.2                   pypi_0    pypi
send2trash                1.8.0                    pypi_0    pypi
session-info              1.0.0                    pypi_0    pypi
setuptools                68.0.0                   pypi_0    pypi
six                       1.16.0             pyhd3eb1b0_1  
sklearn                   0.0.post1                pypi_0    pypi
slicerator                1.1.0                    pypi_0    pypi
smart-open                6.4.0                    pypi_0    pypi
sniffio                   1.3.0                    pypi_0    pypi
snowballstemmer           2.2.0                    pypi_0    pypi
sorted-nearest            0.0.39                   pypi_0    pypi
sortedcontainers          2.4.0                    pypi_0    pypi
soupsieve                 2.4                      pypi_0    pypi
spagcn                    1.2.5                    pypi_0    pypi
sphinx                    7.2.6                    pypi_0    pypi
sphinx-rtd-theme          1.3.0                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.7                    pypi_0    pypi
sphinxcontrib-devhelp     1.0.5                    pypi_0    pypi
sphinxcontrib-htmlhelp    2.0.4                    pypi_0    pypi
sphinxcontrib-jquery      4.1                      pypi_0    pypi
sphinxcontrib-jsmath      1.0.1                    pypi_0    pypi
sphinxcontrib-qthelp      1.0.6                    pypi_0    pypi
sphinxcontrib-serializinghtml 1.1.9                    pypi_0    pypi
sqlite                    3.41.1               h6c40b1e_0  
squidpy                   1.2.3                    pypi_0    pypi
stack-data                0.6.2                    pypi_0    pypi
statannot                 0.2.3                    pypi_0    pypi
statannotations           0.5.0                    pypi_0    pypi
statsmodels               0.13.5                   pypi_0    pypi
stdlib-list               0.8.0                    pypi_0    pypi
sympy                     1.11.1           py39hecd8cb5_0  
tables                    3.8.0                    pypi_0    pypi
tabulate                  0.9.0                    pypi_0    pypi
tblib                     2.0.0                    pypi_0    pypi
tenacity                  8.2.3                    pypi_0    pypi
terminado                 0.17.1                   pypi_0    pypi
texttable                 1.6.7                    pypi_0    pypi
textwrap3                 0.9.2                    pypi_0    pypi
threadpoolctl             3.1.0                    pypi_0    pypi
tifffile                  2023.2.28                pypi_0    pypi
tinycss2                  1.2.1                    pypi_0    pypi
tk                        8.6.12               h5d9f67b_0  
tmtoolkit                 0.11.2                   pypi_0    pypi
toolz                     0.12.0                   pypi_0    pypi
torch                     1.10.2                   pypi_0    pypi
torchaudio                2.0.0                  py39_cpu    pytorch
torchvision               0.11.3                   pypi_0    pypi
tornado                   6.2                      pypi_0    pypi
tqdm                      4.65.0                   pypi_0    pypi
traitlets                 5.9.0                    pypi_0    pypi
tspex                     0.6.3                    pypi_0    pypi
typing                    3.7.4.3                  pypi_0    pypi
typing-extensions         4.5.0                    pypi_0    pypi
typing-inspect            0.9.0                    pypi_0    pypi
typing_extensions         4.4.0            py39hecd8cb5_0  
tzdata                    2022g                h04d1e81_0  
umap-learn                0.5.3                    pypi_0    pypi
uri-template              1.2.0                    pypi_0    pypi
url-normalize             1.4.3                    pypi_0    pypi
urllib3                   1.26.15                  pypi_0    pypi
validators                0.20.0                   pypi_0    pypi
virtualenv                20.21.0                  pypi_0    pypi
wcwidth                   0.2.6                    pypi_0    pypi
webcolors                 1.12                     pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.5.1                    pypi_0    pypi
wget                      3.2                      pypi_0    pypi
wheel                     0.38.4           py39hecd8cb5_0  
widgetsnbextension        4.0.5                    pypi_0    pypi
wrapt                     1.15.0                   pypi_0    pypi
xarray                    2023.2.0                 pypi_0    pypi
xlrd                      2.0.1                    pypi_0    pypi
xyzservices               2023.5.0                 pypi_0    pypi
xz                        5.2.10               h6c40b1e_1  
yarl                      1.8.2                    pypi_0    pypi
zarr                      2.14.2                   pypi_0    pypi
zict                      3.0.0                    pypi_0    pypi
zipp                      3.15.0                   pypi_0    pypi
zlib                      1.2.13               h4dc903c_0  
zstd                      1.5.2                hcb37349_0  

Thank you for your help!

josephineyates commented 11 months ago

Hi again, Just to give some more information, I can run find_diff_features without any errors by directly copying the source code and removing the numba decorators.

ghuls commented 11 months ago

Can you install numba >= 0.57.0?

josephineyates commented 11 months ago

Hi Ghuls, Thank you for answering! So installing numba >=0.57.0 unfortunately breaks the UMAP implementation. I was running this on a Mac M1 and maybe this is related - I know there have been issues previously running UMAP on M1 chips. In any case, I've switched to running on the cluster where a fresh install of everything with latest versions worked. Leaving this here in case someone else runs into this issue. Thank you again for your time!

ghuls commented 11 months ago

@josephineyates Does updating pynndescent help? https://github.com/aertslab/pycisTopic/issues/92#issuecomment-1735438244

SteveTur commented 9 months ago

Hello @ghuls @SeppeDeWinter

I got the same following error by running:

from pycisTopic.diff_features import * imputed_acc_obj = impute_accessibility(cistopic_obj, selected_cells=None, selected_regions=None, scale_factor=106) normalized_imputed_acc_obj = normalize_scores(imputed_acc_obj, scale_factor=104) variable_regions = find_highly_variable_features(normalized_imputed_acc_obj, plot = False) markers_dict = find_diff_features(cistopic_obj, imputed_acc_obj, variable='celltype', var_features=variable_regions, split_pattern = '-')

Here is the error:

ConstantInferenceError Traceback (most recent call last) /Users/stur/Teaseq/Scenicplus/PB2/Scenicplus_PB2.ipynb Cell 67 line 1 ----> 1 markers_dict = find_diff_features(cistopic_obj, imputed_acc_obj, variable='celltype', var_features=variable_regions, split_pattern = '-')

File /opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/pycisTopic/diff_features.py:800, in find_diff_features(cistopic_obj, imputed_features_obj, variable, var_features, contrasts, adjpval_thr, log2fc_thr, split_pattern, n_cpu, **kwargs) 798 ray.shutdown() 799 else: --> 800 markers_list = [ 801 markers( 802 subset_imputed_features_obj, 803 barcode_groups[i], 804 contrasts_names[i], 805 adjpval_thr=adjpval_thr, 806 log2fc_thr=log2fc_thr, 807 n_cpu=1, 808 ) 809 for i in range(len(contrasts)) 810 ] 812 markers_dict = { 813 contrasts_name: marker 814 for contrasts_name, marker in zip(contrasts_names, markers_list) 815 } 817 return markers_dict

File /opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/pycisTopic/diff_features.py:801, in (.0) 798 ray.shutdown() 799 else: 800 markers_list = [ --> 801 markers( 802 subset_imputed_features_obj, 803 barcode_groups[i], 804 contrasts_names[i], 805 adjpval_thr=adjpval_thr, 806 log2fc_thr=log2fc_thr, 807 n_cpu=1, 808 ) 809 for i in range(len(contrasts)) 810 ] 812 markers_dict = { 813 contrasts_name: marker 814 for contrasts_name, marker in zip(contrasts_names, markers_list) 815 } 817 return markers_dict

File /opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/pycisTopic/diff_features.py:884, in markers(input_mat, barcode_group, contrast_name, adjpval_thr, log2fc_thr, n_cpu) 882 bg_mat = mat[:, bg_cells_index].toarray() 883 else: --> 884 fg_mat = subset_array_second_axis(arr=mat, col_indices=fg_cells_index) 885 bg_mat = subset_array_second_axis(arr=mat, col_indices=bg_cells_index) 887 log.info(f"Computing p-value for {contrast_name}")

File /opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/numba/core/dispatcher.py:480, in _DispatcherBase._compile_for_args(self, *args, **kws) 476 error_rewrite(e, 'interpreter') 477 except errors.ConstantInferenceError as e: 478 # this is from trying to infer something as constant when it isn't 479 # or isn't supported as a constant --> 480 error_rewrite(e, 'constant_inference') 481 except Exception as e: 482 if config.SHOW_HELP:

File /opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/numba/core/dispatcher.py:409, in _DispatcherBase._compile_for_args..error_rewrite(e, issue_type) 407 raise e 408 else: --> 409 raise e.with_traceback(None)

ConstantInferenceError: Failed in nopython mode pipeline (step: nopython rewrites) Constant inference not possible for: $102build_string.16 + $100format_value.14

File "../../../../opt/homebrew/Caskroom/miniconda/base/lib/python3.10/site-packages/pycisTopic/diff_features.py", line 1042: def subset_array_second_axis(arr, col_indices):

if np.min(col_indices) < -arr.shape[1]:
    raise IndexError(f"index {np.min(col_indices)} is out of bounds for axis 1 with size {arr.shape[1]}")
    ^

Can you help me to troubleshoot this?

Best,

Steven

ghuls commented 5 months ago

You need numba >= 0.57.