Unable to run Scrublet in v1.10

jasonleongbio commented 5 months ago

Please make sure these conditions are met

[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of scanpy.
[ ] (optional) I have confirmed this bug exists on the main branch of scanpy.

What happened?

Dear scanpy developers,

I was exploring the new features in the latest version of Scanpy, but encountered a prolonged pause when running the sc.pp.scrublet(adata).

Initially I thought the problem was due to the large size (~100k cells) of the dataset I was exploring (I let it run for almost a whole week and nothing changed). However, even if I switched to my own dataset (unpublished, around 5k celIs), it paused at the same step.

Running Scrublet
filtered out 1419 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:00)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
normalizing counts per cell
    finished (0:00:00)
normalizing counts per cell
    finished (0:00:00)
Embedding transcriptomes using PCA...

I was running this analysis on my Intel-core iMac. Surprisingly, when I ran the same line of code (under a similar virtual environment) on my M2-chip laptop, it finished in a flash of time.

filtered out 1419 genes that are detected in less than 3 cells
normalizing counts per cell
    finished (0:00:00)
extracting highly variable genes
    finished (0:00:00)
--> added
    'highly_variable', boolean vector (adata.var)
    'means', float vector (adata.var)
    'dispersions', float vector (adata.var)
    'dispersions_norm', float vector (adata.var)
normalizing counts per cell
    finished (0:00:00)
normalizing counts per cell
    finished (0:00:00)
Embedding transcriptomes using PCA...
    using data matrix X directly
Automatically set threshold at doublet score = 0.42
Detected doublet rate = 0.3%
Estimated detectable doublet fraction = 5.2%
Overall doublet rate:
    Expected   = 5.0%
    Estimated  = 6.6%
    Scrublet finished (0:00:14)

I'm still not sure what actually caused the problem, but it seems that some dependency inconsistency occurred when performing PCA within the pipeline. Perhaps some package required for the sc.pp.scrublet() pipeline needs to be updated to a newer version?

Here are the details of the packages in the virtual environment when I ran the code on my desktop (failed case):

channels:
  - pytorch
  - plotly
  - conda-forge
  - bioconda
  - defaults
dependencies:
  - anndata=0.10.7
  - anyio=4.4.0
  - appnope=0.1.4
  - argcomplete=3.3.0
  - argh=0.31.2
  - argon2-cffi=23.1.0
  - argon2-cffi-bindings=21.2.0
  - arpack=3.8.0
  - array-api-compat=1.7.1
  - arrow=1.3.0
  - asttokens=2.4.1
  - async-lru=2.0.4
  - attrs=23.2.0
  - babel=2.14.0
  - beautifulsoup4=4.12.3
  - biopython=1.83
  - blas=2.120
  - blas-devel=3.9.0
  - bleach=6.1.0
  - blosc=1.21.5
  - brotli=1.1.0
  - brotli-bin=1.1.0
  - brotli-python=1.1.0
  - bzip2=1.0.8
  - c-ares=1.28.1
  - c-blosc2=2.14.4
  - ca-certificates=2024.6.2
  - cached-property=1.5.2
  - cached_property=1.5.2
  - certifi=2024.6.2
  - cffi=1.16.0
  - charset-normalizer=3.3.2
  - colorama=0.4.6
  - colorcet=3.1.0
  - colorful=0.5.6
  - comm=0.2.2
  - contourpy=1.2.1
  - cycler=0.12.1
  - debugpy=1.8.1
  - decorator=5.1.1
  - defusedxml=0.7.1
  - dill=0.3.8
  - dnspython=2.6.1
  - entrypoints=0.4
  - et_xmlfile=1.1.0
  - exceptiongroup=1.2.0
  - executing=2.0.1
  - filelock=3.14.0
  - fonttools=4.53.0
  - fqdn=1.5.1
  - freetype=2.12.1
  - get-annotations=0.1.2
  - gffpandas=1.2.2
  - gffutils=0.13
  - glpk=5.0
  - gmp=6.3.0
  - gmpy2=2.1.5
  - h11=0.14.0
  - h2=4.1.0
  - h5py=3.11.0
  - hdf5=1.14.3
  - hpack=4.0.0
  - httpcore=1.0.5
  - httpx=0.27.0
  - hyperframe=6.0.1
  - icu=73.2
  - idna=3.7
  - igraph=0.10.12
  - importlib-metadata=7.1.0
  - importlib_metadata=7.1.0
  - importlib_resources=6.4.0
  - ipykernel=6.29.4
  - ipython=8.25.0
  - isoduration=20.11.0
  - jedi=0.19.1
  - jinja2=3.1.4
  - joblib=1.4.2
  - json5=0.9.25
  - jsonpointer=2.4
  - jsonschema=4.22.0
  - jsonschema-specifications=2023.12.1
  - jsonschema-with-format-nongpl=4.22.0
  - jupyter-lsp=2.2.5
  - jupyter_client=8.6.2
  - jupyter_core=5.7.2
  - jupyter_events=0.10.0
  - jupyter_server=2.14.1
  - jupyter_server_terminals=0.5.3
  - jupyterlab=4.2.2
  - jupyterlab_pygments=0.3.0
  - jupyterlab_server=2.27.2
  - kaleido-core=0.2.1
  - kiwisolver=1.4.5
  - krb5=1.21.2
  - lcms2=2.16
  - legacy-api-wrap=1.4
  - leidenalg=0.10.2
  - lerc=4.0.0
  - libabseil=20240116.2
  - libaec=1.1.3
  - libblas=3.9.0
  - libbrotlicommon=1.1.0
  - libbrotlidec=1.1.0
  - libbrotlienc=1.1.0
  - libcblas=3.9.0
  - libcurl=8.8.0
  - libcxx=17.0.6
  - libdeflate=1.20
  - libedit=3.1.20191231
  - libev=4.33
  - libexpat=2.6.2
  - libffi=3.4.2
  - libgfortran=5.0.0
  - libgfortran5=13.2.0
  - libhwloc=2.10.0
  - libiconv=1.17
  - libjpeg-turbo=3.0.0
  - liblapack=3.9.0
  - liblapacke=3.9.0
  - libleidenalg=0.11.1
  - libllvm14=14.0.6
  - libnghttp2=1.58.0
  - libopenblas=0.3.27
  - libpng=1.6.43
  - libprotobuf=4.25.3
  - libsodium=1.0.18
  - libsqlite=3.46.0
  - libssh2=1.11.0
  - libtiff=4.6.0
  - libwebp-base=1.4.0
  - libxcb=1.15
  - libxml2=2.12.7
  - libzlib=1.3.1
  - llvm-openmp=18.1.7
  - llvmlite=0.42.0
  - louvain=0.8.2
  - lz4-c=1.9.4
  - markupsafe=2.1.5
  - mathjax=2.7.7
  - matplotlib=3.8.4
  - matplotlib-base=3.8.4
  - matplotlib-inline=0.1.7
  - mistune=3.0.2
  - mkl=2023.2.0
  - mkl-devel=2023.2.0
  - mkl-include=2023.2.0
  - mpc=1.3.1
  - mpfr=4.2.1
  - mpmath=1.3.0
  - mudata=0.2.3
  - multiprocess=0.70.16
  - munkres=1.1.4
  - muon=0.1.6
  - natsort=8.4.0
  - nbclient=0.10.0
  - nbconvert-core=7.16.4
  - nbformat=5.10.4
  - ncurses=6.5
  - nest-asyncio=1.6.0
  - networkx=3.3
  - notebook=7.2.1
  - notebook-shim=0.2.4
  - numba=0.59.1
  - numexpr=2.10.0
  - numpy=1.26.4
  - openjpeg=2.5.2
  - openpyxl=3.1.2
  - openssl=3.3.1
  - overrides=7.7.0
  - packaging=24.0
  - pandas=2.2.2
  - pandocfilters=1.5.0
  - parso=0.8.4
  - patsy=0.5.6
  - pexpect=4.9.0
  - pickleshare=0.7.5
  - pillow=10.3.0
  - pip=24.0
  - pkgutil-resolve-name=1.3.10
  - platformdirs=4.2.2
  - plotly=5.22.0
  - plotly-orca=1.3.1
  - pooch=1.8.2
  - prettyprinter=0.18.0
  - prometheus_client=0.20.0
  - prompt-toolkit=3.0.47
  - prompt_toolkit=3.0.47
  - protobuf=4.25.3
  - psutil=5.9.8
  - pthread-stubs=0.4
  - ptyprocess=0.7.0
  - pure_eval=0.2.2
  - py-cpuinfo=9.0.0
  - pycparser=2.22
  - pyfaidx=0.8.1.1
  - pygments=2.18.0
  - pymde=0.1.18
  - pymongo=4.7.3
  - pynndescent=0.5.12
  - pyobjc-core=10.2
  - pyobjc-framework-cocoa=10.2
  - pyparsing=3.1.2
  - pysocks=1.7.1
  - pytables=3.9.2
  - python=3.11.4
  - python-dateutil=2.9.0
  - python-fastjsonschema=2.19.1
  - python-igraph=0.11.5
  - python-json-logger=2.0.7
  - python-kaleido=0.2.1
  - python-tzdata=2024.1
  - python_abi=3.11
  - pytorch=2.2.2
  - pytz=2024.1
  - pyvcf3=1.0.3
  - pyyaml=6.0.1
  - pyzmq=26.0.3
  - radian=0.6.12
  - rchitect=0.4.6
  - readline=8.2
  - referencing=0.35.1
  - requests=2.32.3
  - rfc3339-validator=0.1.4
  - rfc3986-validator=0.1.1
  - rpds-py=0.18.1
  - scanpy=1.10.1
  - scikit-learn=1.5.0
  - scipy=1.13.1
  - seaborn=0.13.2
  - seaborn-base=0.13.2
  - send2trash=1.8.3
  - session-info=1.0.0
  - setuptools=70.0.0
  - simplejson=3.19.2
  - six=1.16.0
  - snappy=1.2.0
  - sniffio=1.3.1
  - soupsieve=2.5
  - stack_data=0.6.2
  - statsmodels=0.14.2
  - stdlib-list=0.10.0
  - sympy=1.12
  - tbb=2021.12.0
  - tenacity=8.3.0
  - terminado=0.18.1
  - texttable=1.7.0
  - threadpoolctl=3.5.0
  - tinycss2=1.3.0
  - tk=8.6.13
  - tomli=2.0.1
  - torchvision=0.17.2
  - tornado=6.4.1
  - tqdm=4.66.4
  - traitlets=5.14.3
  - types-python-dateutil=2.9.0.20240316
  - typing-extensions=4.12.2
  - typing_extensions=4.12.2
  - typing_utils=0.1.0
  - tzdata=2024a
  - umap-learn=0.5.5
  - uri-template=1.3.0
  - urllib3=2.2.1
  - wcwidth=0.2.13
  - webcolors=24.6.0
  - webencodings=0.5.1
  - websocket-client=1.8.0
  - wheel=0.43.0
  - xlrd=1.2.0
  - xorg-libxau=1.0.11
  - xorg-libxdmcp=1.1.3
  - xz=5.2.6
  - yaml=0.2.5
  - zeromq=4.3.5
  - zipp=3.19.2
  - zlib-ng=2.0.7
  - zstd=1.5.6
  - pip:
      - absl-py==2.1.0
      - astunparse==1.6.3
      - bcbio-gff==0.7.1
      - flatbuffers==24.3.25
      - gast==0.5.4
      - google-pasta==0.2.0
      - grpcio==1.64.1
      - keras==3.3.3
      - libclang==18.1.1
      - markdown==3.6
      - markdown-it-py==3.0.0
      - mdurl==0.1.2
      - ml-dtypes==0.3.2
      - namex==0.0.8
      - opt-einsum==3.3.0
      - optree==0.11.0
      - rich==13.7.1
      - tensorboard==2.16.2
      - tensorboard-data-server==0.7.2
      - tensorflow==2.16.1
      - tensorflow-io-gcs-filesystem==0.37.0
      - termcolor==2.4.0
      - werkzeug==3.0.3
      - wrapt==1.16.0

The virtual environment on my laptop (successful case):

channels:
  - pytorch
  - bioconda
  - conda-forge
dependencies:
  - adjusttext=1.0.4
  - anndata=0.10.5.post1
  - anyio=3.7.1
  - aom=3.5.0
  - appnope=0.1.3
  - argcomplete=3.3.0
  - argh=0.31.2
  - argon2-cffi=23.1.0
  - argon2-cffi-bindings=21.2.0
  - arpack=3.8.0
  - array-api-compat=1.4.1
  - arrow=1.2.3
  - asttokens=2.2.1
  - async-lru=2.0.4
  - attrs=23.1.0
  - babel=2.12.1
  - backcall=0.2.0
  - backports=1.0
  - backports.functools_lru_cache=1.6.5
  - beautifulsoup4=4.12.2
  - bleach=6.0.0
  - blosc=1.21.4
  - brotli=1.0.9
  - brotli-bin=1.0.9
  - brotli-python=1.0.9
  - bzip2=1.0.8
  - c-ares=1.19.1
  - c-blosc2=2.10.2
  - ca-certificates=2024.6.2
  - cached-property=1.5.2
  - cached_property=1.5.2
  - cairo=1.18.0
  - certifi=2024.6.2
  - cffi=1.15.1
  - charset-normalizer=3.2.0
  - colorama=0.4.6
  - colorcet=3.0.1
  - colorful=0.5.4
  - comm=0.1.4
  - contourpy=1.1.0
  - cryptography=41.0.4
  - cycler=0.11.0
  - dav1d=1.2.1
  - debugpy=1.6.8
  - decorator=5.1.1
  - defusedxml=0.7.1
  - dill=0.3.7
  - dnspython=2.4.2
  - entrypoints=0.4
  - et_xmlfile=1.1.0
  - exceptiongroup=1.1.3
  - executing=1.2.0
  - expat=2.5.0
  - ffmpeg=6.0.0
  - filelock=3.12.2
  - font-ttf-dejavu-sans-mono=2.37
  - font-ttf-inconsolata=3.000
  - font-ttf-source-code-pro=2.038
  - font-ttf-ubuntu=0.83
  - fontconfig=2.14.2
  - fonts-conda-ecosystem=1
  - fonts-conda-forge=1
  - fonttools=4.42.1
  - fqdn=1.5.1
  - freetype=2.12.1
  - fribidi=1.0.10
  - get-annotations=0.1.2
  - gettext=0.21.1
  - gffutils=0.13
  - glpk=5.0
  - gmp=6.3.0
  - gmpy2=2.1.2
  - gnutls=3.7.8
  - graphite2=1.3.13
  - h11=0.14.0
  - h2=4.1.0
  - h5py=3.9.0
  - harfbuzz=7.3.0
  - hdf5=1.14.1
  - hpack=4.0.0
  - httpcore=0.18.0
  - hyperframe=6.0.1
  - icu=73.2
  - idna=3.4
  - igraph=0.10.8
  - importlib-metadata=6.8.0
  - importlib_metadata=6.8.0
  - importlib_resources=6.0.1
  - ipykernel=6.25.1
  - ipython=8.14.0
  - isoduration=20.11.0
  - jedi=0.19.0
  - jinja2=3.1.2
  - joblib=1.3.2
  - jpeg=9e
  - json5=0.9.14
  - jsonpointer=2.0
  - jsonschema=4.19.0
  - jsonschema-specifications=2023.7.1
  - jsonschema-with-format-nongpl=4.19.0
  - jupyter-lsp=2.2.0
  - jupyter_client=8.3.0
  - jupyter_core=5.3.1
  - jupyter_events=0.7.0
  - jupyter_server=2.7.1
  - jupyter_server_terminals=0.4.4
  - jupyterlab=4.0.5
  - jupyterlab_pygments=0.2.2
  - jupyterlab_server=2.24.0
  - kaleido-core=0.2.1
  - kiwisolver=1.4.4
  - krb5=1.21.2
  - lame=3.100
  - lcms2=2.15
  - legacy-api-wrap=1.4
  - leidenalg=0.10.2
  - lerc=4.0.0
  - libabseil=20240116.2
  - libaec=1.0.6
  - libass=0.17.1
  - libblas=3.9.0
  - libbrotlicommon=1.0.9
  - libbrotlidec=1.0.9
  - libbrotlienc=1.0.9
  - libcblas=3.9.0
  - libcurl=8.2.1
  - libcxx=16.0.6
  - libdeflate=1.17
  - libedit=3.1.20191231
  - libev=4.33
  - libexpat=2.5.0
  - libffi=3.4.2
  - libgfortran=5.0.0
  - libgfortran5=12.3.0
  - libglib=2.80.0
  - libhwloc=2.9.3
  - libiconv=1.17
  - libidn2=2.3.4
  - libintl=0.22.5
  - libjpeg-turbo=2.1.4
  - liblapack=3.9.0
  - libleidenalg=0.11.1
  - libllvm14=14.0.6
  - libnghttp2=1.52.0
  - libopenblas=0.3.23
  - libopus=1.3.1
  - libpng=1.6.39
  - libprotobuf=4.25.3
  - libsodium=1.0.18
  - libsqlite=3.42.0
  - libssh2=1.11.0
  - libtasn1=4.19.0
  - libtiff=4.5.0
  - libunistring=0.9.10
  - libuv=1.48.0
  - libvpx=1.13.0
  - libwebp-base=1.3.1
  - libxcb=1.13
  - libxml2=2.11.6
  - libzlib=1.2.13
  - llvm-openmp=16.0.6
  - llvmlite=0.40.1
  - lz4-c=1.9.4
  - markupsafe=2.1.3
  - mathjax=2.7.7
  - matplotlib=3.7.2
  - matplotlib-base=3.7.2
  - matplotlib-inline=0.1.6
  - mistune=3.0.1
  - mpc=1.3.1
  - mpfr=4.2.0
  - mpmath=1.3.0
  - mudata=0.2.3
  - multiprocess=0.70.15
  - munkres=1.1.4
  - muon=0.1.6
  - natsort=8.4.0
  - nbclient=0.8.0
  - nbconvert-core=7.7.4
  - nbformat=5.9.2
  - ncurses=6.4
  - nest-asyncio=1.5.6
  - nettle=3.8.1
  - networkx=3.1
  - nodejs=20.9.0
  - notebook=7.0.2
  - notebook-shim=0.2.3
  - numba=0.57.1
  - numexpr=2.8.4
  - openh264=2.3.1
  - openjpeg=2.5.0
  - openpyxl=3.1.2
  - openssl=3.3.1
  - overrides=7.4.0
  - p11-kit=0.24.1
  - packaging=23.1
  - pandas=2.0.3
  - pandocfilters=1.5.0
  - param=2.0.2
  - parso=0.8.3
  - patsy=0.5.3
  - pcre2=10.43
  - pexpect=4.8.0
  - pickleshare=0.7.5
  - pillow=9.4.0
  - pip=23.2.1
  - pixman=0.43.4
  - pkgutil-resolve-name=1.3.10
  - platformdirs=3.10.0
  - plotly=5.16.1
  - plotly-orca=3.4.2
  - pooch=1.7.0
  - prettyprinter=0.18.0
  - prometheus_client=0.17.1
  - prompt-toolkit=3.0.39
  - prompt_toolkit=3.0.39
  - psutil=5.9.5
  - pthread-stubs=0.4
  - ptyprocess=0.7.0
  - pure_eval=0.2.2
  - py-cpuinfo=9.0.0
  - pycparser=2.21
  - pyct=0.5.0
  - pyfaidx=0.8.1.1
  - pygments=2.16.1
  - pymde=0.1.18
  - pymongo=4.5.0
  - pynndescent=0.5.11
  - pyobjc-core=9.2
  - pyobjc-framework-cocoa=9.2
  - pyparsing=3.0.9
  - pysocks=1.7.1
  - pytables=3.8.0
  - python=3.11.4
  - python-dateutil=2.8.2
  - python-fastjsonschema=2.18.0
  - python-igraph=0.11.3
  - python-json-logger=2.0.7
  - python-kaleido=0.2.1
  - python-tzdata=2023.3
  - python_abi=3.11
  - pytorch=2.0.1
  - pytz=2023.3
  - pyvcf3=1.0.3
  - pyyaml=6.0.1
  - pyzmq=25.1.1
  - radian=0.6.7
  - rchitect=0.4.1
  - readline=8.2
  - referencing=0.30.2
  - requests=2.31.0
  - rfc3339-validator=0.1.4
  - rfc3986-validator=0.1.1
  - rpds-py=0.9.2
  - scanpy=1.10.1
  - scikit-learn=1.3.0
  - scipy=1.11.2
  - seaborn=0.13.2
  - seaborn-base=0.13.2
  - send2trash=1.8.2
  - session-info=1.0.0
  - setuptools=68.1.2
  - simplejson=3.19.2
  - six=1.16.0
  - snappy=1.1.10
  - sniffio=1.3.0
  - soupsieve=2.3.2.post1
  - stack_data=0.6.2
  - statsmodels=0.14.0
  - stdlib-list=0.10.0
  - svt-av1=1.6.0
  - sympy=1.12
  - tbb=2021.11.0
  - tenacity=8.2.3
  - terminado=0.17.1
  - texttable=1.7.0
  - threadpoolctl=3.2.0
  - tinycss2=1.2.1
  - tk=8.6.12
  - tomli=2.0.1
  - torchvision=0.15.2
  - tornado=6.3.3
  - traitlets=5.9.0
  - typing_extensions=4.8.0
  - typing_utils=0.1.0
  - tzdata=2023c
  - umap-learn=0.5.5
  - uri-template=1.3.0
  - wcwidth=0.2.6
  - webcolors=1.13
  - webencodings=0.5.1
  - websocket-client=1.6.2
  - wheel=0.41.2
  - x264=1!164.3095
  - x265=3.5
  - xlrd=1.2.0
  - xorg-libxau=1.0.11
  - xorg-libxdmcp=1.1.3
  - xz=5.2.6
  - yaml=0.2.5
  - zeromq=4.3.4
  - zipp=3.16.2
  - zlib=1.2.13
  - zlib-ng=2.0.7
  - zstd=1.5.2
  - pip:
      - absl-py==1.4.0
      - astunparse==1.6.3
      - bcbio-gff==0.7.0
      - biopython==1.81
      - cachetools==5.3.1
      - click==8.1.7
      - flatbuffers==23.5.26
      - gast==0.4.0
      - geoparse==2.0.3
      - gffpandas==1.2.0
      - google-auth==2.22.0
      - google-auth-oauthlib==1.0.0
      - google-pasta==0.2.0
      - grpcio==1.57.0
      - imageio==2.34.1
      - keras==2.13.1
      - lazy-loader==0.4
      - libclang==16.0.6
      - louvain==0.8.2
      - markdown==3.4.4
      - numpy==1.24.3
      - oauthlib==3.2.2
      - opt-einsum==3.3.0
      - protobuf==4.24.1
      - pyasn1==0.5.0
      - pyasn1-modules==0.3.0
      - requests-oauthlib==1.3.1
      - rsa==4.9
      - scikit-image==0.24.0
      - tensorboard==2.13.0
      - tensorboard-data-server==0.7.1
      - tensorflow==2.13.0
      - tensorflow-estimator==2.13.0
      - tensorflow-macos==2.13.0
      - termcolor==2.3.0
      - tifffile==2024.6.18
      - tqdm==4.66.1
      - typing-extensions==4.5.0
      - urllib3==1.26.16
      - werkzeug==2.3.7
      - wrapt==1.15.0

Minimal code sample

sc.pp.scrublet(adata)

Error output

No response

Versions

``` # Successful case ----- anndata 0.10.5.post1 scanpy 1.10.1 ----- PIL 9.4.0 astunparse 1.6.3 cffi 1.15.1 colorama 0.4.6 cycler 0.10.0 cython_runtime NA dateutil 2.8.2 defusedxml 0.7.1 dill 0.3.7 gmpy2 2.1.2 google NA h5py 3.9.0 igraph 0.11.3 joblib 1.3.2 kiwisolver 1.4.4 legacy_api_wrap NA leidenalg 0.10.2 llvmlite 0.40.1 louvain 0.8.2 matplotlib 3.7.2 mpl_toolkits NA mpmath 1.3.0 natsort 8.4.0 numba 0.57.1 numexpr 2.8.4 numpy 1.24.4 opt_einsum v3.3.0 packaging 23.1 pandas 2.0.3 pkg_resources NA plotly 5.16.1 psutil 5.9.5 pyparsing 3.0.9 pytz 2023.3 scipy 1.11.2 session_info 1.0.0 six 1.16.0 sklearn 1.3.0 sympy 1.12 texttable 1.7.0 threadpoolctl 3.2.0 torch 2.0.1 tqdm 4.66.2 typing_extensions NA wcwidth 0.2.6 yaml 6.0.1 ----- Python 3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:08:41) [Clang 15.0.7 ] macOS-14.3-arm64-arm-64bit ----- Session information updated at 2024-06-22 00:24 # Failed case ----- anndata 0.10.7 scanpy 1.10.1 ----- PIL 10.3.0 astunparse 1.6.3 cffi 1.16.0 colorama 0.4.6 cycler 0.12.1 cython_runtime NA dateutil 2.9.0 defusedxml 0.7.1 dill 0.3.8 google NA h5py 3.11.0 igraph 0.11.5 joblib 1.4.2 kiwisolver 1.4.5 legacy_api_wrap NA leidenalg 0.10.2 llvmlite 0.42.0 louvain 0.8.2 matplotlib 3.8.4 mpl_toolkits NA natsort 8.4.0 numba 0.59.1 numexpr 2.10.0 numpy 1.26.4 optree 0.11.0 packaging 24.0 pandas 2.2.2 pkg_resources NA plotly 5.22.0 psutil 5.9.8 pyparsing 3.1.2 pytz 2024.1 scipy 1.13.1 session_info 1.0.0 six 1.16.0 sklearn 1.5.0 texttable 1.7.0 threadpoolctl 3.5.0 torch 2.2.2 torchgen NA tqdm 4.66.4 typing_extensions NA wcwidth 0.2.13 yaml 6.0.1 ----- Python 3.11.4 | packaged by conda-forge | (main, Jun 10 2023, 18:10:28) [Clang 15.0.7 ] macOS-14.4.1-x86_64-i386-64bit ----- Session information updated at 2024-06-22 00:26 ```

flying-sheep commented 5 months ago

hard to say what could cause that, there are a lot of changes between the two envs.

but we might be able to pin it down with that, thank you!

jasonleongbio commented 5 months ago

@flying-sheep Thank you so much for your reply! Indeed quite a lot of packages are different between the two environments. I'm sorry for making this complicated.

The env on my desktop (where the scrublet function stopped) is actually newer and at first I thought that would not create huge problems (I recently switched to mamba instead of conda on my Intel-core desktop. I didn't use the yml from my M2-chip laptop to re-create the environment because of some dependency problems between the Intel/M2 computers).

flying-sheep commented 5 months ago

I'm sorry for making this complicated.

Not at all, giving us environment files to work with is a big improvement over e.g. typing “scanpy 1.9” into the “versions” box haha!

The env on my desktop (where the scrublet function stopped) is actually newer

yeah, I saw that, all around newer versions of things, which makes this issue especially interesting.

scverse / scanpy