search

theislab / scarches

Reference mapping for single-cell genomics
https://docs.scarches.org/en/latest/
BSD 3-Clause "New" or "Revised" License
321 stars 50 forks source link

arm64 osx platform not supported? #241

Open erzakiev opened 1 month ago

erzakiev commented 1 month ago

When trying to install scArches as per instructions in https://docs.scarches.org/en/latest/installation.html, both listed installation options fail on arm64 osx platform.

  1. When installing via a manual route, i.e. by 
conda create -n scArches # creating new environment
conda install python=3.8 # installing the recommended python version
conda install anaconda::jupyter 
pip install -U scarches

the installation finishes correctly, but when importin scPoli in the following code snippet inside of a jupyter .ipynb file:

import numpy as np
import scanpy as sc
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import classification_report
from scarches.models.scpoli import scPoli

it throws an error #211.

  1. When trying to clone the repo, it fails due to inavailability of pytorch-cuda: 
    
git clone https://github.com/theislab/scarches
cd scarches
conda env create -f envs/scarches_linux.yaml
conda activate scarches
Cloning into 'scarches'...
remote: Enumerating objects: 7553, done.
remote: Counting objects: 100% (681/681), done.
remote: Compressing objects: 100% (256/256), done.
remote: Total 7553 (delta 454), reused 618 (delta 421), pack-reused 6872
Receiving objects: 100% (7553/7553), 803.94 MiB | 5.50 MiB/s, done.
Resolving deltas: 100% (5030/5030), done.
Channels:
- pytorch
- nvidia
- defaults
- anaconda
- conda-forge
- bioconda
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: failed
Channels:
- pytorch
- nvidia
- defaults
- anaconda
- conda-forge
- bioconda
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: failed

LibMambaUnsatisfiableError: Encountered problems while solving:

Could not solve for environment specs The following package could not be installed └─ pytorch-cuda 11.8* is not installable because it requires └─ cuda 11.8. , which does not exist (perhaps a missing channel).

EnvironmentNameNotFound: Could not find conda environment: scarches You can list all discoverable environments with conda info --envs.

erzakiev commented 1 month ago

it seems like (for now, at least), when taking the first route but using python v3.9, i.e.

conda create -n scArches # creating new environment
conda install python=3.9 # installing the recommended python version
conda install anaconda::jupyter 
pip install -U scarches

it doesn't complain when importing scPoli anymore. But pip was complaining about incompatibility of phate and pandas was uninstalled, which is concerning:

``` pip install -U scarches Collecting scarches Using cached scArches-0.6.1-py3-none-any.whl.metadata (3.5 kB) Collecting scanpy>=1.6.0 (from scarches) Downloading scanpy-1.10.1-py3-none-any.whl.metadata (8.6 kB) Collecting anndata>=0.7.4 (from scarches) Downloading anndata-0.10.7-py3-none-any.whl.metadata (6.6 kB) Collecting scHPL>=1.0.0 (from scarches) Using cached scHPL-1.0.5-py3-none-any.whl.metadata (3.5 kB) Collecting h5py>=2.10.0 (from scarches) Downloading h5py-3.11.0-cp39-cp39-macosx_11_0_arm64.whl.metadata (2.5 kB) Collecting torch>=1.8.0 (from scarches) Downloading torch-2.3.0-cp39-none-macosx_11_0_arm64.whl.metadata (26 kB) Collecting numpy>=1.19.2 (from scarches) Downloading numpy-1.26.4-cp39-cp39-macosx_11_0_arm64.whl.metadata (61 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.1/61.1 kB 2.7 MB/s eta 0:00:00 Collecting scipy>=1.5.2 (from scarches) Downloading scipy-1.13.1-cp39-cp39-macosx_12_0_arm64.whl.metadata (60 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 60.6/60.6 kB 5.0 MB/s eta 0:00:00 Collecting scikit-learn>=0.23.2 (from scarches) Downloading scikit_learn-1.5.0-cp39-cp39-macosx_12_0_arm64.whl.metadata (11 kB) Collecting matplotlib>=3.3.1 (from scarches) Downloading matplotlib-3.9.0-cp39-cp39-macosx_11_0_arm64.whl.metadata (11 kB) Requirement already satisfied: pandas>=1.1.2 in /Users/administrateur/.local/lib/python3.9/site-packages (from scarches) (2.0.3) Collecting scvi-tools>=0.12.1 (from scarches) Downloading scvi_tools-1.1.2-py3-none-any.whl.metadata (12 kB) Collecting tqdm>=4.56.0 (from scarches) Using cached tqdm-4.66.4-py3-none-any.whl.metadata (57 kB) Requirement already satisfied: requests in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from scarches) (2.32.2) Collecting gdown (from scarches) Using cached gdown-5.2.0-py3-none-any.whl.metadata (5.8 kB) Collecting leidenalg (from scarches) Using cached leidenalg-0.10.2-cp38-abi3-macosx_11_0_arm64.whl.metadata (10 kB) Collecting muon (from scarches) Using cached muon-0.1.6-py3-none-any.whl.metadata (7.2 kB) Collecting array-api-compat!=1.5,>1.4 (from anndata>=0.7.4->scarches) Downloading array_api_compat-1.7.1-py3-none-any.whl.metadata (1.5 kB) Requirement already satisfied: exceptiongroup in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from anndata>=0.7.4->scarches) (1.2.0) Collecting natsort (from anndata>=0.7.4->scarches) Using cached natsort-8.4.0-py3-none-any.whl.metadata (21 kB) Requirement already satisfied: packaging>=20.0 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from anndata>=0.7.4->scarches) (23.2) Collecting contourpy>=1.0.1 (from matplotlib>=3.3.1->scarches) Downloading contourpy-1.2.1-cp39-cp39-macosx_11_0_arm64.whl.metadata (5.8 kB) Collecting cycler>=0.10 (from matplotlib>=3.3.1->scarches) Using cached cycler-0.12.1-py3-none-any.whl.metadata (3.8 kB) Collecting fonttools>=4.22.0 (from matplotlib>=3.3.1->scarches) Downloading fonttools-4.53.0-cp39-cp39-macosx_11_0_arm64.whl.metadata (162 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 162.2/162.2 kB 6.2 MB/s eta 0:00:00 Collecting kiwisolver>=1.3.1 (from matplotlib>=3.3.1->scarches) Downloading kiwisolver-1.4.5-cp39-cp39-macosx_11_0_arm64.whl.metadata (6.4 kB) Collecting pillow>=8 (from matplotlib>=3.3.1->scarches) Downloading pillow-10.3.0-cp39-cp39-macosx_11_0_arm64.whl.metadata (9.2 kB) Collecting pyparsing>=2.3.1 (from matplotlib>=3.3.1->scarches) Using cached pyparsing-3.1.2-py3-none-any.whl.metadata (5.1 kB) Requirement already satisfied: python-dateutil>=2.7 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from matplotlib>=3.3.1->scarches) (2.9.0.post0) Collecting importlib-resources>=3.2.0 (from matplotlib>=3.3.1->scarches) Using cached importlib_resources-6.4.0-py3-none-any.whl.metadata (3.9 kB) Requirement already satisfied: pytz>=2020.1 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from pandas>=1.1.2->scarches) (2024.1) Collecting tzdata>=2022.1 (from pandas>=1.1.2->scarches) Using cached tzdata-2024.1-py2.py3-none-any.whl.metadata (1.4 kB) Collecting get-annotations (from scanpy>=1.6.0->scarches) Using cached get_annotations-0.1.2-py3-none-any.whl.metadata (3.3 kB) Collecting joblib (from scanpy>=1.6.0->scarches) Using cached joblib-1.4.2-py3-none-any.whl.metadata (5.4 kB) Collecting legacy-api-wrap>=1.4 (from scanpy>=1.6.0->scarches) Using cached legacy_api_wrap-1.4-py3-none-any.whl.metadata (1.8 kB) Collecting networkx>=2.7 (from scanpy>=1.6.0->scarches) Using cached networkx-3.2.1-py3-none-any.whl.metadata (5.2 kB) Collecting numba>=0.56 (from scanpy>=1.6.0->scarches) Downloading numba-0.59.1-cp39-cp39-macosx_11_0_arm64.whl.metadata (2.7 kB) Collecting patsy (from scanpy>=1.6.0->scarches) Using cached patsy-0.5.6-py2.py3-none-any.whl.metadata (3.5 kB) Collecting pynndescent>=0.5 (from scanpy>=1.6.0->scarches) Using cached pynndescent-0.5.12-py3-none-any.whl.metadata (6.8 kB) Collecting seaborn>=0.13 (from scanpy>=1.6.0->scarches) Using cached seaborn-0.13.2-py3-none-any.whl.metadata (5.4 kB) Collecting session-info (from scanpy>=1.6.0->scarches) Using cached session_info-1.0.0.tar.gz (24 kB) Preparing metadata (setup.py) ... done Collecting statsmodels>=0.13 (from scanpy>=1.6.0->scarches) Downloading statsmodels-0.14.2-cp39-cp39-macosx_11_0_arm64.whl.metadata (9.2 kB) Collecting umap-learn!=0.5.0,>=0.5 (from scanpy>=1.6.0->scarches) Using cached umap_learn-0.5.6-py3-none-any.whl.metadata (21 kB) Collecting pandas>=1.1.2 (from scarches) Downloading pandas-1.5.3-cp39-cp39-macosx_11_0_arm64.whl.metadata (11 kB) Collecting newick~=1.0.0 (from scHPL>=1.0.0->scarches) Using cached newick-1.0.0-py2.py3-none-any.whl.metadata (2.7 kB) Collecting threadpoolctl>=3.1.0 (from scikit-learn>=0.23.2->scarches) Using cached threadpoolctl-3.5.0-py3-none-any.whl.metadata (13 kB) Collecting docrep>=0.3.2 (from scvi-tools>=0.12.1->scarches) Using cached docrep-0.3.2.tar.gz (33 kB) Preparing metadata (setup.py) ... done Collecting flax (from scvi-tools>=0.12.1->scarches) Downloading flax-0.8.4-py3-none-any.whl.metadata (10 kB) Collecting jax>=0.4.4 (from scvi-tools>=0.12.1->scarches) Downloading jax-0.4.28-py3-none-any.whl.metadata (23 kB) Collecting jaxlib>=0.4.3 (from scvi-tools>=0.12.1->scarches) Downloading jaxlib-0.4.28-cp39-cp39-macosx_11_0_arm64.whl.metadata (1.8 kB) Collecting lightning<2.2,>=2.0 (from scvi-tools>=0.12.1->scarches) Downloading lightning-2.1.4-py3-none-any.whl.metadata (57 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 57.2/57.2 kB 4.9 MB/s eta 0:00:00 Collecting ml-collections>=0.1.1 (from scvi-tools>=0.12.1->scarches) Using cached ml_collections-0.1.1.tar.gz (77 kB) Preparing metadata (setup.py) ... done Collecting mudata>=0.1.2 (from scvi-tools>=0.12.1->scarches) Using cached mudata-0.2.3-py3-none-any.whl.metadata (4.7 kB) Collecting numpyro>=0.12.1 (from scvi-tools>=0.12.1->scarches) Downloading numpyro-0.15.0-py3-none-any.whl.metadata (36 kB) Collecting optax (from scvi-tools>=0.12.1->scarches) Downloading optax-0.2.2-py3-none-any.whl.metadata (8.1 kB) Collecting pyro-ppl>=1.6.0 (from scvi-tools>=0.12.1->scarches) Using cached pyro_ppl-1.9.1-py3-none-any.whl.metadata (7.8 kB) Collecting rich>=12.0.0 (from scvi-tools>=0.12.1->scarches) Using cached rich-13.7.1-py3-none-any.whl.metadata (18 kB) Collecting torchmetrics>=0.11.0 (from scvi-tools>=0.12.1->scarches) Using cached torchmetrics-1.4.0.post0-py3-none-any.whl.metadata (19 kB) Collecting filelock (from torch>=1.8.0->scarches) Using cached filelock-3.14.0-py3-none-any.whl.metadata (2.8 kB) Requirement already satisfied: typing-extensions>=4.8.0 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from torch>=1.8.0->scarches) (4.11.0) Collecting sympy (from torch>=1.8.0->scarches) Using cached sympy-1.12.1-py3-none-any.whl.metadata (12 kB) Requirement already satisfied: jinja2 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from torch>=1.8.0->scarches) (3.1.4) Collecting fsspec (from torch>=1.8.0->scarches) Using cached fsspec-2024.6.0-py3-none-any.whl.metadata (11 kB) Requirement already satisfied: beautifulsoup4 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from gdown->scarches) (4.12.2) Collecting igraph<0.12,>=0.10.0 (from leidenalg->scarches) Downloading igraph-0.11.5-cp39-abi3-macosx_11_0_arm64.whl.metadata (3.9 kB) Collecting protobuf (from muon->scarches) Using cached protobuf-5.27.0-cp38-abi3-macosx_10_9_universal2.whl.metadata (592 bytes) Requirement already satisfied: charset-normalizer<4,>=2 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from requests->scarches) (2.0.4) Requirement already satisfied: idna<4,>=2.5 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from requests->scarches) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from requests->scarches) (2.2.1) Requirement already satisfied: certifi>=2017.4.17 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from requests->scarches) (2024.2.2) Requirement already satisfied: six in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from docrep>=0.3.2->scvi-tools>=0.12.1->scarches) (1.16.0) Collecting texttable>=1.6.2 (from igraph<0.12,>=0.10.0->leidenalg->scarches) Using cached texttable-1.7.0-py2.py3-none-any.whl.metadata (9.8 kB) Requirement already satisfied: zipp>=3.1.0 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib>=3.3.1->scarches) (3.17.0) Collecting ml-dtypes>=0.2.0 (from jax>=0.4.4->scvi-tools>=0.12.1->scarches) Downloading ml_dtypes-0.4.0-cp39-cp39-macosx_10_9_universal2.whl.metadata (20 kB) Collecting opt-einsum (from jax>=0.4.4->scvi-tools>=0.12.1->scarches) Using cached opt_einsum-3.3.0-py3-none-any.whl.metadata (6.5 kB) Requirement already satisfied: importlib-metadata>=4.6 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from jax>=0.4.4->scvi-tools>=0.12.1->scarches) (7.0.1) Requirement already satisfied: PyYAML<8.0,>=5.4 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) (6.0.1) Collecting lightning-utilities<2.0,>=0.8.0 (from lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) Using cached lightning_utilities-0.11.2-py3-none-any.whl.metadata (4.7 kB) Collecting pytorch-lightning (from lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) Downloading pytorch_lightning-2.2.5-py3-none-any.whl.metadata (21 kB) Collecting absl-py (from ml-collections>=0.1.1->scvi-tools>=0.12.1->scarches) Using cached absl_py-2.1.0-py3-none-any.whl.metadata (2.3 kB) Collecting contextlib2 (from ml-collections>=0.1.1->scvi-tools>=0.12.1->scarches) Using cached contextlib2-21.6.0-py2.py3-none-any.whl.metadata (4.1 kB) Collecting llvmlite<0.43,>=0.42.0dev0 (from numba>=0.56->scanpy>=1.6.0->scarches) Downloading llvmlite-0.42.0-cp39-cp39-macosx_11_0_arm64.whl.metadata (4.8 kB) Collecting multipledispatch (from numpyro>=0.12.1->scvi-tools>=0.12.1->scarches) Using cached multipledispatch-1.0.0-py3-none-any.whl.metadata (3.8 kB) Collecting pyro-api>=0.1.1 (from pyro-ppl>=1.6.0->scvi-tools>=0.12.1->scarches) Using cached pyro_api-0.1.2-py3-none-any.whl.metadata (2.5 kB) Collecting markdown-it-py>=2.2.0 (from rich>=12.0.0->scvi-tools>=0.12.1->scarches) Using cached markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB) Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from rich>=12.0.0->scvi-tools>=0.12.1->scarches) (2.15.1) Requirement already satisfied: soupsieve>1.2 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from beautifulsoup4->gdown->scarches) (2.5) Collecting msgpack (from flax->scvi-tools>=0.12.1->scarches) Downloading msgpack-1.0.8-cp39-cp39-macosx_11_0_arm64.whl.metadata (9.1 kB) Collecting orbax-checkpoint (from flax->scvi-tools>=0.12.1->scarches) Downloading orbax_checkpoint-0.5.15-py3-none-any.whl.metadata (1.8 kB) Collecting tensorstore (from flax->scvi-tools>=0.12.1->scarches) Downloading tensorstore-0.1.60-cp39-cp39-macosx_11_0_arm64.whl.metadata (3.0 kB) Requirement already satisfied: MarkupSafe>=2.0 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from jinja2->torch>=1.8.0->scarches) (2.1.3) Collecting chex>=0.1.86 (from optax->scvi-tools>=0.12.1->scarches) Downloading chex-0.1.86-py3-none-any.whl.metadata (17 kB) Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from requests[socks]->gdown->scarches) (1.7.1) Collecting stdlib_list (from session-info->scanpy>=1.6.0->scarches) Using cached stdlib_list-0.10.0-py3-none-any.whl.metadata (3.3 kB) Collecting mpmath<1.4.0,>=1.1.0 (from sympy->torch>=1.8.0->scarches) Using cached mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB) Collecting toolz>=0.9.0 (from chex>=0.1.86->optax->scvi-tools>=0.12.1->scarches) Using cached toolz-0.12.1-py3-none-any.whl.metadata (5.1 kB) Collecting aiohttp!=4.0.0a0,!=4.0.0a1 (from fsspec[http]<2025.0,>=2022.5.0->lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) Downloading aiohttp-3.9.5-cp39-cp39-macosx_11_0_arm64.whl.metadata (7.5 kB) Requirement already satisfied: setuptools in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from lightning-utilities<2.0,>=0.8.0->lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) (69.5.1) Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich>=12.0.0->scvi-tools>=0.12.1->scarches) Using cached mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB) Collecting etils[epath,epy] (from orbax-checkpoint->flax->scvi-tools>=0.12.1->scarches) Downloading etils-1.5.2-py3-none-any.whl.metadata (6.3 kB) Requirement already satisfied: nest_asyncio in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from orbax-checkpoint->flax->scvi-tools>=0.12.1->scarches) (1.6.0) Collecting aiosignal>=1.1.2 (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>=2022.5.0->lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) Using cached aiosignal-1.3.1-py3-none-any.whl.metadata (4.0 kB) Requirement already satisfied: attrs>=17.3.0 in /Users/administrateur/miniforge3/envs/scArches/lib/python3.9/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>=2022.5.0->lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) (23.1.0) Collecting frozenlist>=1.1.1 (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>=2022.5.0->lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) Downloading frozenlist-1.4.1-cp39-cp39-macosx_11_0_arm64.whl.metadata (12 kB) Collecting multidict<7.0,>=4.5 (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>=2022.5.0->lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) Downloading multidict-6.0.5-cp39-cp39-macosx_11_0_arm64.whl.metadata (4.2 kB) Collecting yarl<2.0,>=1.0 (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>=2022.5.0->lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) Downloading yarl-1.9.4-cp39-cp39-macosx_11_0_arm64.whl.metadata (31 kB) Collecting async-timeout<5.0,>=4.0 (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<2025.0,>=2022.5.0->lightning<2.2,>=2.0->scvi-tools>=0.12.1->scarches) Using cached async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB) Using cached scArches-0.6.1-py3-none-any.whl (130 kB) Downloading anndata-0.10.7-py3-none-any.whl (122 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 122.4/122.4 kB 849.7 kB/s eta 0:00:00 Downloading h5py-3.11.0-cp39-cp39-macosx_11_0_arm64.whl (3.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 10.2 MB/s eta 0:00:00 Downloading matplotlib-3.9.0-cp39-cp39-macosx_11_0_arm64.whl (7.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 7.8/7.8 MB 11.0 MB/s eta 0:00:00 Downloading numpy-1.26.4-cp39-cp39-macosx_11_0_arm64.whl (14.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 8.3 MB/s eta 0:00:00 Downloading scanpy-1.10.1-py3-none-any.whl (2.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 10.3 MB/s eta 0:00:00 Using cached scHPL-1.0.5-py3-none-any.whl (22 kB) Downloading pandas-1.5.3-cp39-cp39-macosx_11_0_arm64.whl (11.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.0/11.0 MB 9.6 MB/s eta 0:00:00 Downloading scikit_learn-1.5.0-cp39-cp39-macosx_12_0_arm64.whl (11.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 11.0/11.0 MB 9.9 MB/s eta 0:00:00 Downloading scipy-1.13.1-cp39-cp39-macosx_12_0_arm64.whl (30.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.3/30.3 MB 8.5 MB/s eta 0:00:00 Downloading scvi_tools-1.1.2-py3-none-any.whl (380 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 380.2/380.2 kB 7.5 MB/s eta 0:00:00 Downloading torch-2.3.0-cp39-none-macosx_11_0_arm64.whl (61.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.0/61.0 MB 9.3 MB/s eta 0:00:00 Using cached tqdm-4.66.4-py3-none-any.whl (78 kB) Using cached gdown-5.2.0-py3-none-any.whl (18 kB) Using cached leidenalg-0.10.2-cp38-abi3-macosx_11_0_arm64.whl (1.4 MB) Using cached muon-0.1.6-py3-none-any.whl (293 kB) Downloading array_api_compat-1.7.1-py3-none-any.whl (37 kB) Downloading contourpy-1.2.1-cp39-cp39-macosx_11_0_arm64.whl (244 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 245.0/245.0 kB 8.0 MB/s eta 0:00:00 Using cached cycler-0.12.1-py3-none-any.whl (8.3 kB) Downloading fonttools-4.53.0-cp39-cp39-macosx_11_0_arm64.whl (2.2 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.2/2.2 MB 10.1 MB/s eta 0:00:00 Downloading igraph-0.11.5-cp39-abi3-macosx_11_0_arm64.whl (1.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 4.6 MB/s eta 0:00:00 Using cached importlib_resources-6.4.0-py3-none-any.whl (38 kB) Downloading jax-0.4.28-py3-none-any.whl (1.9 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.9/1.9 MB 6.3 MB/s eta 0:00:00 Downloading jaxlib-0.4.28-cp39-cp39-macosx_11_0_arm64.whl (64.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.1/64.1 MB 6.7 MB/s eta 0:00:00 Using cached joblib-1.4.2-py3-none-any.whl (301 kB) Downloading kiwisolver-1.4.5-cp39-cp39-macosx_11_0_arm64.whl (66 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 66.2/66.2 kB 4.2 MB/s eta 0:00:00 Using cached legacy_api_wrap-1.4-py3-none-any.whl (15 kB) Downloading lightning-2.1.4-py3-none-any.whl (2.0 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 3.8 MB/s eta 0:00:00 Using cached fsspec-2024.6.0-py3-none-any.whl (176 kB) Using cached mudata-0.2.3-py3-none-any.whl (24 kB) Using cached networkx-3.2.1-py3-none-any.whl (1.6 MB) Using cached newick-1.0.0-py2.py3-none-any.whl (10 kB) Downloading numba-0.59.1-cp39-cp39-macosx_11_0_arm64.whl (2.6 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 4.0 MB/s eta 0:00:00 Downloading numpyro-0.15.0-py3-none-any.whl (345 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 345.7/345.7 kB 4.9 MB/s eta 0:00:00 Downloading pillow-10.3.0-cp39-cp39-macosx_11_0_arm64.whl (3.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.4/3.4 MB 4.6 MB/s eta 0:00:00 Using cached pynndescent-0.5.12-py3-none-any.whl (56 kB) Using cached pyparsing-3.1.2-py3-none-any.whl (103 kB) Using cached pyro_ppl-1.9.1-py3-none-any.whl (755 kB) Using cached rich-13.7.1-py3-none-any.whl (240 kB) Using cached seaborn-0.13.2-py3-none-any.whl (294 kB) Downloading statsmodels-0.14.2-cp39-cp39-macosx_11_0_arm64.whl (10.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.1/10.1 MB 4.3 MB/s eta 0:00:00 Using cached patsy-0.5.6-py2.py3-none-any.whl (233 kB) Using cached threadpoolctl-3.5.0-py3-none-any.whl (18 kB) Using cached torchmetrics-1.4.0.post0-py3-none-any.whl (868 kB) Using cached umap_learn-0.5.6-py3-none-any.whl (85 kB) Using cached filelock-3.14.0-py3-none-any.whl (12 kB) Downloading flax-0.8.4-py3-none-any.whl (698 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 698.6/698.6 kB 3.2 MB/s eta 0:00:00 Using cached get_annotations-0.1.2-py3-none-any.whl (4.5 kB) Using cached natsort-8.4.0-py3-none-any.whl (38 kB) Downloading optax-0.2.2-py3-none-any.whl (223 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 223.7/223.7 kB 1.6 MB/s eta 0:00:00 Using cached protobuf-5.27.0-cp38-abi3-macosx_10_9_universal2.whl (412 kB) Using cached sympy-1.12.1-py3-none-any.whl (5.7 MB) Using cached absl_py-2.1.0-py3-none-any.whl (133 kB) Downloading chex-0.1.86-py3-none-any.whl (98 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.2/98.2 kB 2.4 MB/s eta 0:00:00 Using cached lightning_utilities-0.11.2-py3-none-any.whl (26 kB) Downloading llvmlite-0.42.0-cp39-cp39-macosx_11_0_arm64.whl (28.8 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 28.8/28.8 MB 4.1 MB/s eta 0:00:00 Using cached markdown_it_py-3.0.0-py3-none-any.whl (87 kB) Downloading ml_dtypes-0.4.0-cp39-cp39-macosx_10_9_universal2.whl (390 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 390.9/390.9 kB 6.1 MB/s eta 0:00:00 Using cached mpmath-1.3.0-py3-none-any.whl (536 kB) Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB) Using cached pyro_api-0.1.2-py3-none-any.whl (11 kB) Using cached texttable-1.7.0-py2.py3-none-any.whl (10 kB) Using cached contextlib2-21.6.0-py2.py3-none-any.whl (13 kB) Downloading msgpack-1.0.8-cp39-cp39-macosx_11_0_arm64.whl (85 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.1/85.1 kB 6.7 MB/s eta 0:00:00 Using cached multipledispatch-1.0.0-py3-none-any.whl (12 kB) Downloading orbax_checkpoint-0.5.15-py3-none-any.whl (210 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 210.8/210.8 kB 4.7 MB/s eta 0:00:00 Downloading tensorstore-0.1.60-cp39-cp39-macosx_11_0_arm64.whl (13.5 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.5/13.5 MB 5.8 MB/s eta 0:00:00 Downloading pytorch_lightning-2.2.5-py3-none-any.whl (802 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 802.3/802.3 kB 2.0 MB/s eta 0:00:00 Using cached stdlib_list-0.10.0-py3-none-any.whl (79 kB) Downloading aiohttp-3.9.5-cp39-cp39-macosx_11_0_arm64.whl (390 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 390.7/390.7 kB 4.7 MB/s eta 0:00:00 Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB) Using cached toolz-0.12.1-py3-none-any.whl (56 kB) Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB) Using cached async_timeout-4.0.3-py3-none-any.whl (5.7 kB) Downloading frozenlist-1.4.1-cp39-cp39-macosx_11_0_arm64.whl (53 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.7/53.7 kB 2.9 MB/s eta 0:00:00 Downloading multidict-6.0.5-cp39-cp39-macosx_11_0_arm64.whl (30 kB) Downloading yarl-1.9.4-cp39-cp39-macosx_11_0_arm64.whl (81 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 81.8/81.8 kB 644.0 kB/s eta 0:00:00 Downloading etils-1.5.2-py3-none-any.whl (140 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 140.6/140.6 kB 3.1 MB/s eta 0:00:00 Building wheels for collected packages: docrep, ml-collections, session-info Building wheel for docrep (setup.py) ... done Created wheel for docrep: filename=docrep-0.3.2-py3-none-any.whl size=19878 sha256=043c44d5311f9f486d5f67448ca9e0e1952290f7906a87377408f8cd2be7ada3 Stored in directory: /Users/administrateur/Library/Caches/pip/wheels/e5/f0/3f/17394a03ed36922f620186a5a9ebd6bce2b4579020243c7a68 Building wheel for ml-collections (setup.py) ... done Created wheel for ml-collections: filename=ml_collections-0.1.1-py3-none-any.whl size=94508 sha256=ca2b290da869ab787897be78fc88fec1421c144269b85ff1548c6f05ce014c53 Stored in directory: /Users/administrateur/Library/Caches/pip/wheels/fd/c2/0d/5d94d95e5875ea17b85a9f1f99b8dd2e50517137c8042c6468 Building wheel for session-info (setup.py) ... done Created wheel for session-info: filename=session_info-1.0.0-py3-none-any.whl size=8024 sha256=caff56637067b41cef5c5bc154b0a2644613a0e74e5c821f9e13de69e36f53bb Stored in directory: /Users/administrateur/Library/Caches/pip/wheels/d4/fc/2e/00ca60bac7954b84907efd41baa9b4853500eaeec4228410c6 Successfully built docrep ml-collections session-info Installing collected packages: texttable, pyro-api, newick, multipledispatch, mpmath, array-api-compat, tqdm, toolz, threadpoolctl, sympy, stdlib_list, pyparsing, protobuf, pillow, numpy, networkx, natsort, multidict, msgpack, mdurl, llvmlite, lightning-utilities, legacy-api-wrap, kiwisolver, joblib, importlib-resources, igraph, get-annotations, fsspec, frozenlist, fonttools, filelock, etils, docrep, cycler, contextlib2, async-timeout, absl-py, yarl, torch, session-info, scipy, patsy, pandas, opt-einsum, numba, ml-dtypes, ml-collections, markdown-it-py, leidenalg, h5py, contourpy, aiosignal, torchmetrics, tensorstore, statsmodels, scikit-learn, rich, pyro-ppl, matplotlib, jaxlib, jax, gdown, anndata, aiohttp, seaborn, pynndescent, orbax-checkpoint, numpyro, mudata, chex, umap-learn, scHPL, pytorch-lightning, optax, scanpy, lightning, flax, scvi-tools, muon, scarches Attempting uninstall: pandas Found existing installation: pandas 2.0.3 Uninstalling pandas-2.0.3: Successfully uninstalled pandas-2.0.3 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. phate 1.0.11 requires Deprecated, which is not installed. phate 1.0.11 requires future, which is not installed. phate 1.0.11 requires graphtools>=1.5.3, which is not installed. phate 1.0.11 requires s_gd2>=1.8.1, which is not installed. phate 1.0.11 requires scprep>=0.11.1, which is not installed. phate 1.0.11 requires tasklogger>=1.2, which is not installed. Successfully installed absl-py-2.1.0 aiohttp-3.9.5 aiosignal-1.3.1 anndata-0.10.7 array-api-compat-1.7.1 async-timeout-4.0.3 chex-0.1.86 contextlib2-21.6.0 contourpy-1.2.1 cycler-0.12.1 docrep-0.3.2 etils-1.5.2 filelock-3.14.0 flax-0.8.4 fonttools-4.53.0 frozenlist-1.4.1 fsspec-2024.6.0 gdown-5.2.0 get-annotations-0.1.2 h5py-3.11.0 igraph-0.11.5 importlib-resources-6.4.0 jax-0.4.28 jaxlib-0.4.28 joblib-1.4.2 kiwisolver-1.4.5 legacy-api-wrap-1.4 leidenalg-0.10.2 lightning-2.1.4 lightning-utilities-0.11.2 llvmlite-0.42.0 markdown-it-py-3.0.0 matplotlib-3.9.0 mdurl-0.1.2 ml-collections-0.1.1 ml-dtypes-0.4.0 mpmath-1.3.0 msgpack-1.0.8 mudata-0.2.3 multidict-6.0.5 multipledispatch-1.0.0 muon-0.1.6 natsort-8.4.0 networkx-3.2.1 newick-1.0.0 numba-0.59.1 numpy-1.26.4 numpyro-0.15.0 opt-einsum-3.3.0 optax-0.2.2 orbax-checkpoint-0.5.15 pandas-1.5.3 patsy-0.5.6 pillow-10.3.0 protobuf-5.27.0 pynndescent-0.5.12 pyparsing-3.1.2 pyro-api-0.1.2 pyro-ppl-1.9.1 pytorch-lightning-2.2.5 rich-13.7.1 scHPL-1.0.5 scanpy-1.10.1 scarches-0.6.1 scikit-learn-1.5.0 scipy-1.13.1 scvi-tools-1.1.2 seaborn-0.13.2 session-info-1.0.0 statsmodels-0.14.2 stdlib_list-0.10.0 sympy-1.12.1 tensorstore-0.1.60 texttable-1.7.0 threadpoolctl-3.5.0 toolz-0.12.1 torch-2.3.0 torchmetrics-1.4.0.post0 tqdm-4.66.4 umap-learn-0.5.6 yarl-1.9.4 ```
erzakiev commented 4 weeks ago

So I was able to run the code in what seems to be - judging by the speed of training - a cpu mode. But when trying train, for instance, an scGen model - it takes ages. I suspect that it's because on macos there's no pytorch-cuda, but we do have mps.

When running, for instance, a simple training from the tutorial, 

network = sca.models.scgen(adata = source_adata,
                           hidden_layer_sizes=[256,128])
network.train(n_epochs=epoch, early_stopping_kwargs = early_stopping_kwargs)
 |██------------------| 14.0%  - epoch_loss: 0.0647465525 - val_loss: 0.0618337077

the GPU usage is minimal, yet CPU usage is high, so it's def not using the mps:

CPU Power: 30096 mW
GPU Power: 66 mW
ANE Power: 0 mW
Combined Power (CPU + GPU + ANE): 30162 mW

Are scgen and other tools from the scArches toolkit limited to cuda for GPU calculations? Just to know, if it's even worth trying to make it work or if I just need to never ever run it on a mac and always go for linux if I need to train a model