search

kevinrue / velociraptor

Toolkit for Single-Cell Velocity
https://kevinrue.github.io/velociraptor/
Other
55 stars 10 forks source link

Some packages don't install through conda #63

Closed gdagstn closed 5 months ago

gdagstn commented 1 year ago

Hi,

I am working on a M1 Macbook Pro and I had some issues running scvelo() (from the Bioconductor 3.16 release) due to some packages being apparently unavailable:

PackagesNotFoundError: The following packages are not available from current channels:

  - numba=0.51.2
  - numexpr=2.7.1
  - llvmlite=0.34.0
  - stdlib-list=0.6.0
  - statsmodels=0.12.0
  - kiwisolver=1.2.0
  - umap-learn=0.4.6
  - cffi=1.14.1
  - anndata=0.7.4
  - tornado=6.0.4
  - matplotlib=3.3.1
  - pillow=7.2.0
  - pandas=1.1.2
  - numpy=1.19.1

Since the proper arm64 miniconda version was installed, I was able to circumvent this by running the creation of the conda environment and installation of the scvelo module manually, i.e. 

/Users/gdagostino/Library/Caches/org.R-project.R/R/basilisk/1.10.2/0/bin/conda create --yes --prefix /Users/gdagostino/Library/Caches/org.R-project.R/R/basilisk/1.10.2/velociraptor/1.8.0/env python=3.8.13 --quiet -c bioconda -c conda-forge

/Users/gdagostino/Library/Caches/org.R-project.R/R/basilisk/1.10.2/0/bin/conda install --yes --prefix /Users/gdagostino/Library/Caches/org.R-project.R/R/basilisk/1.10.2/velociraptor/1.8.0/env -c bioconda -c conda-forge scvelo

It works properly after this, but I do see that this is likely going to install slightly different package versions from the ones specified in the original manifest in velociraptor, so I'm not sure this hacky fix is the way to go.

Is there something else I should do/should have done?

sessionInfo() ``` R version 4.2.1 (2022-06-23) Platform: aarch64-apple-darwin20 (64-bit) Running under: macOS Monterey 12.5.1 Matrix products: default BLAS: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRblas.0.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib locale: [1] C/UTF-8/C/C/C/C attached base packages: [1] stats4 stats graphics grDevices utils datasets methods [8] base other attached packages: [1] scater_1.26.1 ggplot2_3.4.0 [3] scran_1.26.1 scuttle_1.8.3 [5] SingleCellExperiment_1.20.0 velociraptor_1.8.0 [7] SummarizedExperiment_1.28.0 Biobase_2.58.0 [9] GenomicRanges_1.50.2 GenomeInfoDb_1.34.6 [11] IRanges_2.32.0 S4Vectors_0.36.1 [13] BiocGenerics_0.44.0 MatrixGenerics_1.10.0 [15] matrixStats_0.63.0 loaded via a namespace (and not attached): [1] viridis_0.6.2 edgeR_3.40.1 [3] BiocSingular_1.14.0 viridisLite_0.4.1 [5] jsonlite_1.8.4 here_1.0.1 [7] DelayedMatrixStats_1.20.0 assertthat_0.2.1 [9] statmod_1.4.37 BiocManager_1.30.19 [11] dqrng_0.3.0 vipor_0.4.5 [13] GenomeInfoDbData_1.2.9 ggrepel_0.9.2 [15] pillar_1.8.1 lattice_0.20-45 [17] glue_1.6.2 limma_3.54.0 [19] beachmat_2.14.0 reticulate_1.26 [21] XVector_0.38.0 colorspace_2.0-3 [23] Matrix_1.5-3 pkgconfig_2.0.3 [25] zellkonverter_1.8.0 dir.expiry_1.6.0 [27] zlibbioc_1.44.0 scales_1.2.1 [29] ScaledMatrix_1.6.0 BiocParallel_1.32.5 [31] tibble_3.1.8 generics_0.1.3 [33] withr_2.5.0 cli_3.5.0 [35] magrittr_2.0.3 fansi_1.0.3 [37] bluster_1.8.0 beeswarm_0.4.0 [39] tools_4.2.1 lifecycle_1.0.3 [41] basilisk.utils_1.10.0 munsell_0.5.0 [43] locfit_1.5-9.6 cluster_2.1.4 [45] DelayedArray_0.24.0 irlba_2.3.5.1 [47] compiler_4.2.1 rsvd_1.0.5 [49] rlang_1.0.6 grid_4.2.1 [51] RCurl_1.98-1.9 BiocNeighbors_1.16.0 [53] igraph_1.3.5 bitops_1.0-7 [55] basilisk_1.10.2 gtable_0.3.1 [57] codetools_0.2-18 DBI_1.1.3 [59] R6_2.5.1 gridExtra_2.3 [61] dplyr_1.0.10 utf8_1.2.2 [63] rprojroot_2.0.3 filelock_1.0.2 [65] metapod_1.6.0 ggbeeswarm_0.7.1 [67] parallel_4.2.1 Rcpp_1.0.9 [69] vctrs_0.5.1 png_0.1-8 [71] tidyselect_1.2.0 sparseMatrixStats_1.10.0 ```
LTLA commented 1 year ago

FWIW, it could be possible to detect if the system is Arm (see flags in jlmelville/uwot#100) and have basilisk.R switch between package manifests at environment creation time. This would require a manifest from someone who actually has an M1/2, though.

gdagstn commented 1 year ago

If this is something I can help with I'm happy to do so.

Would it be useful if I went for a fresh manual env creation + install and reported here the versions installed by conda install -c bioconda -c conda-forge scvelo?

julien-roux commented 1 year ago

I am facing the same issue using the latest version of velociraptor (v1.10.0, Bioc 3.17) on an M1 macbook pro

Would it be useful if I went for a fresh manual env creation + install and reported here the versions installed by conda install -c bioconda -c conda-forge scvelo?

Yes definitely... Wouldn't it also be a good opportunity to update scvelo to the latest version (0.2.5)? https://anaconda.org/bioconda/scvelo

conda create -c conda-forge -c bioconda -n scvelo_0.2.5 scvelo=0.2.5
conda activate scvelo_0.2.5
conda list
# packages in environment at /Users/rouxj/miniconda3/envs/scvelo_0.2.5:
#
# Name                    Version                   Build  Channel
anndata                   0.9.2              pyhd8ed1ab_0    conda-forge
brotli                    1.0.9                hb7f2c08_9    conda-forge
brotli-bin                1.0.9                hb7f2c08_9    conda-forge
brotli-python             1.0.9           py311h814d153_9    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
c-ares                    1.19.1               h0dc2134_0    conda-forge
ca-certificates           2023.7.22            h8857fd0_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
certifi                   2023.7.22          pyhd8ed1ab_0    conda-forge
charset-normalizer        3.2.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
contourpy                 1.1.0           py311h5fe6e05_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
fonttools                 4.42.0          py311h2725bcf_0    conda-forge
freetype                  2.12.1               h3f81eb7_1    conda-forge
h5py                      3.9.0           nompi_py311hc915cd5_101    conda-forge
hdf5                      1.14.1          nompi_hedada53_100    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
joblib                    1.3.2              pyhd8ed1ab_0    conda-forge
kiwisolver                1.4.4           py311hd2070f0_1    conda-forge
krb5                      1.21.2               hb884880_0    conda-forge
lcms2                     2.15                 h2dcdeff_1    conda-forge
lerc                      4.0.0                hb486fe8_0    conda-forge
libaec                    1.0.6                hf0c8a7f_1    conda-forge
libblas                   3.9.0           17_osx64_openblas    conda-forge
libbrotlicommon           1.0.9                hb7f2c08_9    conda-forge
libbrotlidec              1.0.9                hb7f2c08_9    conda-forge
libbrotlienc              1.0.9                hb7f2c08_9    conda-forge
libcblas                  3.9.0           17_osx64_openblas    conda-forge
libcurl                   8.2.1                h5f667d7_0    conda-forge
libcxx                    16.0.6               hd57cbcb_0    conda-forge
libdeflate                1.18                 hac1461d_0    conda-forge
libedit                   3.1.20191231         h0678c8f_2    conda-forge
libev                     4.33                 haf1e3a3_1    conda-forge
libexpat                  2.5.0                hf0c8a7f_1    conda-forge
libffi                    3.4.2                h0d85af4_5    conda-forge
libgfortran               5.0.0           12_3_0_h97931a8_1    conda-forge
libgfortran5              12.3.0               hbd3c1fe_1    conda-forge
libjpeg-turbo             2.1.5.1              hb7f2c08_0    conda-forge
liblapack                 3.9.0           17_osx64_openblas    conda-forge
libllvm14                 14.0.6               hc8e404f_4    conda-forge
libnghttp2                1.52.0               he2ab024_0    conda-forge
libopenblas               0.3.23          openmp_h429af6e_0    conda-forge
libpng                    1.6.39               ha978bb4_0    conda-forge
libsqlite                 3.42.0               h58db7d2_0    conda-forge
libssh2                   1.11.0               hd019ec5_0    conda-forge
libtiff                   4.5.1                hf955e92_0    conda-forge
libwebp-base              1.3.1                h0dc2134_0    conda-forge
libxcb                    1.15                 hb7f2c08_0    conda-forge
libzlib                   1.2.13               h8a1eda9_5    conda-forge
llvm-openmp               16.0.6               hff08bdf_0    conda-forge
llvmlite                  0.40.1          py311hcbb5c6d_0    conda-forge
loompy                    3.0.6                      py_0    conda-forge
matplotlib-base           3.7.1           py311h2bf763f_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
natsort                   8.4.0              pyhd8ed1ab_0    conda-forge
ncurses                   6.4                  hf0c8a7f_0    conda-forge
networkx                  3.1                pyhd8ed1ab_0    conda-forge
numba                     0.57.1          py311h5a8220d_0    conda-forge
numpy                     1.24.4          py311hc44ba51_0    conda-forge
numpy_groupies            0.9.22             pyhd8ed1ab_0    conda-forge
openjpeg                  2.5.0                h13ac156_2    conda-forge
openssl                   3.1.2                h8a1eda9_0    conda-forge
packaging                 23.1               pyhd8ed1ab_0    conda-forge
pandas                    2.0.3           py311hab14417_1    conda-forge
patsy                     0.5.3              pyhd8ed1ab_0    conda-forge
pillow                    10.0.0          py311h7cb0e2d_0    conda-forge
pip                       23.2.1             pyhd8ed1ab_0    conda-forge
platformdirs              3.10.0             pyhd8ed1ab_0    conda-forge
pooch                     1.7.0              pyha770c72_3    conda-forge
pthread-stubs             0.4               hc929b4f_1001    conda-forge
pynndescent               0.5.10             pyh1a96a4e_0    conda-forge
pyparsing                 3.1.1              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.11.4          h30d4d87_0_cpython    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-tzdata             2023.3             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    3_cp311    conda-forge
pytz                      2023.3             pyhd8ed1ab_0    conda-forge
readline                  8.2                  h9e318b2_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
scanpy                    1.9.3              pyhd8ed1ab_0    conda-forge
scikit-learn              1.3.0           py311h83feae1_0    conda-forge
scipy                     1.11.1          py311h16c3c4d_0    conda-forge
scvelo                    0.2.5              pyhdfd78af_0    bioconda
seaborn                   0.12.2               hd8ed1ab_0    conda-forge
seaborn-base              0.12.2             pyhd8ed1ab_0    conda-forge
session-info              1.0.0              pyhd8ed1ab_0    conda-forge
setuptools                68.0.0             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
statsmodels               0.14.0          py311h4a70a88_1    conda-forge
stdlib-list               0.8.0              pyhd8ed1ab_0    conda-forge
tbb                       2021.10.0            h1c7c39f_0    conda-forge
threadpoolctl             3.2.0              pyha21a80b_0    conda-forge
tk                        8.6.12               h5dbffcc_0    conda-forge
tqdm                      4.66.1             pyhd8ed1ab_0    conda-forge
typing-extensions         4.7.1                hd8ed1ab_0    conda-forge
typing_extensions         4.7.1              pyha770c72_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
umap-learn                0.5.3           py311h6eed73b_1    conda-forge
urllib3                   2.0.4              pyhd8ed1ab_0    conda-forge
wheel                     0.41.1             pyhd8ed1ab_0    conda-forge
xorg-libxau               1.0.11               h0dc2134_0    conda-forge
xorg-libxdmcp             1.1.3                h35c211d_0    conda-forge
xz                        5.2.6                h775f41a_0    conda-forge
zipp                      3.16.2             pyhd8ed1ab_0    conda-forge
zstd                      1.5.2                h829000d_7    conda-forge

Let me know if I can help!

kevinrue commented 1 year ago

Woops. Thanks for the fresh comment. Completely fell off my radar (@LTLA 's too, I presume).

I would also take the offer for a conda environment manifest. I'm swamped at the moment.

csoneson commented 1 year ago

I think we may have to upgrade scvelo to make things work on the M1 mac. I tried to just set the versions of the missing packages to the oldest versions available for M1, and I get a conflict for scanpy 1.6.0, which doesn't work with these versions that are slightly newer than the ones currently specified. I haven't tried scvelo v0.2.5 yet, not sure if anything changed in the API or if things will just work out of the box.

MikeDMorgan commented 5 months ago

I am experiencing a similar issue to @gdagstn and @julien-roux using an M2 Mac:

PackagesNotFoundError: The following packages are not available from current channels:

  - stdlib-list=0.6.0
  - umap-learn=0.4.6
  - matplotlib=3.3.1
  - llvmlite=0.34.0
  - kiwisolver=1.2.0
  - pandas=1.1.2
  - numexpr=2.7.1
  - cffi=1.14.1
  - anndata=0.7.4
  - statsmodels=0.12.0
  - tornado=6.0.4
  - pillow=7.2.0
  - numpy=1.19.1
  - numba=0.51.2

Similarly, manually installing the packages works, however, only when the versions are not specified, e.g.

/Users/s08mm2/Library/Caches/org.R-project.R/R/basilisk/1.16.0/0/bin/conda install --yes --prefix /Users/s08mm2/Library/Caches/org.R-project.R/R/basilisk/1.16.0/velociraptor/1.14.0/env -c bioconda -c conda-forge 'python=3.8' pandas

But not:

/Users/s08mm2/Library/Caches/org.R-project.R/R/basilisk/1.16.0/0/bin/conda install --yes --prefix /Users/s08mm2/Library/Caches/org.R-project.R/R/basilisk/1.16.0/velociraptor/1.14.0/env -c bioconda -c conda-forge 'python=3.8' 'pandas=1.1.2'

Conda output:

Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): | WARNING conda.models.version:get_matcher(535): Using .* with relational operator is superfluous and deprecated and will be removed in a future version of conda. Your spec was 1.8.0.*, but conda is ignoring the .* and treating it as 1.8.0
WARNING conda.models.version:get_matcher(535): Using .* with relational operator is superfluous and deprecated and will be removed in a future version of conda. Your spec was 1.9.0.*, but conda is ignoring the .* and treating it as 1.9.0
done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.

PackagesNotFoundError: The following packages are not available from current channels:

  - pandas=1.1.2

Current channels:

  - https://conda.anaconda.org/bioconda/osx-arm64
  - https://conda.anaconda.org/bioconda/noarch
  - https://conda.anaconda.org/conda-forge/osx-arm64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/osx-arm64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/r/osx-arm64
  - https://repo.anaconda.com/pkgs/r/noarch

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Subsequently, scvelo will fail when trying to import pandas.is_categorical, which is deprecated since v.1.1.0 (https://pandas.pydata.org/pandas-docs/version/1.5/reference/api/pandas.api.types.is_categorical.html):

itec.velo <- scvelo(loom.sce, assay.X="X", assay.spliced="spliced", assay.unspliced="unspliced",
+                     dimred="PCA", mode="dynamical")
Error in py_module_import(module, convert = convert) : 
  ImportError: cannot import name 'is_categorical' from 'pandas.api.types' (/Users/s08mm2/Library/Caches/org.R-project.R/R/basilisk/1.16.0/velociraptor/1.14.0/env/lib/python3.8/site-packages/pandas/api/types/__init__.py)
Run `reticulate::py_last_error()` for details.
Error in .activate_fallback(proc, testload, env = env, envpath = envpath,  : 
  ImportError: cannot import name 'is_categorical' from 'pandas.api.types' (/Users/s08mm2/Library/Caches/org.R-project.R/R/basilisk/1.16.0/velociraptor/1.14.0/env/lib/python3.8/site-packages/pandas/api/types/__init__.py)
Run `reticulate::py_last_error()` for details.
SessionInfo > sessionInfo() R version 4.4.0 RC (2024-04-16 r86457) Platform: aarch64-apple-darwin20 Running under: macOS Sonoma 14.1.1 Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib; LAPACK version 3.12.0 locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8 time zone: Europe/London tzcode source: internal attached base packages: [1] stats4 stats graphics grDevices utils datasets methods base other attached packages: [1] velociraptor_1.14.0 scran_1.32.0 scater_1.32.0 ggplot2_3.5.1 scuttle_1.14.0 SingleCellExperiment_1.26.0 [7] SummarizedExperiment_1.34.0 Biobase_2.64.0 GenomicRanges_1.56.0 GenomeInfoDb_1.40.0 IRanges_2.38.0 S4Vectors_0.42.0 [13] BiocGenerics_0.50.0 MatrixGenerics_1.16.0 matrixStats_1.3.0 basilisk_1.16.0 reticulate_1.36.1 zellkonverter_1.14.0 [19] miloR_2.0.0 edgeR_4.2.0 limma_3.60.0 loaded via a namespace (and not attached): [1] gridExtra_2.3 rlang_1.1.3 magrittr_2.0.3 compiler_4.4.0 dir.expiry_1.12.0 DelayedMatrixStats_1.26.0 [7] png_0.1-8 vctrs_0.6.5 stringr_1.5.1 pkgconfig_2.0.3 crayon_1.5.2 fastmap_1.2.0 [13] XVector_0.44.0 ggraph_2.2.1 utf8_1.2.4 UCSC.utils_1.0.0 ggbeeswarm_0.7.2 purrr_1.0.2 [19] bluster_1.14.0 xfun_0.44 zlibbioc_1.50.0 cachem_1.1.0 beachmat_2.20.0 jsonlite_1.8.8 [25] DelayedArray_0.30.0 BiocParallel_1.38.0 tweenr_2.0.3 cluster_2.1.6 irlba_2.3.5.1 parallel_4.4.0 [31] R6_2.5.1 stringi_1.8.4 RColorBrewer_1.1-3 numDeriv_2016.8-1.1 Rcpp_1.0.12 knitr_1.46 [37] Matrix_1.7-0 igraph_2.0.3 tidyselect_1.2.1 rstudioapi_0.16.0 abind_1.4-5 viridis_0.6.5 [43] codetools_0.2-20 lattice_0.22-6 tibble_3.2.1 basilisk.utils_1.16.0 withr_3.0.0 polyclip_1.10-6 [49] pillar_1.9.0 filelock_1.0.3 generics_0.1.3 sparseMatrixStats_1.16.0 munsell_0.5.1 scales_1.3.0 [55] gtools_3.9.5 glue_1.7.0 metapod_1.12.0 tools_4.4.0 BiocNeighbors_1.22.0 ScaledMatrix_1.12.0 [61] locfit_1.5-9.9 graphlayouts_1.1.1 tidygraph_1.3.1 cowplot_1.1.3 grid_4.4.0 tidyr_1.3.1 [67] colorspace_2.1-0 GenomeInfoDbData_1.2.12 patchwork_1.2.0 beeswarm_0.4.0 BiocSingular_1.20.0 ggforce_0.4.2 [73] vipor_0.4.7 cli_3.6.2 rsvd_1.0.5 fansi_1.0.6 S4Arrays_1.4.0 viridisLite_0.4.2 [79] dplyr_1.1.4 gtable_0.3.5 dqrng_0.4.0 SparseArray_1.4.1 ggrepel_0.9.5 farver_2.1.2 [85] memoise_2.0.1 lifecycle_1.0.4 httr_1.4.7 statmod_1.5.0 MASS_7.3-60.2
kevinrue commented 5 months ago

Looking into it today.

I've used the versions listed in https://github.com/kevinrue/velociraptor/issues/63#issuecomment-1683514855 to try quickly update the versions of dependencies, but ran into

Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

I've also tried to figure out new environment specs a few different ways, all leading to various kinds of errors. Will continue investigating as time allows.

Help welcome if anyone is keen. Can't blame anyone for the complexity of the task!

kevinrue commented 5 months ago

Following 

mamba create -c conda-forge -c bioconda -n scvelo_0.2.5 scvelo=0.2.5
mamba activate scvelo_0.2.5
conda activate scvelo_0.2.5

I get the environment

(scvelo_0.2.5) kevin@Kevins-Laptop ~ % conda list
# packages in environment at /Users/kevin/miniforge3/envs/scvelo_0.2.5:
#
# Name                    Version                   Build  Channel
absl-py                   2.1.0              pyhd8ed1ab_0    conda-forge
anndata                   0.10.7             pyhd8ed1ab_0    conda-forge
anyio                     4.3.0              pyhd8ed1ab_0    conda-forge
aom                       3.9.0                h537b5a7_0    conda-forge
array-api-compat          1.6                pyhd8ed1ab_0    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
backoff                   2.2.1              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
blessed                   1.19.1             pyhe4f9e05_2    conda-forge
blosc                     1.21.5               h9c252e8_1    conda-forge
boto3                     1.34.109           pyhd8ed1ab_0    conda-forge
botocore                  1.34.110        pyge310_1234567_0    conda-forge
brotli                    1.1.0                hb547adb_1    conda-forge
brotli-bin                1.1.0                hb547adb_1    conda-forge
brotli-python             1.1.0           py312h9f69965_1    conda-forge
bzip2                     1.0.8                h93a5062_5    conda-forge
c-ares                    1.28.1               h93a5062_0    conda-forge
ca-certificates           2024.2.2             hf0a4a13_0    conda-forge
cachecontrol              0.14.0             pyhd8ed1ab_0    conda-forge
cachecontrol-with-filecache 0.14.0             pyhd8ed1ab_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py312h8e38eb3_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
chex                      0.1.86             pyhd8ed1ab_0    conda-forge
cleo                      2.1.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
contextlib2               21.6.0             pyhd8ed1ab_0    conda-forge
contourpy                 1.2.1           py312h0fef576_0    conda-forge
crashtest                 0.4.1              pyhd8ed1ab_0    conda-forge
croniter                  1.4.1              pyhd8ed1ab_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
dateutils                 0.6.12                     py_0    conda-forge
dav1d                     1.2.1                hb547adb_0    conda-forge
deepdiff                  7.0.1              pyhd8ed1ab_0    conda-forge
distlib                   0.3.8              pyhd8ed1ab_0    conda-forge
dnspython                 2.6.1              pyhd8ed1ab_1    conda-forge
docrep                    0.3.2              pyh44b312d_0    conda-forge
dulwich                   0.21.7          py312he37b823_0    conda-forge
email-validator           2.1.1              pyhd8ed1ab_0    conda-forge
email_validator           2.1.1                hd8ed1ab_0    conda-forge
et_xmlfile                1.1.0              pyhd8ed1ab_0    conda-forge
etils                     1.8.0              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
fastapi                   0.111.0            pyhd8ed1ab_0    conda-forge
fastapi-cli               0.0.4              pyhd8ed1ab_0    conda-forge
filelock                  3.14.0             pyhd8ed1ab_0    conda-forge
flax                      0.8.3              pyhd8ed1ab_0    conda-forge
fonttools                 4.51.0          py312he37b823_0    conda-forge
freetype                  2.12.1               hadb7bae_2    conda-forge
fsspec                    2024.5.0           pyhff2d567_0    conda-forge
get-annotations           0.1.2              pyhd8ed1ab_0    conda-forge
gmp                       6.3.0                hebf3989_1    conda-forge
gmpy2                     2.1.5           py312hfa9fade_1    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
h5py                      3.11.0          nompi_py312h903599c_101    conda-forge
hdf5                      1.14.3          nompi_h751145d_101    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  1.0.5              pyhd8ed1ab_0    conda-forge
httpx                     0.27.0             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
idna                      3.7                pyhd8ed1ab_0    conda-forge
importlib-metadata        7.1.0              pyha770c72_0    conda-forge
importlib_metadata        7.1.0                hd8ed1ab_0    conda-forge
importlib_resources       6.4.0              pyhd8ed1ab_0    conda-forge
inquirer                  3.1.4              pyhd8ed1ab_0    conda-forge
itsdangerous              2.2.0              pyhd8ed1ab_0    conda-forge
jaraco.classes            3.4.0              pyhd8ed1ab_1    conda-forge
jax                       0.4.27             pyhd8ed1ab_0    conda-forge
jaxlib                    0.4.23          cpu_py312h43c41d3_2    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jmespath                  1.0.1              pyhd8ed1ab_0    conda-forge
joblib                    1.4.2              pyhd8ed1ab_0    conda-forge
keyring                   24.3.1          py312h81bd7bf_0    conda-forge
kiwisolver                1.4.5           py312h389731b_1    conda-forge
krb5                      1.21.2               h92f50d5_0    conda-forge
lcms2                     2.16                 ha0e7c42_0    conda-forge
legacy-api-wrap           1.4                pyhd8ed1ab_1    conda-forge
lerc                      4.0.0                h9a09cb3_0    conda-forge
libabseil                 20240116.2      cxx17_hebf3989_0    conda-forge
libaec                    1.1.3                hebf3989_0    conda-forge
libavif16                 1.0.4                hb18d617_4    conda-forge
libblas                   3.9.0           22_osxarm64_openblas    conda-forge
libbrotlicommon           1.1.0                hb547adb_1    conda-forge
libbrotlidec              1.1.0                hb547adb_1    conda-forge
libbrotlienc              1.1.0                hb547adb_1    conda-forge
libcblas                  3.9.0           22_osxarm64_openblas    conda-forge
libcurl                   8.8.0                h7b6f9a7_0    conda-forge
libcxx                    17.0.6               h5f092b4_0    conda-forge
libdeflate                1.20                 h93a5062_0    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.6.2                hebf3989_0    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libgfortran               5.0.0           13_2_0_hd922786_3    conda-forge
libgfortran5              13.2.0               hf226fd6_3    conda-forge
libgrpc                   1.62.2               h9c18a4f_0    conda-forge
libhwloc                  2.10.0          default_h7685b71_1001    conda-forge
libiconv                  1.17                 h0d3ecfb_2    conda-forge
libjpeg-turbo             3.0.0                hb547adb_1    conda-forge
liblapack                 3.9.0           22_osxarm64_openblas    conda-forge
libllvm14                 14.0.6               hd1a9a77_4    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libopenblas               0.3.27          openmp_h6c19121_0    conda-forge
libpng                    1.6.43               h091b4b1_0    conda-forge
libprotobuf               4.25.3               hbfab5d5_0    conda-forge
libre2-11                 2023.09.01           h7b2c953_2    conda-forge
libsqlite                 3.45.3               h091b4b1_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libtiff                   4.6.0                h07db509_3    conda-forge
libtorch                  2.3.0           cpu_generic_hed86ac2_0    conda-forge
libuv                     1.48.0               h93a5062_0    conda-forge
libwebp-base              1.4.0                h93a5062_0    conda-forge
libxcb                    1.15                 hf346824_0    conda-forge
libxml2                   2.12.7               ha661575_0    conda-forge
libzlib                   1.2.13               h53f4e23_5    conda-forge
lightning                 2.0.9.post0        pyhd8ed1ab_0    conda-forge
lightning-cloud           0.5.69             pyhd8ed1ab_0    conda-forge
lightning-utilities       0.11.2             pyhd8ed1ab_0    conda-forge
llvm-openmp               18.1.5               hde57baf_0    conda-forge
llvmlite                  0.42.0          py312h17030e7_1    conda-forge
loompy                    3.0.6                      py_0    conda-forge
lz4-c                     1.9.4                hb7217d7_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5           py312he37b823_0    conda-forge
matplotlib-base           3.8.4           py312h4479663_2    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
ml-collections            0.1.1              pyhd8ed1ab_0    conda-forge
ml_dtypes                 0.4.0           py312h8ae5369_1    conda-forge
more-itertools            10.2.0             pyhd8ed1ab_0    conda-forge
mpc                       1.3.1                h91ba8db_0    conda-forge
mpfr                      4.2.1                h41d338b_1    conda-forge
mpmath                    1.3.0              pyhd8ed1ab_0    conda-forge
msgpack-python            1.0.8           py312h157fec4_0    conda-forge
mudata                    0.2.3              pyhd8ed1ab_0    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
natsort                   8.4.0              pyhd8ed1ab_0    conda-forge
ncurses                   6.5                  hb89a1cb_0    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
networkx                  3.3                pyhd8ed1ab_1    conda-forge
nomkl                     1.0                  h5ca1d4c_0    conda-forge
numba                     0.59.1          py312hbaff935_0    conda-forge
numpy                     1.26.4          py312h8442bc7_0    conda-forge
numpy_groupies            0.11.1             pyhd8ed1ab_0    conda-forge
numpyro                   0.15.0             pyhd8ed1ab_0    conda-forge
openjpeg                  2.5.2                h9f1df11_0    conda-forge
openpyxl                  3.1.2           py312h02f2b3b_1    conda-forge
openssl                   3.3.0                hfb2fe0b_2    conda-forge
opt-einsum                3.3.0                hd8ed1ab_2    conda-forge
opt_einsum                3.3.0              pyhc1e730c_2    conda-forge
optax                     0.2.2              pyhd8ed1ab_0    conda-forge
orbax-checkpoint          0.4.4              pyhd8ed1ab_0    conda-forge
ordered-set               4.1.0              pyhd8ed1ab_0    conda-forge
orjson                    3.10.3          py312h552d48e_0    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pandas                    2.2.2           py312h8ae5369_1    conda-forge
patsy                     0.5.6              pyhd8ed1ab_0    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
pillow                    10.3.0          py312h8a801b1_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pkginfo                   1.10.0             pyhd8ed1ab_0    conda-forge
platformdirs              4.2.2              pyhd8ed1ab_0    conda-forge
poetry                    1.8.3           osx_pyh534df25_1    conda-forge
poetry-core               1.9.0              pyhd8ed1ab_0    conda-forge
poetry-plugin-export      1.8.0              pyhd8ed1ab_0    conda-forge
protobuf                  4.25.3          py312h1a610f3_0    conda-forge
psutil                    5.9.8           py312he37b823_0    conda-forge
pthread-stubs             0.4               h27ca646_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pydantic                  1.10.14         py312he37b823_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyjwt                     2.8.0              pyhd8ed1ab_1    conda-forge
pynndescent               0.5.12             pyhca7485f_0    conda-forge
pyparsing                 3.1.2              pyhd8ed1ab_0    conda-forge
pyproject_hooks           1.1.0              pyhd8ed1ab_0    conda-forge
pyro-api                  0.1.2              pyhd8ed1ab_0    conda-forge
pyro-ppl                  1.9.0              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.12.3          h4a7b5fc_0_cpython    conda-forge
python-build              1.2.1              pyhd8ed1ab_0    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-editor             1.0.4                      py_0    conda-forge
python-fastjsonschema     2.19.1             pyhd8ed1ab_0    conda-forge
python-installer          0.7.0              pyhd8ed1ab_0    conda-forge
python-multipart          0.0.9              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python_abi                3.12                    4_cp312    conda-forge
pytorch                   2.3.0           cpu_generic_py312h16b77fb_0    conda-forge
pytorch-lightning         2.2.2              pyhd8ed1ab_0    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1           py312h02f2b3b_1    conda-forge
rapidfuzz                 3.9.1           py312h5c2e7bc_0    conda-forge
rav1e                     0.6.6                h69fbcac_2    conda-forge
re2                       2023.09.01           h4cba328_2    conda-forge
readchar                  4.1.0              pyhd8ed1ab_0    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
requests                  2.32.2             pyhd8ed1ab_0    conda-forge
requests-toolbelt         1.0.0              pyhd8ed1ab_0    conda-forge
rich                      13.7.1             pyhd8ed1ab_0    conda-forge
s3transfer                0.10.1             pyhd8ed1ab_0    conda-forge
scanpy                    1.10.1             pyhd8ed1ab_0    conda-forge
scikit-learn              1.5.0           py312h1b546db_0    conda-forge
scipy                     1.13.0          py312h14ffa8f_1    conda-forge
scvelo                    0.2.5              pyhd8ed1ab_0    conda-forge
scvi-tools                1.1.2              pyhd8ed1ab_0    conda-forge
seaborn                   0.13.2               hd8ed1ab_2    conda-forge
seaborn-base              0.13.2             pyhd8ed1ab_2    conda-forge
session-info              1.0.0              pyhd8ed1ab_0    conda-forge
setuptools                70.0.0             pyhd8ed1ab_0    conda-forge
shellingham               1.5.4              pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sleef                     3.5.1                h156473d_2    conda-forge
snappy                    1.2.0                hd04f947_1    conda-forge
sniffio                   1.3.1              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
sparse                    0.15.3             pyhd8ed1ab_0    conda-forge
starlette                 0.37.2             pyhd8ed1ab_0    conda-forge
starsessions              1.3.0              pyhd8ed1ab_0    conda-forge
statsmodels               0.14.2          py312hbebd99a_0    conda-forge
stdlib-list               0.10.0             pyhd8ed1ab_0    conda-forge
svt-av1                   2.1.0                h7bae524_0    conda-forge
sympy                     1.12            pypyh9d50eac_103    conda-forge
tbb                       2021.12.0            h420ef59_1    conda-forge
tensorstore               0.1.59          py312h38f0887_0    conda-forge
threadpoolctl             3.5.0              pyhc1e730c_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomlkit                   0.12.5             pyha770c72_0    conda-forge
toolz                     0.12.1             pyhd8ed1ab_0    conda-forge
torchmetrics              1.4.0.post0        pyhd8ed1ab_0    conda-forge
tqdm                      4.66.4             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
trove-classifiers         2024.5.17          pyhd8ed1ab_0    conda-forge
typer                     0.12.3             pyhd8ed1ab_0    conda-forge
typer-slim                0.12.3             pyhd8ed1ab_0    conda-forge
typer-slim-standard       0.12.3               hd8ed1ab_0    conda-forge
types-python-dateutil     2.9.0.20240316     pyhd8ed1ab_0    conda-forge
typing-extensions         4.11.0               hd8ed1ab_0    conda-forge
typing_extensions         4.11.0             pyha770c72_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
ujson                     5.10.0          py312h5c2e7bc_0    conda-forge
umap-learn                0.5.5           py312h81bd7bf_1    conda-forge
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
uvicorn                   0.29.0          py312h81bd7bf_0    conda-forge
virtualenv                20.26.2            pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
websocket-client          1.8.0              pyhd8ed1ab_0    conda-forge
websockets                12.0            py312he37b823_0    conda-forge
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
xarray                    2024.5.0           pyhd8ed1ab_0    conda-forge
xattr                     1.1.0           py312he37b823_0    conda-forge
xlrd                      1.2.0              pyh9f0ad1d_1    conda-forge
xorg-libxau               1.0.11               hb547adb_0    conda-forge
xorg-libxdmcp             1.1.3                h27ca646_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zstd                      1.5.6                hb46c0d2_0    conda-forge

But the error

(scvelo_0.2.5) kevin@Kevins-Laptop ~ % python
Python 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:35:20) [Clang 16.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import scvelo as scv

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/kevin/miniforge3/envs/scvelo_0.2.5/lib/python3.12/site-packages/scvelo/__init__.py", line 5, in <module>
    from scvelo import datasets, logging
  File "/Users/kevin/miniforge3/envs/scvelo_0.2.5/lib/python3.12/site-packages/scvelo/datasets/__init__.py", line 1, in <module>
    from ._datasets import (
  File "/Users/kevin/miniforge3/envs/scvelo_0.2.5/lib/python3.12/site-packages/scvelo/datasets/_datasets.py", line 10, in <module>
    from scvelo.core import cleanup
  File "/Users/kevin/miniforge3/envs/scvelo_0.2.5/lib/python3.12/site-packages/scvelo/core/__init__.py", line 1, in <module>
    from ._anndata import (
  File "/Users/kevin/miniforge3/envs/scvelo_0.2.5/lib/python3.12/site-packages/scvelo/core/_anndata.py", line 15, in <module>
    from scvelo import logging as logg
  File "/Users/kevin/miniforge3/envs/scvelo_0.2.5/lib/python3.12/site-packages/scvelo/logging.py", line 12, in <module>
    from scvelo import settings
  File "/Users/kevin/miniforge3/envs/scvelo_0.2.5/lib/python3.12/site-packages/scvelo/settings.py", line 91, in <module>
    warnings.filterwarnings("ignore", category=cbook.mplDeprecation)
                                               ^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'matplotlib.cbook' has no attribute 'mplDeprecation'
kevinrue commented 5 months ago

Note to self

With:

name: scvelo_0.2.5
channels:
  - conda-forge
  - bioconda
dependencies:
  - scvelo=0.2.5
  - matplotlib=3.7.3

I successfully ran

import scvelo as scv

Source; https://stackoverflow.com/questions/77128061/ydata-profiling-profilereport-attributeerror-module-matplotlib-cbook-has-no

kevinrue commented 5 months ago

Good news is I'm making progress with scvelo actually running.

Bad news is I'm running into https://github.com/theislab/scvelo/issues/1212 Which requires a bit of custom logic in the velociraptor R code to implement the (temporary 🤞 ) workaround described in https://github.com/theislab/scvelo/issues/1212#issuecomment-1979120563

kevinrue commented 5 months ago 
  0%|          | 0/200 [00:00<?, ?cells/s]
Error in py_call_impl(callable, call_args$unnamed, call_args$named) : 
  ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 2 dimensions. The detected shape was (1, 4) + inhomogeneous part.
Run `reticulate::py_last_error()` for details.
In addition: Warning message:
In fun(...) : setting moments

😭

kevinrue commented 5 months ago

success b9717cacbf70f840dcb1ad9377579eeb5e73230a

using 

# scvelo 0.2.5
# and dependencies pinned to fix various issues linked below
.scvelo_dependencies <- c(
    "scvelo=0.2.5",
    "matplotlib=3.7.3", # https://stackoverflow.com/questions/77128061/ydata-profiling-profilereport-attributeerror-module-matplotlib-cbook-has-no
    "tqdm=4.66.4", # required for progress bar
    "ipywidgets=8.1.2", # required for progress bar
    "jupyterlab=4.2.0", # required for progress bar
    "numpy=1.23.1" # https://github.com/OpenTalker/video-retalking/issues/35
)

now the tedious task of pinning all the other dependencies to make my current M1 environment 100% reproducible on other M1's

> basilisk::listPackages("/Users/kevin/Library/Caches/org.R-project.R/R/basilisk/1.14.3/velociraptor/1.15.0/env")
                                     full                   package
1                          absl-py==2.1.0                   absl-py
2                         anndata==0.10.7                   anndata
3                  annotated-types==0.7.0           annotated-types
4                            anyio==4.3.0                     anyio
5                          appnope==0.1.4                   appnope
6                     argon2-cffi==23.1.0               argon2-cffi
7            argon2-cffi-bindings==21.2.0      argon2-cffi-bindings
8                   array_api_compat==1.6          array_api_compat
9                            arrow==1.3.0                     arrow
10                       asttokens==2.4.1                 asttokens
11                       async-lru==2.0.4                 async-lru
12                          attrs==23.2.0                     attrs
13                          Babel==2.14.0                     Babel
14                 beautifulsoup4==4.12.3            beautifulsoup4
15                          bleach==6.1.0                    bleach
16                        blessed==1.19.1                   blessed
17                        boto3==1.34.109                     boto3
18                     botocore==1.34.110                  botocore
19                          Brotli==1.1.0                    Brotli
20                           build==1.2.1                     build
21                   CacheControl==0.14.0              CacheControl
22                 cached-property==1.5.2           cached-property
23                      certifi==2024.2.2                   certifi
24                           cffi==1.16.0                      cffi
25              charset-normalizer==3.3.2        charset-normalizer
26                           chex==0.1.83                      chex
27                            cleo==2.1.0                      cleo
28                           click==8.1.7                     click
29                        colorama==0.4.6                  colorama
30                            comm==0.2.2                      comm
31                    contextlib2==21.6.0               contextlib2
32                       contourpy==1.2.1                 contourpy
33                       crashtest==0.4.1                 crashtest
34                       croniter==1.3.15                  croniter
35                         cycler==0.12.1                    cycler
36                      dateutils==0.6.12                 dateutils
37                         debugpy==1.8.1                   debugpy
38                       decorator==5.1.1                 decorator
39                        deepdiff==7.0.1                  deepdiff
40                      defusedxml==0.7.1                defusedxml
41                         distlib==0.3.8                   distlib
42                         dm-tree==0.1.8                   dm-tree
43                       dnspython==2.6.1                 dnspython
44                          docrep==0.3.2                    docrep
45                        dulwich==0.21.7                   dulwich
46                 email_validator==2.1.1           email_validator
47                       entrypoints==0.4               entrypoints
48                      et-xmlfile==1.1.0                et-xmlfile
49                           etils==1.6.0                     etils
50                  exceptiongroup==1.2.0            exceptiongroup
51                       executing==2.0.1                 executing
52                       fastapi==0.111.0                   fastapi
53                     fastapi-cli==0.0.4               fastapi-cli
54                 fastjsonschema==2.19.1            fastjsonschema
55                       filelock==3.14.0                  filelock
56                            flax==0.8.3                      flax
57                      fonttools==4.51.0                 fonttools
58                            fqdn==1.5.1                      fqdn
59                      fsspec==2023.12.2                    fsspec
60                 get-annotations==0.1.2           get-annotations
61                           gmpy2==2.1.5                     gmpy2
62                            h11==0.14.0                       h11
63                              h2==4.1.0                        h2
64                           h5py==3.11.0                      h5py
65                           hpack==4.0.0                     hpack
66                        httpcore==1.0.5                  httpcore
67                          httpx==0.27.0                     httpx
68                      hyperframe==6.0.1                hyperframe
69                              idna==3.7                      idna
70              importlib_metadata==7.1.0        importlib_metadata
71             importlib_resources==6.4.0       importlib_resources
72                        inquirer==3.1.4                  inquirer
73                       installer==0.7.0                 installer
74                      ipykernel==6.29.3                 ipykernel
75                        ipython==8.18.1                   ipython
76                      ipywidgets==8.1.2                ipywidgets
77                   isoduration==20.11.0               isoduration
78                    itsdangerous==2.2.0              itsdangerous
79                  jaraco.classes==3.4.0            jaraco.classes
80                            jax==0.4.27                       jax
81             jaxlib==0.4.23.dev20240522                    jaxlib
82                           jedi==0.19.1                      jedi
83                          Jinja2==3.1.4                    Jinja2
84                        jmespath==1.0.1                  jmespath
85                          joblib==1.4.2                    joblib
86                          json5==0.9.25                     json5
87                       jsonpointer==2.4               jsonpointer
88                     jsonschema==4.22.0                jsonschema
89   jsonschema-specifications==2023.12.1 jsonschema-specifications
90                  jupyter_client==8.6.1            jupyter_client
91                    jupyter_core==5.7.2              jupyter_core
92                 jupyter-events==0.10.0            jupyter-events
93                     jupyter-lsp==2.2.5               jupyter-lsp
94                 jupyter_server==2.14.0            jupyter_server
95        jupyter_server_terminals==0.5.3  jupyter_server_terminals
96                      jupyterlab==4.2.0                jupyterlab
97             jupyterlab_pygments==0.3.0       jupyterlab_pygments
98              jupyterlab_server==2.27.1         jupyterlab_server
99             jupyterlab_widgets==3.0.10        jupyterlab_widgets
100                       keyring==24.3.1                   keyring
101                     kiwisolver==1.4.5                kiwisolver
102                  legacy-api-wrap==1.4           legacy-api-wrap
103                      lightning==2.0.4                 lightning
104               lightning_cloud==0.5.69           lightning_cloud
105           lightning-utilities==0.11.2       lightning-utilities
106                      llvmlite==0.42.0                  llvmlite
107                         loompy==3.0.6                    loompy
108                 markdown-it-py==3.0.0            markdown-it-py
109                     MarkupSafe==2.1.5                MarkupSafe
110                     matplotlib==3.7.3                matplotlib
111              matplotlib-inline==0.1.7         matplotlib-inline
112                          mdurl==0.1.2                     mdurl
113                        mistune==3.0.2                   mistune
114                 ml-collections==0.1.1            ml-collections
115                      ml-dtypes==0.4.0                 ml-dtypes
116                more-itertools==10.2.0            more-itertools
117                         mpmath==1.3.0                    mpmath
118                        msgpack==1.0.8                   msgpack
119                         mudata==0.2.3                    mudata
120               multipledispatch==0.6.0          multipledispatch
121                        munkres==1.1.4                   munkres
122                        natsort==8.4.0                   natsort
123                      nbclient==0.10.0                  nbclient
124                     nbconvert==7.16.4                 nbconvert
125                      nbformat==5.10.4                  nbformat
126                   nest_asyncio==1.6.0              nest_asyncio
127                       networkx==3.2.1                  networkx
128                  notebook_shim==0.2.4             notebook_shim
129                         numba==0.59.1                     numba
130                         numpy==1.23.1                     numpy
131                numpy-groupies==0.11.1            numpy-groupies
132                       numpyro==0.15.0                   numpyro
133                       openpyxl==3.1.2                  openpyxl
134                     opt-einsum==3.3.0                opt-einsum
135                          optax==0.2.2                     optax
136               orbax-checkpoint==0.4.4          orbax-checkpoint
137                    ordered-set==4.1.0               ordered-set
138                        orjson==3.10.3                    orjson
139                      overrides==7.7.0                 overrides
140                       packaging==24.0                 packaging
141                         pandas==2.2.2                    pandas
142                  pandocfilters==1.5.0             pandocfilters
143                          parso==0.8.4                     parso
144                          patsy==0.5.6                     patsy
145                        pexpect==4.9.0                   pexpect
146                    pickleshare==0.7.5               pickleshare
147                        pillow==10.3.0                    pillow
148                             pip==24.0                       pip
149                       pkginfo==1.10.0                   pkginfo
150          pkgutil_resolve_name==1.3.10      pkgutil_resolve_name
151                   platformdirs==4.2.2              platformdirs
152                         poetry==1.8.3                    poetry
153                    poetry-core==1.9.0               poetry-core
154           poetry-plugin-export==1.8.0      poetry-plugin-export
155             prometheus_client==0.20.0         prometheus_client
156                prompt-toolkit==3.0.42            prompt-toolkit
157                      protobuf==4.25.3                  protobuf
158                         psutil==5.9.8                    psutil
159                     ptyprocess==0.7.0                ptyprocess
160                      pure-eval==0.2.2                 pure-eval
161                       pycparser==2.22                 pycparser
162                       pydantic==2.7.1                  pydantic
163                 pydantic_core==2.18.2             pydantic_core
164                      Pygments==2.18.0                  Pygments
165                          PyJWT==2.8.0                     PyJWT
166                   pynndescent==0.5.12               pynndescent
167                     pyobjc-core==10.2               pyobjc-core
168          pyobjc-framework-Cocoa==10.2    pyobjc-framework-Cocoa
169                      pyparsing==3.1.2                 pyparsing
170                pyproject_hooks==1.1.0           pyproject_hooks
171                       pyro-api==0.1.2                  pyro-api
172               pyro-ppl==1.9.0+f02dfb9                  pyro-ppl
173                        PySocks==1.7.1                   PySocks
174                python-dateutil==2.9.0           python-dateutil
175                  python-editor==1.0.4             python-editor
176             python-json-logger==2.0.7        python-json-logger
177               python-multipart==0.0.9          python-multipart
178              pytorch-lightning==2.2.2         pytorch-lightning
179                          pytz==2024.1                      pytz
180                         PyYAML==6.0.1                    PyYAML
181                         pyzmq==26.0.3                     pyzmq
182                      rapidfuzz==3.9.1                 rapidfuzz
183                  readchar==4.1.0.dev3                  readchar
184                   referencing==0.35.1               referencing
185                      requests==2.32.2                  requests
186              requests-toolbelt==1.0.0         requests-toolbelt
187              rfc3339-validator==0.1.4         rfc3339-validator
188              rfc3986-validator==0.1.1         rfc3986-validator
189                          rich==13.7.1                      rich
190                       rpds-py==0.18.1                   rpds-py
191                    s3transfer==0.10.1                s3transfer
192                        scanpy==1.10.1                    scanpy
193                   scikit-learn==1.5.0              scikit-learn
194                         scipy==1.13.0                     scipy
195                         scvelo==0.2.5                    scvelo
196                     scvi-tools==1.1.2                scvi-tools
197                       seaborn==0.13.2                   seaborn
198                     Send2Trash==1.8.3                Send2Trash
199                   session-info==1.0.0              session-info
200                    setuptools==70.0.0                setuptools
201                    shellingham==1.5.4               shellingham
202                           six==1.16.0                       six
203                        sniffio==1.3.1                   sniffio
204                        soupsieve==2.5                 soupsieve
205                        sparse==0.15.3                    sparse
206                     stack-data==0.6.2                stack-data
207                     starlette==0.37.2                 starlette
208                   starsessions==1.3.0              starsessions
209                   statsmodels==0.14.2               statsmodels
210                   stdlib-list==0.10.0               stdlib-list
211                           sympy==1.12                     sympy
212                   tensorstore==0.1.59               tensorstore
213                     terminado==0.18.1                 terminado
214                  threadpoolctl==3.5.0             threadpoolctl
215                       tinycss2==1.3.0                  tinycss2
216                          tomli==2.0.1                     tomli
217                       tomlkit==0.12.5                   tomlkit
218                         toolz==0.12.1                     toolz
219                          torch==2.3.0                     torch
220             torchmetrics==1.4.0.post0              torchmetrics
221                          tornado==6.4                   tornado
222                          tqdm==4.66.4                      tqdm
223                     traitlets==5.14.3                 traitlets
224          trove-classifiers==2024.5.17         trove-classifiers
225                         typer==0.12.3                     typer
226                    typer-slim==0.12.3                typer-slim
227 types-python-dateutil==2.9.0.20240316     types-python-dateutil
228             typing_extensions==4.11.0         typing_extensions
229                   typing-utils==0.1.0              typing-utils
230                        tzdata==2024.1                    tzdata
231                         ujson==5.10.0                     ujson
232                     umap-learn==0.5.5                umap-learn
233                  unicodedata2==15.1.0              unicodedata2
234                   uri-template==1.3.0              uri-template
235                      urllib3==1.26.18                   urllib3
236                       uvicorn==0.29.0                   uvicorn
237                   virtualenv==20.26.2                virtualenv
238                       wcwidth==0.2.13                   wcwidth
239                       webcolors==1.13                 webcolors
240                   webencodings==0.5.1              webencodings
241               websocket-client==1.8.0          websocket-client
242                    websockets==11.0.3                websockets
243                         wheel==0.43.0                     wheel
244            widgetsnbextension==4.0.10        widgetsnbextension
245                      xarray==2024.5.0                    xarray
246                          xattr==1.1.0                     xattr
247                           xlrd==1.2.0                      xlrd
248                          zipp==3.17.0                      zipp
kevinrue commented 5 months ago

Anyone willing to try out the updated versions on branch kra-63 ? I've manually (and successfully) tested it on three different machines: macOS M1, Windows and Linux. Otherwise I'll merge in a week and see what happens in the wild :)

csoneson commented 5 months ago

I tried it on an M1 - the kra-63 branch installs, builds the environment and runs without error (I noticed that the results/pseudotimes look a bit different from the previous version for the example data - do you see the same?)

kevinrue commented 5 months ago

indeed this is my side by side comparison (left: branch kra-63 v 1.15.1; right: latest github vignette v1.9.2)

image

seems to me it's more an issue with the t-SNE that looks more 'scattered' potentially forcing the vectors to stretch further (I don't know how to formally check if that's the case)

kevinrue commented 5 months ago

for a more up to date comparison, on the right, v1.14.0 (current bioc release)

image
kevinrue commented 5 months ago

problem solved in https://github.com/kevinrue/velociraptor/pull/71/commits/975bd9e106a06c4d7046daf742f1abd40298233c

essentially, the move from scvelo to scanpy for neighbor calculation changed the default value of n_neighbors from 30 to 15

rather than forcing the old default value, the vignette documents the change and educates users to explicitly set the value to 30 themselves. (new default is thus 15)

kevinrue commented 5 months ago

For the record, after fix, the t-SNE looks like this

image