search

conda / conda-libmamba-solver

The libmamba based solver for conda.
https://conda.github.io/conda-libmamba-solver/
Other
203 stars 24 forks source link

Conda 23.10.0 fails to find and install local files with the new solver libmamba on MacOS (M1) #398

Open huydhn opened 11 months ago

huydhn commented 11 months ago

Checklist

What happened?

As part of PyTorch release workflow, we build conda release for PyTorch and other libraries like TorchAudio. As part of the process, we run smoke test to install the conda build artifacts locally with the following commands conda install -c pytorch-nightly -c file:///Users/huydo/Downloads/distr distr::torchaudio, and it starts to fail with the latest 23.10.0 conda release:

Channels:
 - pytorch-nightly
 - file:///Users/huydo/Downloads/distr
 - defaults
 - distr
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  - distr::torchaudio

Current channels:

  - https://conda.anaconda.org/pytorch-nightly
  - file:///Users/huydo/Downloads/distr
  - defaults
  - https://conda.anaconda.org/distr

To search for alternate channels that may provide the conda package you're
looking for, navigate to

    https://anaconda.org

and use the search bar at the top of the page.

Here are some observation:

Package Plan

environment location: /Users/huydo/miniconda3/envs/py3.9

added / updated specs:

The following packages will be downloaded:

package                    |            build
---------------------------|-----------------
gmpy2-2.1.2                |   py39h8c48613_0         144 KB
mpc-1.1.0                  |       h8c48613_1         104 KB
mpfr-4.0.2                 |       h695f6f0_1         423 KB
mpmath-1.3.0               |   py39hca03da5_0         845 KB
networkx-3.1               |   py39hca03da5_0         2.7 MB
pytorch-2.2.0.dev20231129  |          py3.9_0        52.0 MB  pytorch-nightly
sympy-1.11.1               |   py39hca03da5_0        11.7 MB
torchaudio-2.2.0.dev20231129|         py39_cpu         4.8 MB  file:///Users/huydo/Downloads/distr
------------------------------------------------------------
                                       Total:        72.7 MB

* Instead of checking for the local file, the solver seems to prefer the remote https://conda.anaconda.org/distr, which coincidentally has the same name `distr`
* The issue only manifests on MacOS M1.  We run similar tests on Linux and Windows with the same conda version but don't encounter any issues on these platforms

### Conda Info

```shell
active environment : py3.9
    active env location : /Users/huydo/miniconda3/envs/py3.9
            shell level : 4
       user config file : /Users/huydo/.condarc
 populated config files :
          conda version : 23.10.0
    conda-build version : not installed
         python version : 3.9.17.final.0
       virtual packages : __archspec=1=m1
                          __osx=14.1.1=0
                          __unix=0=0
       base environment : /Users/huydo/miniconda3  (writable)
      conda av data dir : /Users/huydo/miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/huydo/miniconda3/pkgs
                          /Users/huydo/.conda/pkgs
       envs directories : /Users/huydo/miniconda3/envs
                          /Users/huydo/.conda/envs
               platform : osx-arm64
             user-agent : conda/23.10.0 requests/2.29.0 CPython/3.9.17 Darwin/23.1.0 OSX/14.1.1 solver/libmamba conda-libmamba-solver/23.11.1 libmambapy/1.5.3
                UID:GID : 501:20
             netrc file : /Users/huydo/.netrc
           offline mode : False

Conda Config

N/A

Conda list

WARNING conda.core.prefix_data:_load_site_packages(338): Problem reading non-conda package record at lib/python3.9/site-packages/numpy-1.26.1.dist-info/RECORD. Please verify that you still need this, and if so, that this is still installed correctly. Reinstalling this package may help.
# packages in environment at /Users/huydo/miniconda3/envs/py3.9:
#
# Name                    Version                   Build  Channel
absl-py                   1.4.0                    pypi_0    pypi
accelerate                0.16.0                   pypi_0    pypi
aiohttp                   3.8.4                    pypi_0    pypi
aiosignal                 1.3.1                    pypi_0    pypi
anaconda-client           1.12.0           py39hca03da5_0    defaults
antlr4-python3-runtime    4.9.3                    pypi_0    pypi
anyio                     3.6.2                    pypi_0    pypi
appdirs                   1.4.4                    pypi_0    pypi
appnope                   0.1.3                    pypi_0    pypi
archspec                  0.2.1              pyhd3eb1b0_0    defaults
argon2-cffi               21.3.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow                     1.2.3                    pypi_0    pypi
astor                     0.8.1                    pypi_0    pypi
asttokens                 2.1.0                    pypi_0    pypi
async-timeout             4.0.2                    pypi_0    pypi
attrs                     23.1.0                   pypi_0    pypi
audioread                 3.0.0                    pypi_0    pypi
autogluon-common          0.7.0                    pypi_0    pypi
autogluon-core            0.7.0                    pypi_0    pypi
autogluon-features        0.7.0                    pypi_0    pypi
autogluon-tabular         0.7.0                    pypi_0    pypi
awscli                    1.29.83                  pypi_0    pypi
awscurl                   0.26                     pypi_0    pypi
backcall                  0.2.0                    pypi_0    pypi
beautifulsoup4            4.12.2           py39hca03da5_0    defaults
black                     23.3.0                   pypi_0    pypi
blas                      1.0                    openblas    defaults
bleach                    6.0.0                    pypi_0    pypi
blis                      0.7.9                    pypi_0    pypi
boltons                   23.0.0           py39hca03da5_0    defaults
boost-cpp                 1.73.0              h1a28f6b_12    defaults
boto3                     1.19.12                  pypi_0    pypi
botocore                  1.31.83                  pypi_0    pypi
bottleneck                1.3.5            py39heec5a64_0    defaults
brotli                    1.0.9                h1a28f6b_7    defaults
brotli-bin                1.0.9                h1a28f6b_7    defaults
brotli-python             1.0.9            py39hc377ac9_7    defaults
brotlipy                  0.7.0           py39h1a28f6b_1002    defaults
bzip2                     1.0.8                h620ffc9_4    defaults
c-ares                    1.19.1               h80987f9_0    defaults
ca-certificates           2023.08.22           hca03da5_0    defaults
cachetools                5.3.0                    pypi_0    pypi
cairo                     1.16.0               h302bd0f_5    defaults
catalogue                 2.0.8                    pypi_0    pypi
catboost                  1.1.1                    pypi_0    pypi
cctools                   949.0.1             hc179dcd_25    defaults
cctools_osx-arm64         949.0.1             h332cad3_25    defaults
certifi                   2023.7.22                pypi_0    pypi
cffi                      1.15.1           py39h22df2f2_0    defaults
chardet                   4.0.0           py39hca03da5_1003    defaults
charset-normalizer        3.3.2                    pypi_0    pypi
clang-format              12.0.1                   pypi_0    pypi
click                     8.1.3                    pypi_0    pypi
cloudpickle               2.2.1                    pypi_0    pypi
clyent                    1.2.2            py39hca03da5_1    defaults
cmake                     3.22.1               hae769c0_0    defaults
cmakelang                 0.6.13                   pypi_0    pypi
cmakelint                 1.4.1                    pypi_0    pypi
colorama                  0.4.4                    pypi_0    pypi
comm                      0.1.3                    pypi_0    pypi
commonmark                0.9.1                    pypi_0    pypi
conda                     23.10.0          py39hca03da5_0    defaults
conda-build               3.25.0           py39hca03da5_0    defaults
conda-index               0.2.3            py39hca03da5_0    defaults
conda-libmamba-solver     23.11.1          py39hca03da5_0    defaults
conda-package-handling    2.2.0            py39hca03da5_0    defaults
conda-package-streaming   0.9.0            py39hca03da5_0    defaults
confection                0.0.4                    pypi_0    pypi
configargparse            1.5.3                    pypi_0    pypi
configparser              5.3.0                    pypi_0    pypi
contourpy                 1.0.5            py39h525c30c_0    defaults
coremltools               5.0b5                    pypi_0    pypi
coverage                  7.3.1                    pypi_0    pypi
cryptography              41.0.2           py39h6204c90_0    defaults
cycler                    0.11.0             pyhd3eb1b0_0    defaults
cymem                     2.0.7                    pypi_0    pypi
cython                    0.29.34                  pypi_0    pypi
dataclasses               0.8                pyh6d0b6a4_7    defaults
datasets                  2.12.0                   pypi_0    pypi
debugpy                   1.6.7                    pypi_0    pypi
decorator                 5.1.1                    pypi_0    pypi
defusedxml                0.7.1              pyhd3eb1b0_0    defaults
deprecated                1.2.13                   pypi_0    pypi
dill                      0.3.6                    pypi_0    pypi
distlib                   0.3.7                    pypi_0    pypi
docutils                  0.16                     pypi_0    pypi
ecdsa                     0.14.1                   pypi_0    pypi
elastic-transport         8.4.0                    pypi_0    pypi
evaluate                  0.3.0                    pypi_0    pypi
exceptiongroup            1.0.4                    pypi_0    pypi
execnet                   1.9.0                    pypi_0    pypi
executing                 1.2.0                    pypi_0    pypi
executorch                0.1.0                    pypi_0    pypi
expat                     2.4.9                hc377ac9_0    defaults
expecttest                0.1.6                    pypi_0    pypi
fairscale                 0.4.13                   pypi_0    pypi
fastcore                  1.5.29                   pypi_0    pypi
fastdownload              0.0.7                    pypi_0    pypi
fastjsonschema            2.16.3                   pypi_0    pypi
fastprogress              1.0.3                    pypi_0    pypi
filelock                  3.13.1                   pypi_0    pypi
flake8                    6.0.0                    pypi_0    pypi
flake8-breakpoint         1.1.0                    pypi_0    pypi
flake8-bugbear            23.3.23                  pypi_0    pypi
flake8-comprehensions     3.12.0                   pypi_0    pypi
flake8-executable         2.1.3                    pypi_0    pypi
flake8-logging-format     0.9.0                    pypi_0    pypi
flake8-plugin-utils       1.3.3                    pypi_0    pypi
flake8-pyi                23.3.1                   pypi_0    pypi
flake8-simplify           0.19.3                   pypi_0    pypi
flatbuffers               23.5.26                  pypi_0    pypi
fmt                       9.1.0                h48ca7d4_0    defaults
font-ttf-dejavu-sans-mono 2.37                 hd3eb1b0_0    defaults
font-ttf-inconsolata      2.001                hcb22688_0    defaults
font-ttf-source-code-pro  2.030                hd3eb1b0_0    defaults
font-ttf-ubuntu           0.83                 h8b1ccd4_0    defaults
fontconfig                2.14.1               hee714a5_2    defaults
fonts-anaconda            1                    h8fa9717_0    defaults
fonts-conda-ecosystem     1                    hd3eb1b0_0    defaults
fonttools                 4.25.0             pyhd3eb1b0_0    defaults
fqdn                      1.5.1                    pypi_0    pypi
freetype                  2.12.1               h1192e45_0    defaults
fribidi                   1.0.10               h1a28f6b_0    defaults
frozenlist                1.3.3                    pypi_0    pypi
fsspec                    2023.10.0                pypi_0    pypi
future                    0.18.2                   pypi_0    pypi
gdk-pixbuf                2.42.10              h80987f9_0    defaults
gdown                     4.7.1                    pypi_0    pypi
geojson                   2.5.0                    pypi_0    pypi
gettext                   0.21.0               h13f89a0_1    defaults
ghstack                   0.8.0                    pypi_0    pypi
giflib                    5.2.1                h80987f9_3    defaults
glib                      2.69.1               h514c7bf_2    defaults
glob2                     0.7                pyhd3eb1b0_0    defaults
gluonts                   0.12.7                   pypi_0    pypi
gmp                       6.2.1                hc377ac9_3    defaults
gmpy2                     2.1.2            py39h8c48613_0    defaults
google-auth               2.17.3                   pypi_0    pypi
google-auth-oauthlib      1.0.0                    pypi_0    pypi
gprof2dot                 2022.7.29                pypi_0    pypi
graphite2                 1.3.14               hc377ac9_1    defaults
graphviz                  2.50.0               hf331ead_1    defaults
grpcio                    1.54.0                   pypi_0    pypi
gts                       0.7.6                hde733a8_3    defaults
harfbuzz                  4.3.0                he9eebac_1    defaults
huggingface-hub           0.17.0rc0                pypi_0    pypi
hyperopt                  0.2.7                    pypi_0    pypi
hypothesis                6.56.4                   pypi_0    pypi
icu                       68.1                 hc377ac9_0    defaults
idna                      2.10                     pypi_0    pypi
imageio                   2.28.1                   pypi_0    pypi
importlib-metadata        6.5.0                    pypi_0    pypi
importlib_resources       5.2.0              pyhd3eb1b0_1    defaults
iniconfig                 1.1.1                    pypi_0    pypi
ipykernel                 6.22.0                   pypi_0    pypi
ipython                   8.6.0                    pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                8.0.6                    pypi_0    pypi
isoduration               20.11.0                  pypi_0    pypi
jaraco-classes            3.2.3                    pypi_0    pypi
jedi                      0.18.1                   pypi_0    pypi
jinja2                    3.1.2            py39hca03da5_0    defaults
jmespath                  0.10.0                   pypi_0    pypi
joblib                    1.2.0                    pypi_0    pypi
jpeg                      9e                   h80987f9_1    defaults
jsonpatch                 1.32               pyhd3eb1b0_0    defaults
jsonpointer               2.3                      pypi_0    pypi
jsonschema                4.17.3           py39hca03da5_0    defaults
junitparser               2.1.1                    pypi_0    pypi
jupyter                   1.0.0                    pypi_0    pypi
jupyter-client            8.2.0                    pypi_0    pypi
jupyter-console           6.6.3                    pypi_0    pypi
jupyter-events            0.6.3                    pypi_0    pypi
jupyter-server            2.5.0                    pypi_0    pypi
jupyter-server-terminals  0.4.4                    pypi_0    pypi
jupyter_core              5.3.0            py39hca03da5_0    defaults
jupyterlab-pygments       0.2.2                    pypi_0    pypi
jupyterlab-widgets        3.0.7                    pypi_0    pypi
keyring                   23.13.1                  pypi_0    pypi
kiwisolver                1.4.4            py39h313beb8_0    defaults
krb5                      1.20.1               hf3e1bf2_1    defaults
langcodes                 3.3.0                    pypi_0    pypi
lazy-loader               0.2                      pypi_0    pypi
lcms2                     2.12                 hba8e193_0    defaults
ld64                      530                 hb29bf3f_25    defaults
ld64_osx-arm64            530                 h001ce53_25    defaults
ldid                      2.1.5                h20b2a84_3    defaults
lerc                      3.0                  hc377ac9_0    defaults
libarchive                3.6.2                h62fee54_2    defaults
libboost                  1.73.0              h49e8a49_12    defaults
libbrotlicommon           1.0.9                h1a28f6b_7    defaults
libbrotlidec              1.0.9                h1a28f6b_7    defaults
libbrotlienc              1.0.9                h1a28f6b_7    defaults
libcst                    1.1.0                    pypi_0    pypi
libcurl                   8.4.0                h3e2b118_0    defaults
libcxx                    14.0.6               h848a8c0_0    defaults
libdeflate                1.17                 h80987f9_0    defaults
libedit                   3.1.20221030         h80987f9_0    defaults
libev                     4.33                 h1a28f6b_1    defaults
libffi                    3.4.4                hca03da5_0    defaults
libgd                     2.3.3                h313beb8_2    defaults
libgfortran               5.0.0           11_3_0_hca03da5_28    defaults
libgfortran5              11.3.0              h009349e_28    defaults
libiconv                  1.16                 h1a28f6b_2    defaults
liblief                   0.12.3               h313beb8_0    defaults
libllvm14                 14.0.6               h7ec7a93_3    defaults
libmamba                  1.5.3                h15e39b3_0    defaults
libmambapy                1.5.3            py39h1c5506f_0    defaults
libnghttp2                1.57.0               h62f6fdd_0    defaults
libopenblas               0.3.21               h269037a_0    defaults
libpng                    1.6.39               h80987f9_0    defaults
librosa                   0.9.2                    pypi_0    pypi
librsvg                   2.54.4               hb3bd4c3_3    defaults
libsolv                   0.7.24               h514c7bf_0    defaults
libssh2                   1.10.0               h02f6b3c_2    defaults
libtiff                   4.5.0                h313beb8_2    defaults
libtool                   2.4.6             h313beb8_1009    defaults
libuv                     1.39.0               h1a28f6b_0    defaults
libwebp                   1.2.4                ha3663a8_1    defaults
libwebp-base              1.2.4                h80987f9_1    defaults
libxml2                   2.10.3               h372ba2a_0    defaults
lightgbm                  3.3.5                    pypi_0    pypi
lightning-utilities       0.8.0                    pypi_0    pypi
lintrunner                0.11.0                   pypi_0    pypi
lintrunner-adapters       0.9.0                    pypi_0    pypi
llvm-openmp               14.0.6               hc6e5704_0    defaults
llvmlite                  0.39.1                   pypi_0    pypi
lxml                      4.9.1                    pypi_0    pypi
lz4-c                     1.9.4                h313beb8_0    defaults
markdown                  3.4.3                    pypi_0    pypi
markdown-it-py            2.2.0                    pypi_0    pypi
markupsafe                2.1.3                    pypi_0    pypi
matplotlib                3.7.0            py39hca03da5_0    defaults
matplotlib-base           3.7.0            py39h78102c4_0    defaults
matplotlib-inline         0.1.6                    pypi_0    pypi
mccabe                    0.7.0                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mistune                   2.0.5                    pypi_0    pypi
model-index               0.1.11                   pypi_0    pypi
more-itertools            9.1.0                    pypi_0    pypi
moreorless                0.4.0                    pypi_0    pypi
mpc                       1.1.0                h8c48613_1    defaults
mpfr                      4.0.2                h695f6f0_1    defaults
mpmath                    1.3.0            py39hca03da5_0    defaults
msgpack                   1.0.5                    pypi_0    pypi
multidict                 6.0.4                    pypi_0    pypi
multiprocess              0.70.14                  pypi_0    pypi
munkres                   1.1.4                      py_0    defaults
murmurhash                1.0.9                    pypi_0    pypi
mypy                      1.6.0                    pypi_0    pypi
mypy-extensions           1.0.0                    pypi_0    pypi
nbclassic                 0.5.6                    pypi_0    pypi
nbclient                  0.7.4                    pypi_0    pypi
nbconvert                 7.3.1                    pypi_0    pypi
nbformat                  5.8.0                    pypi_0    pypi
ncurses                   6.4                  h313beb8_0    defaults
nest-asyncio              1.5.6                    pypi_0    pypi
networkx                  3.2.1                    pypi_0    pypi
ninja                     1.10.2               hca03da5_5    defaults
ninja-base                1.10.2               h525c30c_5    defaults
nlpaug                    1.1.11                   pypi_0    pypi
nltk                      3.8.1                    pypi_0    pypi
notebook                  6.5.4                    pypi_0    pypi
notebook-shim             0.2.3                    pypi_0    pypi
nptyping                  2.4.1                    pypi_0    pypi
nspr                      4.35                 h313beb8_0    defaults
nss                       3.89.1               h313beb8_0    defaults
numba                     0.56.0                   pypi_0    pypi
numexpr                   2.8.4            py39h79ee842_1    defaults
numpy                     1.22.4                   pypi_0    pypi
numpy-base                1.23.3           py39hadd41eb_1    defaults
nvidia-ml-py              11.525.84                pypi_0    pypi
oauthlib                  3.2.2                    pypi_0    pypi
octokitpy                 0.15.0                   pypi_0    pypi
octokitpy-routes          2.5.0                    pypi_0    pypi
omegaconf                 2.2.3                    pypi_0    pypi
openjpeg                  2.3.0                h7a6adac_2    defaults
openmim                   0.3.7                    pypi_0    pypi
opensearch-dsl            2.1.0                    pypi_0    pypi
opensearch-py             2.3.0                    pypi_0    pypi
openssl                   3.0.12               h1a28f6b_0    defaults
opt-einsum                3.3.0                    pypi_0    pypi
optree                    0.10.0                   pypi_0    pypi
ordered-set               4.1.0                    pypi_0    pypi
packaging                 23.1                     pypi_0    pypi
pandas                    1.4.4            py39hc377ac9_0    defaults
pandocfilters             1.5.0                    pypi_0    pypi
pango                     1.50.7               h7271ec9_0    defaults
parameterized             0.9.0                    pypi_0    pypi
parso                     0.8.3                    pypi_0    pypi
patch                     2.7.6             h1a28f6b_1001    defaults
pathspec                  0.10.3                   pypi_0    pypi
pathy                     0.10.1                   pypi_0    pypi
patsy                     0.5.3                    pypi_0    pypi
pcre                      8.45                 hc377ac9_0    defaults
pcre2                     10.42                hb066dcc_0    defaults
pexpect                   4.8.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    10.1.0                   pypi_0    pypi
pip                       22.2.2           py39hca03da5_0    defaults
pixman                    0.40.0               h1a28f6b_0    defaults
pkg-config                0.29.2               h1a28f6b_0    defaults
pkginfo                   1.9.6            py39hca03da5_0    defaults
platformdirs              3.5.0                    pypi_0    pypi
plotly                    5.14.1                   pypi_0    pypi
pluggy                    1.0.0            py39hca03da5_1    defaults
pmdarima                  1.8.5                    pypi_0    pypi
pooch                     1.6.0                    pypi_0    pypi
poppler                   22.12.0              h497017c_0    defaults
poppler-data              0.4.11               hca03da5_1    defaults
preshed                   3.0.8                    pypi_0    pypi
prometheus-client         0.16.0                   pypi_0    pypi
prompt-toolkit            3.0.32                   pypi_0    pypi
protobuf                  3.20.2                   pypi_0    pypi
psutil                    5.9.1                    pypi_0    pypi
ptyprocess                0.7.0                    pypi_0    pypi
pure-eval                 0.2.2                    pypi_0    pypi
py-lief                   0.12.3           py39h313beb8_0    defaults
py4j                      0.10.9.7                 pypi_0    pypi
pyarrow                   12.0.0                   pypi_0    pypi
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.3.0                    pypi_0    pypi
pybind11-abi              4                    hd3eb1b0_1    defaults
pycodestyle               2.10.0                   pypi_0    pypi
pycosat                   0.6.4            py39h1a28f6b_0    defaults
pycparser                 2.21               pyhd3eb1b0_0    defaults
pydantic                  1.10.7                   pypi_0    pypi
pydeprecate               0.3.2                    pypi_0    pypi
pyflakes                  3.0.1                    pypi_0    pypi
pygments                  2.15.0                   pypi_0    pypi
pynvml                    11.4.1                   pypi_0    pypi
pyopenssl                 23.2.0           py39hca03da5_0    defaults
pyparsing                 3.0.9            py39hca03da5_0    defaults
pyrsistent                0.19.3                   pypi_0    pypi
pysocks                   1.7.1            py39hca03da5_0    defaults
pytesseract               0.3.10                   pypi_0    pypi
pytest                    7.3.2                    pypi_0    pypi
pytest-cov                4.1.0                    pypi_0    pypi
pytest-cpp                2.3.0                    pypi_0    pypi
pytest-flakefinder        1.1.0                    pypi_0    pypi
pytest-profiling          1.7.0                    pypi_0    pypi
pytest-repeat             0.9.1                    pypi_0    pypi
pytest-rerunfailures      10.3                     pypi_0    pypi
pytest-resume             0.0.1                    pypi_0    pypi
pytest-shard              0.1.2                    pypi_0    pypi
pytest-xdist              3.3.1                    pypi_0    pypi
python                    3.9.17               hb885b13_0    defaults
python-dateutil           2.8.2              pyhd3eb1b0_0    defaults
python-fastjsonschema     2.16.2           py39hca03da5_0    defaults
python-graphviz           0.20.1                   pypi_0    pypi
python-jose               3.2.0                    pypi_0    pypi
python-json-logger        2.0.7                    pypi_0    pypi
python-libarchive-c       2.9                pyhd3eb1b0_1    defaults
pytorch                   2.2.0.dev20231129         py3.9_0    pytorch-nightly
pytorch-lightning         1.9.5                    pypi_0    pypi
pytorch-metric-learning   1.7.3                    pypi_0    pypi
pytorch-pkg-helpers       0.1.5                    pypi_0    pypi
pytz                      2022.7           py39hca03da5_0    defaults
pywavelets                1.4.1                    pypi_0    pypi
pyyaml                    6.0              py39h80987f9_1    defaults
pyzmq                     25.0.2                   pypi_0    pypi
qtconsole                 5.4.2                    pypi_0    pypi
qtpy                      2.3.1                    pypi_0    pypi
ray                       2.2.0                    pypi_0    pypi
readline                  8.2                  h1a28f6b_0    defaults
readme-renderer           37.3                     pypi_0    pypi
regex                     2022.10.31               pypi_0    pypi
reproc                    14.2.4               hc377ac9_1    defaults
reproc-cpp                14.2.4               hc377ac9_1    defaults
requests                  2.25.1                   pypi_0    pypi
requests-aws4auth         1.2.3                    pypi_0    pypi
requests-oauthlib         1.3.1                    pypi_0    pypi
requests-toolbelt         0.10.1                   pypi_0    pypi
resampy                   0.4.2                    pypi_0    pypi
responses                 0.18.0                   pypi_0    pypi
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986                   2.0.0                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rhash                     1.4.3                h02f6b3c_0    defaults
rich                      10.9.0                   pypi_0    pypi
rockset                   1.0.3                    pypi_0    pypi
rsa                       4.7.2                    pypi_0    pypi
ruamel-yaml               0.17.4                   pypi_0    pypi
ruamel-yaml-clib          0.2.7                    pypi_0    pypi
ruamel.yaml               0.17.21          py39h1a28f6b_0    defaults
ruamel.yaml.clib          0.2.6            py39h1a28f6b_1    defaults
ruamel_yaml               0.15.100         py39h1a28f6b_0    defaults
ruff                      0.1.6                    pypi_0    pypi
s3transfer                0.7.0                    pypi_0    pypi
safetensors               0.3.3                    pypi_0    pypi
scikit-image              0.19.3                   pypi_0    pypi
scikit-learn              1.1.3                    pypi_0    pypi
scipy                     1.10.1                   pypi_0    pypi
send2trash                1.8.2                    pypi_0    pypi
sentencepiece             0.1.99                   pypi_0    pypi
seqeval                   1.2.2                    pypi_0    pypi
setuptools                67.6.1                   pypi_0    pypi
shellcheck-py             0.7.2.1                  pypi_0    pypi
six                       1.16.0             pyhd3eb1b0_1    defaults
sktime                    0.15.1                   pypi_0    pypi
smart-open                6.3.0                    pypi_0    pypi
snakeviz                  2.0.1            py39hca03da5_0    defaults
sniffio                   1.3.0                    pypi_0    pypi
sortedcontainers          2.4.0                    pypi_0    pypi
soundfile                 0.11.0                   pypi_0    pypi
soupsieve                 2.4              py39hca03da5_0    defaults
soxr                      0.3.5                    pypi_0    pypi
spacy                     3.5.2                    pypi_0    pypi
spacy-legacy              3.0.12                   pypi_0    pypi
spacy-loggers             1.0.4                    pypi_0    pypi
sqlite                    3.41.2               h80987f9_0    defaults
srsly                     2.4.6                    pypi_0    pypi
stack-data                0.6.1                    pypi_0    pypi
statsforecast             1.4.0                    pypi_0    pypi
statsmodels               0.13.5                   pypi_0    pypi
stdlibs                   2022.10.9                pypi_0    pypi
sympy                     1.12                     pypi_0    pypi
tabulate                  0.8.10           py39hca03da5_0    defaults
tapi                      1100.0.11            h8754e6a_1    defaults
tbats                     1.1.3                    pypi_0    pypi
tenacity                  8.2.2                    pypi_0    pypi
tensorboard               2.13.0                   pypi_0    pypi
tensorboard-data-server   0.7.0                    pypi_0    pypi
tensorboardx              2.6                      pypi_0    pypi
terminado                 0.17.1                   pypi_0    pypi
text-unidecode            1.3                      pypi_0    pypi
thinc                     8.1.10                   pypi_0    pypi
threadpoolctl             3.1.0                    pypi_0    pypi
tifffile                  2023.4.12                pypi_0    pypi
tiktoken                  0.4.0                    pypi_0    pypi
timm                      0.6.13                   pypi_0    pypi
tinycss2                  1.2.1                    pypi_0    pypi
tk                        8.6.12               hb8d0fd4_0    defaults
tokenizers                0.13.2                   pypi_0    pypi
toml                      0.10.2                   pypi_0    pypi
tomli                     2.0.1            py39hca03da5_0    defaults
tomlkit                   0.11.6                   pypi_0    pypi
toolz                     0.12.0           py39hca03da5_0    defaults
torchaudio                2.2.0.dev20231129        py39_cpu    file:///Users/huydo/Downloads/distr
torchfix                  0.2.0                    pypi_0    pypi
torchmetrics              0.8.2                    pypi_0    pypi
tornado                   6.3.2            py39h80987f9_0    defaults
tqdm                      4.65.0           py39h86d0a89_0    defaults
trailrunner               1.2.1                    pypi_0    pypi
traitlets                 5.9.0                    pypi_0    pypi
transformers              4.31.0                   pypi_0    pypi
twine                     4.0.2                    pypi_0    pypi
typer                     0.7.0                    pypi_0    pypi
types-colorama            0.4.6                    pypi_0    pypi
types-jinja2              2.11.9                   pypi_0    pypi
types-markupsafe          1.1.10                   pypi_0    pypi
types-pkg-resources       0.1.3                    pypi_0    pypi
types-protobuf            3.19.18                  pypi_0    pypi
types-pyyaml              6.0.7                    pypi_0    pypi
types-requests            2.27.25                  pypi_0    pypi
types-six                 1.16.15                  pypi_0    pypi
types-tabulate            0.8.8                    pypi_0    pypi
types-urllib3             1.26.25.4                pypi_0    pypi
typing-extensions         4.8.0                    pypi_0    pypi
typing-inspect            0.8.0                    pypi_0    pypi
typing_extensions         4.3.0            py39hca03da5_0    defaults
tzdata                    2023c                h04d1e81_0    defaults
ufmt                      2.1.0                    pypi_0    pypi
ujson                     5.7.0                    pypi_0    pypi
unittest-xml-reporting    3.2.0                    pypi_0    pypi
uri-template              1.2.0                    pypi_0    pypi
urllib3                   1.26.18          py39hca03da5_0    defaults
urllib3-secure-extra      0.1.0                    pypi_0    pypi
usort                     1.0.6                    pypi_0    pypi
virtualenv                20.24.6                  pypi_0    pypi
wasabi                    1.1.1                    pypi_0    pypi
wcwidth                   0.2.5                    pypi_0    pypi
webcolors                 1.13                     pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.5.1                    pypi_0    pypi
werkzeug                  2.3.4                    pypi_0    pypi
wheel                     0.37.1             pyhd3eb1b0_0    defaults
widgetsnbextension        4.0.7                    pypi_0    pypi
wrapt                     1.15.0                   pypi_0    pypi
xdoctest                  1.1.0                    pypi_0    pypi
xgboost                   1.7.5                    pypi_0    pypi
xxhash                    3.2.0                    pypi_0    pypi
xz                        5.4.2                h80987f9_0    defaults
yaml                      0.2.5                h1a28f6b_0    defaults
yaml-cpp                  0.8.0                h313beb8_0    defaults
yarl                      1.9.2                    pypi_0    pypi
z3-solver                 4.12.2.0                 pypi_0    pypi
zipp                      3.11.0           py39hca03da5_0    defaults
zlib                      1.2.13               h5a0b063_0    defaults
zstandard                 0.19.0           py39h80987f9_0    defaults
zstd                      1.5.5.1                  pypi_0    pypi

Additional Context

No response

jaimergp commented 11 months ago

I think the problem comes from how the package is being specified:

conda install -c pytorch-nightly -c file:///Users/huydo/Downloads/distr distr::torchaudio

This will be understood as the distr channel in anaconda.org, not "a channel named distr in the current list of channels". This might be a bug in conda-libmamba-solver (easily fixable), but I find that behaviour surprising and I would have assumed that the following:

conda install -c pytorch-nightly -c file:///Users/huydo/Downloads/distr file:///Users/huydo/Downloads/distr::torchaudio

is more explicit and accurate. Does it work with this alternative syntax?

jaimergp commented 11 months ago

Ah wait, this is a bit hacky 🤔 In the distr channel description, it says:

Empty channel whose sole purpose of testing local installs. This enables conda install -c /path/to/distr distr::pkg

🤔 Is it because the osx-arm64 was not part of the default subdir collection when it was created?

jaimergp commented 11 months ago

I think you can use custom_channels in .condarc:

# # custom_channels (map: primitive)
# #   A map of key-value pairs where the key is a channel name and the value
# #   is a channel location. Channels defined here override the default
# #   'channel_alias' value. The channel name (key) is not included in the
# #   channel location (value).  For example, to override the location of
# #   the 'conda-forge' channel where the url to repodata is
# #   https://anaconda-repo.dev/packages/conda-forge/linux-64/repodata.json,
# #   add an entry 'conda-forge: https://anaconda-repo.dev/packages'.
# # 
# custom_channels:
#   pkgs/pro: https://repo.anaconda.com

to do something like this:

custom_channels:
  distr: file:///path/to/Downloads/or/parent/directorty
huydhn commented 11 months ago

Let me try with the custom channels config.

On the other hand, we have tried using the full name, i.e. conda install -c pytorch-nightly -c file:///Users/huydo/Downloads/distr file:///Users/huydo/Downloads/distr::torchaudio, but it doesn't work and still refer to the remote distr channel https://github.com/pytorch/test-infra/actions/runs/7035821668/job/19147091795?pr=4767#step:13:152

jaimergp commented 11 months ago

it doesn't work and still refer to the remote distr channel

Ok, thanks! That's useful to know. I'll transfer this over to conda/conda-libmamba-solver.

kenodegard commented 10 months ago

Ran into a similar (same?) issue using the local channel:

🟢 --solver=classic local::package ``` $ CONDA_BLD_PATH=tests/test-recipes conda create -n scratch --solver=classic local::failing_post_link --dry-run Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: /Users/kodegard/.conda/devenv/Darwin/arm64/envs/devenv-3.10-c/envs/scratch added / updated specs: - local::failing_post_link The following packages will be downloaded: package | build ---------------------------|----------------- failing_post_link-1.0.0 | 0 3 KB local ------------------------------------------------------------ Total: 3 KB The following NEW packages will be INSTALLED: failing_post_link Users/kodegard/dev/conda/tests/test-recipes/noarch::failing_post_link-1.0.0-0 DryRunExit: Dry run. Exiting. ```
🔴 --solver=libmamba local::package ``` $ CONDA_BLD_PATH=tests/test-recipes conda create -n scratch --solver=libmamba local::failing_post_link --dry-run Channels: - defaults - local Platform: osx-arm64 Collecting package metadata (repodata.json): done Solving environment: failed PackagesNotFoundError: The following packages are not available from current channels: - local::failing_post_link Current channels: - defaults - local To search for alternate channels that may provide the conda package you're looking for, navigate to https://anaconda.org/ and use the search bar at the top of the page. ```
🟢 --solver=classic -c local package ``` $ CONDA_BLD_PATH=tests/test-recipes conda create -n scratch --solver=classic -c local failing_post_link --dry-run Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: /Users/kodegard/.conda/devenv/Darwin/arm64/envs/devenv-3.10-c/envs/scratch added / updated specs: - failing_post_link The following packages will be downloaded: package | build ---------------------------|----------------- failing_post_link-1.0.0 | 0 3 KB local ------------------------------------------------------------ Total: 3 KB The following NEW packages will be INSTALLED: failing_post_link Users/kodegard/dev/conda/tests/test-recipes/noarch::failing_post_link-1.0.0-0 DryRunExit: Dry run. Exiting. ```
🟢 --solver=libmamba -c local package ``` $ CONDA_BLD_PATH=tests/test-recipes conda create -n scratch --solver=libmamba -c local failing_post_link --dry-run Channels: - local - defaults Platform: osx-arm64 Collecting package metadata (repodata.json): done Solving environment: done ## Package Plan ## environment location: /Users/kodegard/.conda/devenv/Darwin/arm64/envs/devenv-3.10-c/envs/scratch added / updated specs: - failing_post_link The following packages will be downloaded: package | build ---------------------------|----------------- failing_post_link-1.0.0 | 0 3 KB local ------------------------------------------------------------ Total: 3 KB The following NEW packages will be INSTALLED: failing_post_link Users/kodegard/dev/conda/tests/test-recipes/noarch::failing_post_link-1.0.0-0 DryRunExit: Dry run. Exiting. ```
jaimergp commented 10 months ago

I am looking into this now. For the record, mamba does not allow this either. Note we have to symlink the custom CONDA_BLD_PATH into $CONDA_ROOT/conda-bld so it's actually loaded:

(base) test_user@43aaa076cf48:/opt/conda-libmamba-solver-src$ sudo ln -s /opt/conda-libmamba-solver-
src/tests/data/mamba_repo /opt/conda/conda-bld
(base) test_user@43aaa076cf48:/opt/conda-libmamba-solver-src$ mamba install local::test-package

Looking for: ['local::test-package']

conda-forge/linux-aarch64                                   Using cache
conda-forge/noarch                                          Using cache
opt/conda/conda-bld/linux-aarch64 (check zst)      Checked  0.0s
opt/conda/conda-bld/noarch (check zst)             Checked  0.0s
opt/conda/conda-bld/linux-aarch64                   ??.?MB @  ??.?MB/s 0 failed  0.0s
opt/conda/conda-bld/noarch                         586.0 B @ 394.9kB/s  0.0s

Pinned packages:
  - python 3.11.*

Could not solve for environment specs
The following package could not be installed
└─ test-package does not exist (perhaps a typo or a missing channel).
(base) test_user@43aaa076cf48:/opt/conda-libmamba-solver-src$ mamba install -c local test-package

Looking for: ['test-package']

opt/conda/conda-bld/linux-aarch64 (check zst)      Checked  0.0s
opt/conda/conda-bld/noarch (check zst)             Checked  0.0s
conda-forge/linux-aarch64                                   Using cache
conda-forge/noarch                                          Using cache
opt/conda/conda-bld/linux-aarch64                   ??.?MB @  ??.?MB/s 0 failed  0.0s
opt/conda/conda-bld/noarch                         586.0 B @ 535.2kB/s  0.0s

Pinned packages:
  - python 3.11.*

Transaction

  Prefix: /opt/conda

  Updating specs:

   - test-package
   - ca-certificates
   - certifi
   - openssl

  Package         Version  Build  Channel                  Size
─────────────────────────────────────────────────────────────────
  Install:
─────────────────────────────────────────────────────────────────

  + test-package      0.1  0      /opt/conda/conda-bld      6kB

  Summary:

  Install: 1 packages

  Total download: 6kB

─────────────────────────────────────────────────────────────────

Confirm changes: [Y/n] N
jaimergp commented 10 months ago

Submitted https://github.com/mamba-org/mamba/issues/3055

isuruf commented 10 months ago

One thing I noticed is that while following works,

conda install -c file:///home/isuru/distr distr::test-package --solver=classic

the following does not

conda install -c file:///home/isuru/distr2 distr2::test-package --solver=classic

because there is no distr2 channel in anaconda.org. So, even though this is supposed to work, it feels like a hack.

jaimergp commented 10 months ago

Ran into a similar (same?) issue using the local channel:

FWIW conda install -c local local::test-package --solver=libmamba doesn't work either.

isuruf commented 10 months ago

I think we should allow the following

conda install -c local local::test-package --solver=libmamba
conda install -c file:///home/isuru/distr2 file:///home/isuru/distr2::test-package --solver=libmamba

but not allow

conda install -c file:///home/isuru/distr2 distr2::test-package --solver=libmamba

What do you think?

jaimergp commented 10 months ago

Yep, my gut tells me that the last one feels weird and happens to work. Sadly some folks are somehow relying on that (with some effort, though, given the existence of the remote distr channel). As I said earlier, ergonomics like that belong to the local configuration. cc @conda/conda-core for input too.

isuruf commented 10 months ago

conda install -c local local::test-package --solver=libmamba

Fixed in https://github.com/mamba-org/mamba/pull/3056

conda install -c file:///home/isuru/distr2 file:///home/isuru/distr2::test-package --solver=libmamba

Fixed in https://github.com/conda/conda/pull/13418

isuruf commented 9 months ago

I don't think the original issue is fixed. We need https://github.com/conda/conda/pull/13418 for a partial fix.