search

conda / conda-lock

Lightweight lockfile for conda environments
https://conda.github.io/conda-lock/
Other
495 stars 104 forks source link

Pip dependency won't resolve on osx-64 - RuntimeError: Unable to find installation candidates for ortools #403

Closed DJohnston-Veritas closed 1 year ago

DJohnston-Veritas commented 1 year ago

Checklist

What happened?

This environment.yml fails to resolve with conda-lock. Conda env create is successful on an Intel Mac.

name: test
platforms:
  - osx-64
channels:
  - conda-forge
dependencies:
  - conda-lock
  - pip:
    - ortools
Locking dependencies for ['osx-64']...
INFO:conda_lock.conda_solver:osx-64 using specs ['conda-lock', 'pip *']
Traceback (most recent call last):
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/bin/conda-lock", line 10, in <module>
    sys.exit(main())
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 1353, in lock
    lock_func(
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 1083, in run_lock
    make_lock_files(
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 408, in make_lock_files
    lock_content = lock_content | create_lockfile_from_spec(
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 801, in create_lockfile_from_spec
    deps = _solve_for_arch(
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/conda_lock/conda_lock.py", line 737, in _solve_for_arch
    pip_deps = solve_pypi(
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/conda_lock/pypi_solver.py", line 290, in solve_pypi
    link = chooser.choose_for(op.package)
  File "~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test/lib/python3.10/site-packages/conda_lock/_vendor/poetry/installation/chooser.py", line 72, in choose_for
    raise RuntimeError(
RuntimeError: Unable to find installation candidates for ortools (9.6.2534)

If I remove 'conda-forge', then I get an error message identical to #396

Conda Info

active environment : test
    active env location : ~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test
            shell level : 1
       user config file : ~/.condarc
 populated config files : ~/.conda/condarc
          conda version : 22.9.0
    conda-build version : not installed
         python version : 3.10.8.final.0
       virtual packages : __osx=10.16=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : ~/.pyenv/versions/miniconda3-3.10-22.11.1-1  (writable)
      conda av data dir : ~/.pyenv/versions/miniconda3-3.10-22.11.1-1/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : ~/.pyenv/versions/miniconda3-3.10-22.11.1-1/pkgs
                          ~/.conda/pkgs
       envs directories : ~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs
                          ~/.conda/envs
               platform : osx-64
             user-agent : conda/22.9.0 requests/2.28.1 CPython/3.10.8 Darwin/22.4.0 OSX/10.16
                UID:GID : 502:20
             netrc file : ~/.netrc
           offline mode : False

Conda Config

==> ~/.conda/condarc <==
auto_update_conda: False
auto_activate_base: False
safety_checks: enabled
extra_safety_checks: True
ssl_verify: ~/.ssh/ca_bundle.pem

Conda list

# packages in environment at ~/.pyenv/versions/miniconda3-3.10-22.11.1-1/envs/test:
#
# Name                    Version                   Build  Channel
absl-py                   1.4.0                    pypi_0    pypi
appdirs                   1.4.4              pyhd3eb1b0_0    defaults
brotlipy                  0.7.0           py310hca72f7f_1002    defaults
bzip2                     1.0.8                h1de35cc_0    defaults
ca-certificates           2022.12.7            h033912b_0    conda-forge
cachecontrol              0.12.11         py310hecd8cb5_1    defaults
cachecontrol-with-filecache 0.12.11         py310hecd8cb5_1    defaults
cachy                     0.3.0              pyhd3eb1b0_0    defaults
certifi                   2022.12.7          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1          py310h6c40b1e_3    defaults
charset-normalizer        2.0.4              pyhd3eb1b0_0    defaults
click                     8.0.4           py310hecd8cb5_0    defaults
click-default-group       1.2.2           py310hecd8cb5_0    defaults
clikit                    0.6.2                      py_0    defaults
conda-lock                1.4.0              pyhd8ed1ab_2    conda-forge
crashtest                 0.3.1              pyhd3eb1b0_1    defaults
cryptography              39.0.1          py310hf6deb26_0    defaults
distlib                   0.3.6           py310hecd8cb5_0    defaults
ensureconda               1.4.3           py310hecd8cb5_0    defaults
filelock                  3.9.0           py310hecd8cb5_0    defaults
html5lib                  1.1                pyhd3eb1b0_0    defaults
idna                      3.4             py310hecd8cb5_0    defaults
importlib-metadata        6.0.0           py310hecd8cb5_0    defaults
importlib_metadata        6.0.0                hd3eb1b0_0    defaults
jaraco.classes            3.2.1              pyhd3eb1b0_0    defaults
jinja2                    3.1.2           py310hecd8cb5_0    defaults
keyring                   23.13.1         py310hecd8cb5_0    defaults
libcxx                    14.0.6               h9765a3e_0    defaults
libffi                    3.4.2                hecd8cb5_6    defaults
lockfile                  0.12.2          py310hecd8cb5_0    defaults
markupsafe                2.1.1           py310hca72f7f_0    defaults
more-itertools            8.12.0             pyhd3eb1b0_0    defaults
msgpack-python            1.0.3           py310haf03e11_0    defaults
ncurses                   6.4                  hcec6c5f_0    defaults
numpy                     1.24.3                   pypi_0    pypi
openssl                   1.1.1t               hfd90126_0    conda-forge
ortools                   9.6.2534                 pypi_0    pypi
packaging                 23.0            py310hecd8cb5_0    defaults
pastel                    0.2.1                      py_0    defaults
pip                       23.1.2             pyhd8ed1ab_0    conda-forge
pkginfo                   1.9.6           py310hecd8cb5_0    defaults
platformdirs              2.5.2           py310hecd8cb5_0    defaults
protobuf                  4.22.3                   pypi_0    pypi
pycparser                 2.21               pyhd3eb1b0_0    defaults
pydantic                  1.10.2          py310hca72f7f_0    defaults
pylev                     1.3.0                      py_0    defaults
pyopenssl                 23.0.0          py310hecd8cb5_0    defaults
pysocks                   1.7.1           py310hecd8cb5_0    defaults
python                    3.10.11              h218abb5_2    defaults
pyyaml                    6.0             py310h6c40b1e_1    defaults
readline                  8.2                  hca72f7f_0    defaults
requests                  2.29.0          py310hecd8cb5_0    defaults
ruamel.yaml               0.17.21         py310hca72f7f_0    defaults
ruamel.yaml.clib          0.2.6           py310hca72f7f_1    defaults
scipy                     1.10.1                   pypi_0    pypi
setuptools                66.0.0          py310hecd8cb5_0    defaults
six                       1.16.0             pyhd3eb1b0_1    defaults
sqlite                    3.41.2               h6c40b1e_0    defaults
tk                        8.6.12               h5d9f67b_0    defaults
tomli                     2.0.1           py310hecd8cb5_0    defaults
tomlkit                   0.11.1          py310hecd8cb5_0    defaults
toolz                     0.12.0          py310hecd8cb5_0    defaults
typing-extensions         4.5.0           py310hecd8cb5_0    defaults
typing_extensions         4.5.0           py310hecd8cb5_0    defaults
tzdata                    2023c                h04d1e81_0    defaults
urllib3                   1.26.15         py310hecd8cb5_0    defaults
virtualenv                20.17.1         py310hecd8cb5_0    defaults
webencodings              0.5.1           py310hecd8cb5_1    defaults
wheel                     0.38.4          py310hecd8cb5_0    defaults
xz                        5.4.2                h6c40b1e_0    defaults
yaml                      0.2.5                haf1e3a3_0    defaults
zipp                      3.11.0          py310hecd8cb5_0    defaults
zlib                      1.2.13               h4dc903c_0    defaults

Additional Context

Username has been redacted by replacing /Users/username with ~

Miniconda 22.11.1 is installed using pyenv, then downgraded to 22.9 (22.9 is not in the pyenv installable list). I've also tried this with the latest Miniconda installed by Homebrew and get the same error.

maresb commented 1 year ago

Do you have an installation of Mamba? If so, could you please try mamba --version and see if the version of Conda it finds is also 22.9? (One hypothesis is that conda-lock is calling Mamba which uses some other version of Conda.)

DJohnston-Veritas commented 1 year ago

@maresb I do not use Mamba. In both test cases I did a clean install of Miniconda, then create the environment from the environment.yml file. which mamba does not find anything in the search path.

davidkipstar commented 1 year ago

I ran into the same issue and spent some time in the debugger. Finally I found the source of this issue which origins here since only macosx_10_9_x86_64 is considered. If you change the line to *(f"macosx_10_{version}_x86_64" for version in range(16, 3, -1)), it will find the packages.

If this is a valid fix I can open a PR.

maresb commented 1 year ago

Thanks a lot for putting in the debugger time to track this down, it's much appreciated!!!

I don't understand this part of the code so well. It was masterminded by @jvansanten. Perhaps he has some idea about how we should maintain this section?

Any thoughts @mariusvniekerk?