search

conda / conda-build

Commands and tools for building conda packages
https://docs.conda.io/projects/conda-build/
Other
379 stars 420 forks source link

`PackagesNotFound` not converted to `DependencyNeedsBuildingError` #5256

Open mbargull opened 5 months ago

mbargull commented 5 months ago

Checklist

What happened?

For some unsatisfiable dependencies reported via conda-libmamba-solver, conda-build=24.3.0 does not capture those properly in a DependencyNeedsBuildingError.

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=902575&view=logs&j=80cb4bd3-7e0a-50cc-19d9-637946946f1a&t=046bad2a-bca6-54c7-3f51-632649b9ec75&l=1018 :

[...]
Encountered problems while solving:
  - nothing provides pandoc >=1.12.1,<3.0.0 needed by nbconvert-5.6.0-py27_1

Could not solve for environment specs
The following packages are incompatible
├─ nbconvert is installable with the potential options
│  ├─ nbconvert 5.6.1 would require
│  │  └─ jinja2 >=2.4,<3a0 , which requires
│  │     └─ markupsafe [>=0.23,<2 |>=0.23,<2.1 ] with the potential options
│  │        ├─ markupsafe [1.1.0|1.1.1] would require
│  │        │  └─ python >=2.7,<2.8.0a0  with the potential options
│  │        │     ├─ python 2.7.15, which can be installed;
│  │        │     └─ python [2.7.15|3.6.7|3.7.1] would require
│  │        │        └─ openssl >=1.0.2p,<1.0.3a , which does not exist (perhaps a missing channel);
│  │        ├─ markupsafe [1.1.0|1.1.1] would require
│  │        │  └─ python >=3.6,<3.7.0a0  with the potential options
│  │        │     ├─ python [2.7.15|3.6.7|3.7.1], which cannot be installed (as previously explained);
│  │        │     └─ python [3.6.10|3.6.11|...|3.6.9], which can be installed;
│  │        ├─ markupsafe [1.1.0|1.1.1] would require
│  │        │  └─ python >=3.7,<3.8.0a0  with the potential options
│  │        │     ├─ python [2.7.15|3.6.7|3.7.1], which cannot be installed (as previously explained);
│  │        │     └─ python [3.7.1|3.7.10|...|3.7.9], which can be installed;
│  │        ├─ markupsafe [1.1.1|2.0.1] would require
│  │        │  └─ python_abi 3.10.* *_cp310, which can be installed;
│  │        ├─ markupsafe [1.1.1|2.0.0|2.0.1] would require
│  │        │  └─ python_abi 3.6.* *_cp36m, which can be installed;
│  │        ├─ markupsafe [1.1.1|2.0.0|2.0.1] would require
│  │        │  └─ python_abi 3.6 *_pypy36_pp73, which can be installed;
│  │        ├─ markupsafe [1.1.1|2.0.0|2.0.1] would require
│  │        │  └─ python_abi 3.7 *_pypy37_pp73, which can be installed;
│  │        ├─ markupsafe [1.1.1|2.0.0|2.0.1] would require
│  │        │  └─ python_abi 3.7.* *_cp37m, which can be installed;
│  │        ├─ markupsafe [1.1.1|2.0.0|2.0.1] would require
│  │        │  └─ python_abi 3.8.* *_cp38, which can be installed;
│  │        ├─ markupsafe 1.1.1 would require
│  │        │  └─ python >=3.8,<3.9.0a0 , which can be installed;
│  │        └─ markupsafe [1.1.1|2.0.0|2.0.1] would require
│  │           └─ python_abi 3.9.* *_cp39, which can be installed;
│  ├─ nbconvert [5.5.0|5.6.0|...|6.0.7] would require
│  │  └─ pandoc >=1.12.1,<3.0.0 , which does not exist (perhaps a missing channel);
│  ├─ nbconvert 6.0.7 would require
│  │  └─ python_abi 3.6.* *_cp36m, which can be installed;
│  ├─ nbconvert 6.0.7 would require
│  │  └─ python_abi 3.6 *_pypy36_pp73, which can be installed;
│  ├─ nbconvert [6.0.7|6.1.0|...|6.4.5] would require
│  │  └─ python_abi 3.7.* *_cp37m, which can be installed;
│  ├─ nbconvert [6.0.7|6.1.0|...|6.4.5] would require
│  │  └─ python_abi 3.7 *_pypy37_pp73, which can be installed;
│  ├─ nbconvert [6.0.7|6.1.0|...|6.4.5] would require
│  │  └─ python_abi 3.8.* *_cp38, which can be installed;
│  ├─ nbconvert [6.0.7|6.1.0|...|6.4.5] would require
│  │  └─ python_abi 3.9.* *_cp39, which can be installed;
│  ├─ nbconvert [6.3.0|6.4.0|...|6.4.5] would require
│  │  └─ python_abi 3.10.* *_cp310, which can be installed;
│  ├─ nbconvert 5.4.1 would require
│  │  └─ pandoc >=1.12.1,<2.0.0 , which does not exist (perhaps a missing channel);
│  └─ nbconvert [6.4.5|6.5.0|...|7.9.2] would require
│     └─ nbconvert-pandoc [6.4.5 pyhd8ed1ab_2|6.5.0 pyhd8ed1ab_0|...|7.9.2 pyhd8ed1ab_0], which requires
│        └─ pandoc, which does not exist (perhaps a missing channel);
└─ python >=3.11,<3.12.0a0,>=3.11,<3.12.0a0 *_cpython is not installable because it conflicts with any installable versions previously reported.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/bin/conda-build", line 11, in <module>
    sys.exit(execute())
  File "/opt/conda/lib/python3.10/site-packages/conda_build/cli/main_build.py", line 581, in execute
    api.build(
  File "/opt/conda/lib/python3.10/site-packages/conda_build/api.py", line 250, in build
    return build_tree(
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 3762, in build_tree
    packages_from_this = build(
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2537, in build
    create_build_envs(top_level_pkg, notest)
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2368, in create_build_envs
    missing_deps = {MatchSpec(pkg).name for pkg in e.packages} - {
  File "/opt/conda/lib/python3.10/site-packages/conda_build/build.py", line 2368, in <setcomp>
    missing_deps = {MatchSpec(pkg).name for pkg in e.packages} - {
  File "/opt/conda/lib/python3.10/site-packages/conda/models/match_spec.py", line 47, in __call__
    parsed = _parse_spec_str(spec_arg)
  File "/opt/conda/lib/python3.10/site-packages/conda/models/match_spec.py", line 707, in _parse_spec_str
    raise InvalidMatchSpec(
conda.exceptions.InvalidMatchSpec: Invalid spec '>=1.12.1,<3.0.0': no package name found in '>=1.12.1,<3.0.0'

Conda Info

active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /home/conda/.condarc
 populated config files : /opt/conda/.condarc
                          /home/conda/.condarc
          conda version : 24.1.2
    conda-build version : 24.3.0
         python version : 3.10.14.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=broadwell
                          __conda=24.1.2=0
                          __glibc=2.17=0
                          __linux=6.5.0=0
                          __unix=0=0
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/conda/feedstock_root/build_artifacts/pkg_cache
                          /opt/conda/pkgs
       envs directories : /opt/conda/envs
                          /home/conda/.conda/envs
               platform : linux-64
             user-agent : conda/24.1.2 requests/2.31.0 CPython/3.10.14 Linux/6.5.0-1016-azure centos/7.9.2009 glibc/2.17 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.7
                UID:GID : 1001:1001
             netrc file : None
           offline mode : False

Conda Config

==> /opt/conda/.condarc <==
add_pip_as_python_dependency: False
auto_update_conda: False
aggressive_update_packages:
  - ca-certificates
  - certifi
channel_priority: strict
channels:
  - conda-forge
show_channel_urls: True
conda_build:
  pkg_format: 2
  zstd_compression_level: 19

==> /home/conda/.condarc <==
pkgs_dirs:
  - /home/conda/feedstock_root/build_artifacts/pkg_cache
  - /opt/conda/pkgs
solver: libmamba
conda-build:
  root-dir: /home/conda/feedstock_root/build_artifacts

==> envvars <==
bld_path: /home/conda/feedstock_root/build_artifacts

Conda list

# packages in environment at /opt/conda:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
anaconda-client           1.12.3             pyhd8ed1ab_0    conda-forge
anaconda-project          0.11.1             pyhd8ed1ab_0    conda-forge
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
attrs                     23.2.0             pyh71513ae_0    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
boltons                   23.1.1             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py310hc6cd4ac_1    conda-forge
bzip2                     1.0.8                hd590300_5    conda-forge
c-ares                    1.27.0               hd590300_0    conda-forge
ca-certificates           2024.2.2             hbcca054_0    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py310h2fee648_0    conda-forge
chardet                   5.2.0           py310hff52083_1    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     24.1.2          py310hff52083_0    conda-forge
conda-build               24.3.0          py310hff52083_0    conda-forge
conda-env                 2.6.0                         1    conda-forge
conda-forge-ci-setup      4.3.3           py310h7a2d8a0_100    conda-forge
conda-forge-metadata      0.8.0              pyhd8ed1ab_0    conda-forge
conda-index               0.4.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     24.1.0             pyhd8ed1ab_0    conda-forge
conda-oci-mirror          0.1.0              pyhd8ed1ab_0    conda-forge
conda-pack                0.7.1              pyhd8ed1ab_0    conda-forge
conda-package-handling    2.2.0              pyh38be061_0    conda-forge
conda-package-streaming   0.9.0              pyhd8ed1ab_0    conda-forge
cryptography              42.0.5          py310h75e40e8_0    conda-forge
curl                      8.6.0                hca28451_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
deprecated                1.2.14             pyh1a96a4e_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
filelock                  3.13.1             pyhd8ed1ab_0    conda-forge
fmt                       10.2.1               h00ab1b0_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
git                       2.44.0          pl5321h709897a_0    conda-forge
icu                       73.2                 h59595ed_0    conda-forge
idna                      3.6                pyhd8ed1ab_0    conda-forge
importlib_resources       6.4.0              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.3              pyhd8ed1ab_0    conda-forge
jq                        1.7.1                hd590300_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               2.4             py310hff52083_3    conda-forge
jsonschema                4.21.1             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py310hff52083_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.2               h659d440_0    conda-forge
lcms2                     2.16                 hb7c19ff_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libarchive                3.7.2                h2aa1ff5_1    conda-forge
libcurl                   8.6.0                hca28451_0    conda-forge
libdeflate                1.20                 hd590300_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libexpat                  2.6.2                h59595ed_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_5    conda-forge
libgomp                   13.2.0               h807b86a_5    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
liblief                   0.12.3               h27087fc_0    conda-forge
libmamba                  1.5.7                had39da4_0    conda-forge
libmambapy                1.5.7           py310h39ff949_0    conda-forge
libnghttp2                1.58.0               h47da74e_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libpng                    1.6.43               h2797004_0    conda-forge
libsolv                   0.7.28               hfc55251_0    conda-forge
libsqlite                 3.45.2               h2797004_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_5    conda-forge
libtiff                   4.6.0                h1dd3fc0_3    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libwebp-base              1.3.2                hd590300_0    conda-forge
libxcb                    1.15                 h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.12.6               h232c23b_0    conda-forge
libzlib                   1.2.13               hd590300_5    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
mamba                     1.5.7           py310h51d5547_0    conda-forge
markupsafe                2.1.5           py310h2372a71_0    conda-forge
menuinst                  2.0.2           py310hff52083_0    conda-forge
more-itertools            10.2.0             pyhd8ed1ab_0    conda-forge
nbformat                  5.10.3             pyhd8ed1ab_0    conda-forge
ncurses                   6.4.20240210         h59595ed_0    conda-forge
oniguruma                 6.9.9                hd590300_0    conda-forge
openjpeg                  2.5.2                h488ebb8_0    conda-forge
openssl                   3.2.1                hd590300_1    conda-forge
oras-py                   0.1.14             pyhd8ed1ab_0    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
patch                     2.7.6             h7f98852_1002    conda-forge
patchelf                  0.17.2               h58526e2_0    conda-forge
pcre2                     10.43                hcad00b1_0    conda-forge
perl                      5.32.1          7_hd590300_perl5    conda-forge
pillow                    10.2.0          py310h01dd4db_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pkginfo                   1.10.0             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
pluggy                    1.4.0              pyhd8ed1ab_0    conda-forge
psutil                    5.9.8           py310h2372a71_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
py-lief                   0.12.3          py310hd8f1fbe_0    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6           py310h2372a71_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pyopenssl                 24.0.0             pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.10.14         hd12c33a_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.19.1             pyhd8ed1ab_0    conda-forge
python-libarchive-c       5.1             py310hff52083_0    conda-forge
python_abi                3.10                    4_cp310    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1           py310h2372a71_1    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.34.0             pyhd8ed1ab_0    conda-forge
reproc                    14.2.4.post0         hd590300_1    conda-forge
reproc-cpp                14.2.4.post0         h59595ed_1    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
requests-toolbelt         1.0.0              pyhd8ed1ab_0    conda-forge
ripgrep                   14.1.0               he8a937b_0    conda-forge
rpds-py                   0.18.0          py310hcb5633a_0    conda-forge
ruamel.yaml               0.18.6          py310h2372a71_0    conda-forge
ruamel.yaml.clib          0.2.8           py310h2372a71_0    conda-forge
ruamel_yaml               0.15.80         py310h2372a71_1009    conda-forge
setuptools                69.2.0             pyhd8ed1ab_0    conda-forge
shyaml                    0.6.2              pyhd3deb0d_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
su-exec                   0.2               h166bdaf_1003    conda-forge
tini                      0.19.0               h166bdaf_1    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toolz                     0.12.1             pyhd8ed1ab_0    conda-forge
tornado                   6.4             py310h2372a71_0    conda-forge
tqdm                      4.66.2             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.2             pyhd8ed1ab_0    conda-forge
truststore                0.8.0              pyhd8ed1ab_0    conda-forge
typing-extensions         4.10.0               hd8ed1ab_0    conda-forge
typing_extensions         4.10.0             pyha770c72_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
wheel                     0.43.0             pyhd8ed1ab_0    conda-forge
wrapt                     1.16.0          py310h2372a71_0    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yaml-cpp                  0.8.0                h59595ed_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zstandard                 0.22.0          py310h1275a96_0    conda-forge
zstd                      1.5.5                hfc55251_0    conda-forge

Additional Context

Commit https://github.com/conda/conda-build/pull/5203/commits/7acbbf200be198e978e08391436695db14298a05 caused this regression.

mbargull commented 5 months ago

Turns out this is not a regression in conda-build=24.3 but rather conda-build=24.1 did not exercise code paths which resulted in PackagesNotFoundError which is why this has not been shown in 24.1 (can be reproduced in conda-build<24.1, though). See discussion at https://github.com/conda/conda-build/pull/5257#discussion_r1538117516.