conda / conda-build

Commands and tools for building conda packages
https://docs.conda.io/projects/conda-build/
Other
381 stars 423 forks source link

Unexpect detection of selectors #5465

Open jjhelmus opened 1 month ago

jjhelmus commented 1 month ago

Checklist

What happened?

Because of the regex used by conda-build, selectors will be detected in locations where users would likely not expect them to be.

Some examples:

package:
  name: example
  version: 1.0.0
test:
  commands:
    - echo "An example of slicing an array is arr[1:None]
      neat right!"

(combining these last two lines or adding a comma results in a working recipe)

and

package:
  name: example
  version: 1.0.0
test:
  commands:
    - echo "I told #user to slice the array with array[1:None] which worked"

Both of these recipes will fail to build because the selector is invalid:

CondaBuildUserError: Invalid selector in meta.yaml line 6:
offending selector:
  [1:None]
exception:
  SyntaxError: invalid syntax (<string>, line 1)

For clarity a selector will be detected on any line where

I suspect this issue has been in conda-build for so long that is cannot be nicely addressed without breaking backwards compatibility. I would be happy to see this issue marked as wont-fix.

Similar issues have been reported in the past but are now closed. For example #4654.

Conda Info

active environment : base
    active env location : /opt/conda
            shell level : 1
       user config file : /root/.condarc
 populated config files :
          conda version : 24.7.1
    conda-build version : 24.7.1
         python version : 3.12.4.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=x86_64
                          __conda=24.7.1=0
                          __glibc=2.36=0
                          __linux=6.6.16=0
                          __unix=0=0
       base environment : /opt/conda  (writable)
      conda av data dir : /opt/conda/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/conda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/conda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/24.7.1 requests/2.32.2 CPython/3.12.4 Linux/6.6.16-linuxkit debian/12.5 glibc/2.36 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8 aau/0.4.4 c/. s/. e/.
                UID:GID : 0:0
             netrc file : None

Conda Config

==> envvars <==
allow_softlinks: False

Conda list

# packages in environment at /opt/conda:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    defaults
_openmp_mutex             5.1                       1_gnu    defaults
anaconda-anon-usage       0.4.4           py312hfc0e8ea_100    defaults
archspec                  0.2.3              pyhd3eb1b0_0    defaults
attrs                     23.1.0          py312h06a4308_0    defaults
beautifulsoup4            4.12.3          py312h06a4308_0    defaults
boltons                   23.0.0          py312h06a4308_0    defaults
brotli-python             1.0.9           py312h6a678d5_8    defaults
bzip2                     1.0.8                h5eee18b_6    defaults
c-ares                    1.19.1               h5eee18b_0    defaults
ca-certificates           2024.7.2             h06a4308_0    defaults
certifi                   2024.7.4        py312h06a4308_0    defaults
cffi                      1.16.0          py312h5eee18b_1    defaults
chardet                   4.0.0           py312h06a4308_1003    defaults
charset-normalizer        2.0.4              pyhd3eb1b0_0    defaults
click                     8.1.7           py312h06a4308_0    defaults
conda                     24.7.1          py312h06a4308_0    defaults
conda-build               24.7.1          py312h06a4308_0    defaults
conda-content-trust       0.2.0           py312h06a4308_1    defaults
conda-index               0.5.0           py312h06a4308_0    defaults
conda-libmamba-solver     24.1.0             pyhd3eb1b0_0    defaults
conda-package-handling    2.3.0           py312h06a4308_0    defaults
conda-package-streaming   0.10.0          py312h06a4308_0    defaults
cryptography              42.0.5          py312hdda0065_1    defaults
distro                    1.9.0           py312h06a4308_0    defaults
expat                     2.6.2                h6a678d5_0    defaults
filelock                  3.13.1          py312h06a4308_0    defaults
fmt                       9.1.0                hdb19cb5_1    defaults
frozendict                2.4.2           py312h06a4308_0    defaults
icu                       73.1                 h6a678d5_0    defaults
idna                      3.7             py312h06a4308_0    defaults
jinja2                    3.1.4           py312h06a4308_0    defaults
jsonpatch                 1.33            py312h06a4308_1    defaults
jsonpointer               2.1                pyhd3eb1b0_0    defaults
jsonschema                4.19.2          py312h06a4308_0    defaults
jsonschema-specifications 2023.7.1        py312h06a4308_0    defaults
krb5                      1.20.1               h143b758_1    defaults
ld_impl_linux-64          2.38                 h1181459_1    defaults
libarchive                3.6.2                h6ac8c49_3    defaults
libcurl                   8.7.1                h251f7ec_0    defaults
libedit                   3.1.20230828         h5eee18b_0    defaults
libev                     4.33                 h7f8727e_1    defaults
libffi                    3.4.4                h6a678d5_1    defaults
libgcc-ng                 11.2.0               h1234567_1    defaults
libgomp                   11.2.0               h1234567_1    defaults
liblief                   0.12.3               h6a678d5_0    defaults
libmamba                  1.5.8                hfe524e5_2    defaults
libmambapy                1.5.8           py312h2dafd23_2    defaults
libnghttp2                1.57.0               h2d74bed_0    defaults
libsolv                   0.7.24               he621ea3_1    defaults
libssh2                   1.11.0               h251f7ec_0    defaults
libstdcxx-ng              11.2.0               h1234567_1    defaults
libuuid                   1.41.5               h5eee18b_0    defaults
libxml2                   2.10.4               hfdd30dd_2    defaults
lz4-c                     1.9.4                h6a678d5_1    defaults
markupsafe                2.1.3           py312h5eee18b_0    defaults
menuinst                  2.1.1           py312h06a4308_0    defaults
more-itertools            10.1.0          py312h06a4308_0    defaults
ncurses                   6.4                  h6a678d5_0    defaults
openssl                   3.0.14               h5eee18b_0    defaults
packaging                 23.2            py312h06a4308_0    defaults
patch                     2.7.6             h7b6447c_1001    defaults
patchelf                  0.17.2               h6a678d5_0    defaults
pcre2                     10.42                hebb0a14_1    defaults
pip                       24.0            py312h06a4308_0    defaults
pkginfo                   1.10.0          py312h06a4308_0    defaults
platformdirs              3.10.0          py312h06a4308_0    defaults
pluggy                    1.0.0           py312h06a4308_1    defaults
psutil                    5.9.0           py312h5eee18b_0    defaults
py-lief                   0.12.3          py312h6a678d5_0    defaults
pybind11-abi              5                    hd3eb1b0_0    defaults
pycosat                   0.6.6           py312h5eee18b_1    defaults
pycparser                 2.21               pyhd3eb1b0_0    defaults
pysocks                   1.7.1           py312h06a4308_0    defaults
python                    3.12.4               h5148396_1    defaults
python-libarchive-c       2.9                pyhd3eb1b0_1    defaults
pytz                      2024.1          py312h06a4308_0    defaults
pyyaml                    6.0.1           py312h5eee18b_0    defaults
readline                  8.2                  h5eee18b_0    defaults
referencing               0.30.2          py312h06a4308_0    defaults
reproc                    14.2.4               h6a678d5_2    defaults
reproc-cpp                14.2.4               h6a678d5_2    defaults
requests                  2.32.2          py312h06a4308_0    defaults
rpds-py                   0.10.6          py312hb02cf49_0    defaults
ruamel.yaml               0.17.21         py312h5eee18b_0    defaults
setuptools                69.5.1          py312h06a4308_0    defaults
soupsieve                 2.5             py312h06a4308_0    defaults
sqlite                    3.45.3               h5eee18b_0    defaults
tk                        8.6.14               h39e8969_0    defaults
tqdm                      4.66.4          py312he106c6f_0    defaults
truststore                0.8.0           py312h06a4308_0    defaults
tzdata                    2024a                h04d1e81_0    defaults
urllib3                   2.2.2           py312h06a4308_0    defaults
wheel                     0.43.0          py312h06a4308_0    defaults
xz                        5.4.6                h5eee18b_1    defaults
yaml                      0.2.5                h7b6447c_0    defaults
yaml-cpp                  0.8.0                h6a678d5_1    defaults
zlib                      1.2.13               h5eee18b_1    defaults
zstandard                 0.22.0          py312h2c38b39_0    defaults
zstd                      1.5.5                hc292b87_2    defaults

Additional Context

No response

skupr-anaconda commented 2 days ago

I saw this error in the about/description section of the meta.yaml, where inside the string was a substring [repack of MINGW-packages libidn2 for UCRT64]. conda-build thinks that it's a selector, not a part of the YAML string. It can be fixed manually by replacing square brackets with round brackets. But yeah, it's a conda-build bug.