search

conda-forge / conda-smithy

The tool for managing conda-forge feedstocks.
https://conda-forge.org/
BSD 3-Clause "New" or "Revised" License
152 stars 176 forks source link

Error with `register-ci` introduced in 3.38.0: `stderr: 'fatal: pathspec './disabled_appveyor.yml' did not match any files'` #2059

Open jdblischak opened 1 month ago

jdblischak commented 1 month ago

Solution to issue cannot be found in the documentation.

Issue

I often use register-ci locally to activate Azure builds for my feedstock forks. Today this surprisingly failed. I got the same error with both 3.39.1 and 3.38.0. Once I downgraded to 3.37.2, it worked as expected. Looking at the diff between 3.38.0 and 3.37.2, I don't see why the register-ci behavior should have changed.

Here is how I discovered the error:

git clone https://github.com/jdblischak/r-git2r-feedstock.git
cd r-git2r-feedstock/
export AZURE_ORG_OR_USER="jdblischak"
conda smithy register-ci --user "jdblischak" --feedstock_directory . --without-all --with-azure
## No circle token.  Create a token at https://circleci.com/account/api and
## put it in ~/.conda-smithy/circle.token
## No appveyor token. Create a token at https://ci.appveyor.com/api-token and
## Put one in ~/.conda-smithy/appveyor.token
## No drone token. Create a token at https://cloud.drone.io/account and
## Put one in ~/.conda-smithy/drone.token
## No anaconda token. Create a token via
##   anaconda auth --create --name conda-smithy --scopes "repos conda api"
## and put it in ~/.conda-smithy/anaconda.token
## Traceback (most recent call last):
##   File "/home/wsl/mambaforge/bin/conda-smithy", line 10, in <module>
##     sys.exit(main())
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/conda_smithy/cli.py", line 758, in main
##     args.subcommand_func(args)
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/conda_smithy/cli.py", line 334, in __call__
##     forge_config = _load_forge_config(args.feedstock_directory, None)
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/conda_smithy/configure_feedstock.py", line 2400, in _load_forge_config
##     config = _legacy_compatibility_checks(config, forge_dir)
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/conda_smithy/configure_feedstock.py", line 2290, in _legacy_compatibility_checks
##     remove_file_or_dir(os.path.join(forge_dir, old_file))
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/conda_smithy/feedstock_io.py", line 67, in remove_file_or_dir
##     return remove_file(filename)
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/conda_smithy/feedstock_io.py", line 80, in remove_file
##     repo.index.remove([filename])
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/git/index/util.py", line 73, in post_clear_cache_if_not_raised
##     rval = func(self, *args, **kwargs)
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/git/index/util.py", line 95, in check_default_index
##     return func(self, *args, **kwargs)
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/git/index/base.py", line 1029, in remove
##     removed_paths = self.repo.git.rm(args, paths, **kwargs).splitlines()
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/git/cmd.py", line 986, in <lambda>
##     return lambda *args, **kwargs: self._call_process(name, *args, **kwargs)
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/git/cmd.py", line 1598, in _call_process
##     return self.execute(call, **exec_kwargs)
##   File "/home/wsl/mambaforge/lib/python3.10/site-packages/git/cmd.py", line 1388, in execute
##     raise GitCommandError(redacted_command, status, stderr_value, stdout_value)
## git.exc.GitCommandError: Cmd('git') failed due to: exit code(128)
##   cmdline: git rm --cached -- ./disabled_appveyor.yml
##   stderr: 'fatal: pathspec './disabled_appveyor.yml' did not match any files'
git status
## On branch main
## Your branch is up to date with 'origin/main'.
## 
## Changes to be committed:
##   (use "git restore --staged <file>..." to unstage)
##         new file:   ./disabled_appveyor.yml
## 
## Untracked files:
##   (use "git add <file>..." to include in what will be committed)
##         disabled_appveyor.yml

With conda-smithy 3.37.2, it worked as expected:

conda smithy register-ci --user "jdblischak" --feedstock_directory . --without-all --with-azure
## No circle token.  Create a token at https://circleci.com/account/api and
## put it in ~/.conda-smithy/circle.token
## No appveyor token. Create a token at https://ci.appveyor.com/api-token and
## Put one in ~/.conda-smithy/appveyor.token
## No drone token. Create a token at https://cloud.drone.io/account and
## Put one in ~/.conda-smithy/drone.token
## No anaconda token. Create a token via
##   anaconda auth --create --name conda-smithy --scopes "repos conda api"
## and put it in ~/.conda-smithy/anaconda.token
## WARNING: No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.22
## Adding in variants from internal_defaults
## CI Summary for jdblischak/r-git2r-feedstock (can take ~30s):
## Travis registration disabled.
## Circle registration disabled.
##  * jdblischak/r-git2r-feedstock has been enabled on azure pipelines
## Appveyor registration disabled.
## Drone registration disabled.
## Cirun registration disabled.
## Heroku webservice registration disabled.
## 
## CI services have been enabled. You may wish to regenerate the feedstock.
## Any changes will need commiting to the repo.

Installed packages

# The working config with conda-smithy 3.37.2.
# The only downgraded package was rattler-build-conda-compat from 1.2.1 to 0.2.5

# packages in environment at /home/wsl/mambaforge:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
anyio                     4.4.0              pyhd8ed1ab_0    conda-forge
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
attrs                     24.2.0             pyh71513ae_0    conda-forge
backports                 1.0                pyhd8ed1ab_4    conda-forge
backports.strenum         1.3.1              pyhd8ed1ab_0    conda-forge
backports.tarfile         1.0.0              pyhd8ed1ab_1    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
black                     24.8.0          py310hff52083_0    conda-forge
blinker                   1.8.2              pyhd8ed1ab_0    conda-forge
boa                       0.17.0             pyhd8ed1ab_3    conda-forge
boltons                   24.0.0             pyhd8ed1ab_0    conda-forge
boolean.py                4.0                pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py310hf71b8c6_2    conda-forge
bzip2                     1.0.8                h4bc722e_7    conda-forge
c-ares                    1.33.1               heb4867d_0    conda-forge
ca-certificates           2024.8.30            hbcca054_0    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py310h8deb56e_0    conda-forge
chardet                   5.2.0           py310hff52083_2    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
cirun                     0.30               pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     24.7.1          py310hff52083_0    conda-forge
conda-build               24.5.1          py310hff52083_0    conda-forge
conda-forge-pinning       2024.09.09.07.32.13      hd8ed1ab_0    conda-forge
conda-index               0.5.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     24.7.0             pyhd8ed1ab_0    conda-forge
conda-package-handling    2.3.0              pyh7900ff3_0    conda-forge
conda-package-streaming   0.10.0             pyhd8ed1ab_0    conda-forge
conda-smithy              3.37.2          unix_pyh707e725_1    conda-forge
conda-souschef            2.2.3              pyhd8ed1ab_0    conda-forge
cryptography              43.0.1          py310h6c63255_0    conda-forge
curl                      8.9.1                h18eb788_0    conda-forge
dbus                      1.13.6               h5008d03_3    conda-forge
deprecated                1.2.14             pyh1a96a4e_0    conda-forge
distlib                   0.3.8              pyhd8ed1ab_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
editables                 0.5                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.2              pyhd8ed1ab_0    conda-forge
expat                     2.6.3                h5888daf_0    conda-forge
filelock                  3.16.0             pyhd8ed1ab_0    conda-forge
fmt                       10.2.1               h00ab1b0_0    conda-forge
frozendict                2.4.4           py310hc51659f_0    conda-forge
gettext                   0.22.5               he02047a_3    conda-forge
gettext-tools             0.22.5               he02047a_3    conda-forge
git                       2.46.0          pl5321hb5640b7_0    conda-forge
gitdb                     4.0.11             pyhd8ed1ab_0    conda-forge
gitpython                 3.1.43             pyhd8ed1ab_0    conda-forge
gmp                       6.3.0                hac33072_2    conda-forge
grayskull                 2.6.0              pyhd8ed1ab_1    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hatch                     1.12.0             pyhd8ed1ab_0    conda-forge
hatchling                 1.25.0             pyhd8ed1ab_0    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  1.0.5              pyhd8ed1ab_0    conda-forge
httpx                     0.27.2             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
hyperlink                 21.0.0             pyhd3deb0d_0    conda-forge
icu                       75.1                 he02047a_0    conda-forge
idna                      3.8                pyhd8ed1ab_0    conda-forge
importlib-metadata        8.4.0              pyha770c72_0    conda-forge
importlib_metadata        8.4.0                hd8ed1ab_0    conda-forge
importlib_resources       6.4.4              pyhd8ed1ab_0    conda-forge
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
isodate                   0.6.1              pyhd8ed1ab_0    conda-forge
jaraco.classes            3.4.0              pyhd8ed1ab_1    conda-forge
jaraco.context            5.3.0              pyhd8ed1ab_1    conda-forge
jaraco.functools          4.0.0              pyhd8ed1ab_0    conda-forge
jeepney                   0.8.0              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
joblib                    1.4.2              pyhd8ed1ab_0    conda-forge
json5                     0.9.25             pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               3.0.0           py310hff52083_1    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
keyring                   25.3.0             pyha804496_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
krb5                      1.21.3               h659f571_0    conda-forge
ld_impl_linux-64          2.40                 hf3520f5_7    conda-forge
libarchive                3.7.4                hfca40fe_0    conda-forge
libasprintf               0.22.5               he8f35ee_3    conda-forge
libasprintf-devel         0.22.5               he8f35ee_3    conda-forge
libblas                   3.9.0           23_linux64_openblas    conda-forge
libcblas                  3.9.0           23_linux64_openblas    conda-forge
libcurl                   8.9.1                hdb1bdb2_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libexpat                  2.6.3                h5888daf_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc                    14.1.0               h77fa898_1    conda-forge
libgcc-ng                 14.1.0               h69a702a_1    conda-forge
libgettextpo              0.22.5               he02047a_3    conda-forge
libgettextpo-devel        0.22.5               he02047a_3    conda-forge
libgfortran               14.1.0               h69a702a_1    conda-forge
libgfortran-ng            14.1.0               h69a702a_1    conda-forge
libgfortran5              14.1.0               hc5f4f2c_1    conda-forge
libglib                   2.80.3               h315aac3_2    conda-forge
libgomp                   14.1.0               h77fa898_1    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
liblapack                 3.9.0           23_linux64_openblas    conda-forge
liblief                   0.14.1               hac33072_1    conda-forge
libmamba                  1.5.9                h4cc3d14_0    conda-forge
libmambapy                1.5.9           py310h86cbe3b_0    conda-forge
libnghttp2                1.58.0               h47da74e_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.27          pthreads_hac2b453_1    conda-forge
libsodium                 1.0.20               h4ab18f5_0    conda-forge
libsolv                   0.7.30               h3509ff9_0    conda-forge
libsqlite                 3.46.1               hadc24fc_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx                 14.1.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              14.1.0               h4852527_1    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.12.7               he7c6b58_4    conda-forge
libzlib                   1.3.1                h4ab18f5_1    conda-forge
license-expression        30.1.1             pyhd8ed1ab_0    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              hd590300_1001    conda-forge
mamba                     1.5.9           py310h51d5547_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5           py310ha75aee5_1    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
menuinst                  2.1.2           py310hff52083_1    conda-forge
more-itertools            10.5.0             pyhd8ed1ab_0    conda-forge
msrest                    0.6.21             pyh44b312d_0    conda-forge
mypy_extensions           1.0.0              pyha770c72_0    conda-forge
ncurses                   6.5                  he02047a_1    conda-forge
numpy                     2.1.1           py310hd6e36ab_0    conda-forge
oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
openssl                   3.3.2                hb9d3cd8_0    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
patch                     2.7.6             h7f98852_1002    conda-forge
patchelf                  0.17.2               h58526e2_0    conda-forge
pathspec                  0.12.1             pyhd8ed1ab_0    conda-forge
pcre2                     10.44                hba22ea6_2    conda-forge
perl                      5.32.1          7_hd590300_perl5    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
pkginfo                   1.11.1             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.3.2              pyhd8ed1ab_0    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_0    conda-forge
progressbar2              4.5.0              pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.47             pyha770c72_0    conda-forge
prompt_toolkit            3.0.47               hd8ed1ab_0    conda-forge
psutil                    6.0.0           py310ha75aee5_1    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
py-lief                   0.14.1          py310h76e45a6_1    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6           py310h2372a71_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pycryptodome              3.20.0          py310hb0f0acc_0    conda-forge
pygithub                  2.4.0              pyhd8ed1ab_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyjwt                     2.9.0              pyhd8ed1ab_1    conda-forge
pynacl                    1.5.0           py310ha75aee5_4    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pytest                    8.3.2              pyhd8ed1ab_0    conda-forge
python                    3.10.14         hd12c33a_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-libarchive-c       5.1             py310hff52083_1    conda-forge
python-utils              3.8.2              pyhd8ed1ab_0    conda-forge
python_abi                3.10                    5_cp310    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.2           py310ha75aee5_1    conda-forge
rapidfuzz                 3.9.7           py310hf71b8c6_0    conda-forge
rattler-build             0.21.0               hbfb6850_0    conda-forge
rattler-build-conda-compat 0.2.5              pyhd8ed1ab_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
reproc                    14.2.4.post0         hd590300_1    conda-forge
reproc-cpp                14.2.4.post0         h59595ed_1    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
requests-oauthlib         2.0.0              pyhd8ed1ab_0    conda-forge
rich                      13.7.1             pyhd8ed1ab_0    conda-forge
ripgrep                   14.1.0               he8a937b_0    conda-forge
rpds-py                   0.20.0          py310h505e2c1_1    conda-forge
ruamel.yaml               0.18.6          py310h2372a71_0    conda-forge
ruamel.yaml.clib          0.2.8           py310h2372a71_0    conda-forge
ruamel.yaml.jinja2        0.2.7              pyhd8ed1ab_0    conda-forge
scrypt                    0.8.24          py310h6ec971a_1    conda-forge
secretstorage             3.3.3           py310hff52083_2    conda-forge
semver                    3.0.2              pyhd8ed1ab_0    conda-forge
setuptools                73.0.1             pyhd8ed1ab_0    conda-forge
shellingham               1.5.4              pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smmap                     5.0.0              pyhd8ed1ab_0    conda-forge
sniffio                   1.3.1              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
stdlib-list               0.10.0             pyhd8ed1ab_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomli-w                   1.0.0              pyhd8ed1ab_0    conda-forge
tomlkit                   0.13.2             pyha770c72_0    conda-forge
toolz                     0.12.1             pyhd8ed1ab_0    conda-forge
tqdm                      4.66.5             pyhd8ed1ab_0    conda-forge
trove-classifiers         2024.7.2           pyhd8ed1ab_0    conda-forge
truststore                0.9.2              pyhd8ed1ab_0    conda-forge
typer                     0.12.5             pyhd8ed1ab_0    conda-forge
typer-slim                0.12.5             pyhd8ed1ab_0    conda-forge
typer-slim-standard       0.12.5               hd8ed1ab_0    conda-forge
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024a                h8827d51_1    conda-forge
urllib3                   2.2.2              pyhd8ed1ab_1    conda-forge
userpath                  1.7.0              pyhd8ed1ab_0    conda-forge
uv                        0.4.7                h0f3a69f_0    conda-forge
virtualenv                20.26.4            pyhd8ed1ab_0    conda-forge
vsts-python-api           0.1.25             pyhd8ed1ab_1    conda-forge
watchgod                  0.7                pyhd8ed1ab_0    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
wrapt                     1.16.0          py310ha75aee5_1    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yaml-cpp                  0.8.0                h59595ed_0    conda-forge
zipp                      3.20.1             pyhd8ed1ab_0    conda-forge
zstandard                 0.23.0          py310ha39cb0e_1    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge

Environment info

active environment : base
    active env location : /home/wsl/mambaforge
            shell level : 1
       user config file : /home/wsl/.condarc
 populated config files : /home/wsl/mambaforge/.condarc
          conda version : 24.7.1
    conda-build version : 24.5.1
         python version : 3.10.14.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=skylake
                          __conda=24.7.1=0
                          __glibc=2.35=0
                          __linux=5.15.153.1=0
                          __unix=0=0
       base environment : /home/wsl/mambaforge  (writable)
      conda av data dir : /home/wsl/mambaforge/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/wsl/mambaforge/pkgs
                          /home/wsl/.conda/pkgs
       envs directories : /home/wsl/mambaforge/envs
                          /home/wsl/.conda/envs
               platform : linux-64
             user-agent : conda/24.7.1 requests/2.32.3 CPython/3.10.14 Linux/5.15.153.1-microsoft-standard-WSL2 ubuntu/22.04.4 glibc/2.35 solver/libmamba conda-libmamba-solver/24.7.0 libmambapy/1.5.9
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False
jdblischak commented 1 day ago

I ran into this same error again today with conda-smithy 3.43.0 when trying to locally run register-ci on the feedstock jdblischak/cellxgene-census-feedstock. Again I worked around it by downgrading to conda-smithy 3.37.2.

jdblischak commented 1 day ago

I'm still confused why this error started between 3.37.2 and 3.38.0. The functions _legacy_compatibility_checks, remove_file_or_dir, and remove_file haven't been touched in over a year. See below for links to git blame

https://github.com/conda-forge/conda-smithy/blame/52af540b1e3d9781893498ec5ce90f58c4c0d8e6/conda_smithy/configure_feedstock.py#L2275 https://github.com/conda-forge/conda-smithy/blame/52af540b1e3d9781893498ec5ce90f58c4c0d8e6/conda_smithy/feedstock_io.py#L65 https://github.com/conda-forge/conda-smithy/blame/52af540b1e3d9781893498ec5ce90f58c4c0d8e6/conda_smithy/feedstock_io.py#L75