search

conda-forge / conda-smithy

The tool for managing conda-forge feedstocks.
https://conda-forge.org/
BSD 3-Clause "New" or "Revised" License
147 stars 177 forks source link

`conda smithy rerender -c auto` hangs for >1min in `get_most_recent_version("conda-forge-pinning")` function #1973

Open corneliusroemer opened 1 month ago

corneliusroemer commented 1 month ago

Solution to issue cannot be found in the documentation.

Issue

I locally ran conda smithy rerender -c auto and nothing happened for >1m.

I aborted after 1m4s, at which point the traceback showed that the program was stuck in the function get_most_recent_version("conda-forge-pinning")

Full traceback:

$ micromamba run -n smithy conda smithy rerender -c auto
^CTraceback (most recent call last):
  File "/Users/corneliusromer/micromamba/envs/smithy/bin/conda-smithy", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/cli.py", line 737, in main
    args.subcommand_func(args)
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/cli.py", line 584, in __call__
    self._call(args, tmpdir)
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/cli.py", line 589, in _call
    configure_feedstock.main(
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/configure_feedstock.py", line 2714, in main
    exclusive_config_file, cf_pinning_ver = get_cached_cfp_file_path(
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/configure_feedstock.py", line 2535, in get_cached_cfp_file_path
    get_cfp_file_path(smithy_cache)
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/configure_feedstock.py", line 2459, in get_cfp_file_path
    pkg = get_most_recent_version("conda-forge-pinning")
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/conda_smithy/configure_feedstock.py", line 2395, in get_most_recent_version
    request = requests.get(
              ^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/urllib3/connectionpool.py", line 536, in _make_request
    response = conn.getresponse()
               ^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/site-packages/urllib3/connection.py", line 464, in getresponse
    httplib_response = super().getresponse()
                       ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/http/client.py", line 1428, in getresponse
    response.begin()
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/http/client.py", line 331, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/http/client.py", line 292, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/socket.py", line 708, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/ssl.py", line 1252, in recv_into
    return self.read(nbytes, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/corneliusromer/micromamba/envs/smithy/lib/python3.12/ssl.py", line 1104, in read
    return self._sslobj.read(len, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

Installed packages

# packages in environment at /Users/corneliusromer/micromamba/envs/smithy:
#
# Name                    Version                   Build  Channel
anyio                     4.4.0              pyhd8ed1ab_0    conda-forge
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
attrs                     23.2.0             pyh71513ae_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.strenum         1.2.8              pyhd8ed1ab_0    conda-forge
backports.tarfile         1.0.0              pyhd8ed1ab_1    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
blinker                   1.8.2              pyhd8ed1ab_0    conda-forge
boltons                   24.0.0             pyhd8ed1ab_0    conda-forge
boolean.py                4.0                pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py312h9f69965_1    conda-forge
bzip2                     1.0.8                h93a5062_5    conda-forge
c-ares                    1.28.1               h93a5062_0    conda-forge
ca-certificates           2024.6.2             hf0a4a13_0    conda-forge
cctools                   986                  hf67d63f_0    conda-forge
cctools_osx-arm64         986                  h998149b_0    conda-forge
certifi                   2024.6.2           pyhd8ed1ab_0    conda-forge
cffi                      1.16.0          py312h8e38eb3_0    conda-forge
chardet                   5.2.0           py312h81bd7bf_1    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
cirun                     0.30               pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     24.5.0          py312h81bd7bf_0    conda-forge
conda-build               24.5.1          py312h81bd7bf_0    conda-forge
conda-forge-pinning       2024.07.01.19.34.18      hd8ed1ab_0    conda-forge
conda-index               0.5.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     24.1.0             pyhd8ed1ab_0    conda-forge
conda-package-handling    2.3.0              pyh7900ff3_0    conda-forge
conda-package-streaming   0.10.0             pyhd8ed1ab_0    conda-forge
conda-smithy              3.36.2          unix_pyh707e725_0    conda-forge
cryptography              42.0.8          py312had01cb0_0    conda-forge
deprecated                1.2.14             pyh1a96a4e_0    conda-forge
distlib                   0.3.8              pyhd8ed1ab_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
editables                 0.5                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
filelock                  3.15.4             pyhd8ed1ab_0    conda-forge
fmt                       10.2.1               h2ffa867_0    conda-forge
frozendict                2.4.4           py312h7e5086c_0    conda-forge
git                       2.45.2          pl5321h41514c7_1    conda-forge
gitdb                     4.0.11             pyhd8ed1ab_0    conda-forge
gitpython                 3.1.43             pyhd8ed1ab_0    conda-forge
gmp                       6.3.0                h7bae524_2    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hatch                     1.12.0             pyhd8ed1ab_0    conda-forge
hatchling                 1.25.0             pyhd8ed1ab_0    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
httpcore                  1.0.5              pyhd8ed1ab_0    conda-forge
httpx                     0.27.0             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
hyperlink                 21.0.0             pyhd3deb0d_0    conda-forge
icu                       73.2                 hc8870d7_0    conda-forge
idna                      3.7                pyhd8ed1ab_0    conda-forge
importlib-metadata        8.0.0              pyha770c72_0    conda-forge
importlib_metadata        8.0.0                hd8ed1ab_0    conda-forge
importlib_resources       6.4.0              pyhd8ed1ab_0    conda-forge
isodate                   0.6.1              pyhd8ed1ab_0    conda-forge
jaraco.classes            3.4.0              pyhd8ed1ab_1    conda-forge
jaraco.context            5.3.0              pyhd8ed1ab_1    conda-forge
jaraco.functools          4.0.0              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               3.0.0           py312h81bd7bf_0    conda-forge
jsonschema                4.22.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.12.1          pyhd8ed1ab_0    conda-forge
keyring                   25.2.1             pyh534df25_0    conda-forge
krb5                      1.21.3               h237132a_0    conda-forge
ld64                      711                  h39a299f_0    conda-forge
ld64_osx-arm64            711                  h6a44367_0    conda-forge
libarchive                3.7.4                h83d404f_0    conda-forge
libcurl                   8.8.0                h7b6f9a7_1    conda-forge
libcxx                    17.0.6               he7857fb_2    conda-forge
libedit                   3.1.20191231         hc8eb9b7_2    conda-forge
libev                     4.33                 h93a5062_2    conda-forge
libexpat                  2.6.2                hebf3989_0    conda-forge
libffi                    3.4.2                h3422bc3_5    conda-forge
libiconv                  1.17                 h0d3ecfb_2    conda-forge
libintl                   0.22.5               h8fbad5d_2    conda-forge
liblief                   0.14.1               h3f3aa29_1    conda-forge
libllvm17                 17.0.6               h5090b49_2    conda-forge
libmamba                  1.5.8                h90c426b_0    conda-forge
libmambapy                1.5.8           py312h344e357_0    conda-forge
libnghttp2                1.58.0               ha4dd798_1    conda-forge
libsodium                 1.0.18               h27ca646_1    conda-forge
libsolv                   0.7.29               h1efcc80_0    conda-forge
libsqlite                 3.46.0               hfb93653_0    conda-forge
libssh2                   1.11.0               h7a5bd25_0    conda-forge
libxml2                   2.12.7               ha661575_1    conda-forge
libzlib                   1.3.1                hfb2fe0b_1    conda-forge
license-expression        30.1.1             pyhd8ed1ab_0    conda-forge
lz4-c                     1.9.4                hb7217d7_0    conda-forge
lzo                       2.10              h93a5062_1001    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5           py312he37b823_0    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
menuinst                  2.1.1           py312h81bd7bf_0    conda-forge
more-itertools            10.3.0             pyhd8ed1ab_0    conda-forge
msrest                    0.6.21             pyh44b312d_0    conda-forge
ncurses                   6.5                  hb89a1cb_0    conda-forge
oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
openssl                   3.3.1                hfb2fe0b_1    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
patch                     2.7.6             h27ca646_1002    conda-forge
pathspec                  0.12.1             pyhd8ed1ab_0    conda-forge
pcre2                     10.44                h297a79d_0    conda-forge
perl                      5.32.1          7_h4614cfb_perl5    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pkginfo                   1.11.1             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.2.2              pyhd8ed1ab_0    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_0    conda-forge
psutil                    6.0.0           py312h7e5086c_0    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
py-lief                   0.14.1          py312h2721eaf_1    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6           py312h02f2b3b_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pycryptodome              3.20.0          py312h332fddf_0    conda-forge
pygithub                  2.3.0              pyhd8ed1ab_0    conda-forge
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyjwt                     2.8.0              pyhd8ed1ab_1    conda-forge
pynacl                    1.5.0           py312h02f2b3b_3    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
python                    3.12.4          h30c5eda_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-libarchive-c       5.1             py312h81bd7bf_0    conda-forge
python_abi                3.12                    4_cp312    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1           py312h02f2b3b_1    conda-forge
readline                  8.2                  h92ec313_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
reproc                    14.2.4.post0         h93a5062_1    conda-forge
reproc-cpp                14.2.4.post0         h965bd2d_1    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
requests-oauthlib         2.0.0              pyhd8ed1ab_0    conda-forge
rich                      13.7.1             pyhd8ed1ab_0    conda-forge
ripgrep                   14.1.0               h5ef7bb8_0    conda-forge
rpds-py                   0.18.1          py312h552d48e_0    conda-forge
ruamel.yaml               0.18.6          py312he37b823_0    conda-forge
ruamel.yaml.clib          0.2.8           py312he37b823_0    conda-forge
scrypt                    0.8.24          py312h25fc620_0    conda-forge
setuptools                70.1.1             pyhd8ed1ab_0    conda-forge
shellingham               1.5.4              pyhd8ed1ab_0    conda-forge
sigtool                   0.1.3                h44b9a77_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
smmap                     5.0.0              pyhd8ed1ab_0    conda-forge
sniffio                   1.3.1              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
tapi                      1100.0.11            he4954df_0    conda-forge
tk                        8.6.13               h5083fa2_1    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomli-w                   1.0.0              pyhd8ed1ab_0    conda-forge
tomlkit                   0.12.5             pyha770c72_0    conda-forge
toolz                     0.12.1             pyhd8ed1ab_0    conda-forge
tqdm                      4.66.4             pyhd8ed1ab_0    conda-forge
trove-classifiers         2024.5.22          pyhd8ed1ab_0    conda-forge
truststore                0.8.0              pyhd8ed1ab_0    conda-forge
typer                     0.12.3             pyhd8ed1ab_0    conda-forge
typer-slim                0.12.3             pyhd8ed1ab_0    conda-forge
typer-slim-standard       0.12.3               hd8ed1ab_0    conda-forge
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
urllib3                   2.2.2              pyhd8ed1ab_1    conda-forge
userpath                  1.7.0              pyhd8ed1ab_0    conda-forge
uv                        0.2.18               hc069d6b_0    conda-forge
virtualenv                20.26.3            pyhd8ed1ab_0    conda-forge
vsts-python-api           0.1.25             pyhd8ed1ab_1    conda-forge
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
wrapt                     1.16.0          py312he37b823_0    conda-forge
xz                        5.2.6                h57fd34a_0    conda-forge
yaml                      0.2.5                h3422bc3_2    conda-forge
yaml-cpp                  0.8.0                h13dd4ca_0    conda-forge
zipp                      3.19.2             pyhd8ed1ab_0    conda-forge
zstandard                 0.22.0          py312h721a963_1    conda-forge
zstd                      1.5.6                hb46c0d2_0    conda-forge

Environment info

active environment : base
    active env location : /Users/corneliusromer/micromamba/envs/smithy
            shell level : 3
       user config file : /Users/corneliusromer/.condarc
 populated config files : /Users/corneliusromer/.condarc
          conda version : 24.5.0
    conda-build version : 24.5.1
         python version : 3.12.4.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=m1
                          __conda=24.5.0=0
                          __osx=14.5=0
                          __unix=0=0
       base environment : /Users/corneliusromer/micromamba/envs/smithy  (writable)
      conda av data dir : /Users/corneliusromer/micromamba/envs/smithy/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-arm64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/osx-arm64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/corneliusromer/micromamba/envs/smithy/pkgs
                          /Users/corneliusromer/.conda/pkgs
       envs directories : /Users/corneliusromer/micromamba/envs/smithy/envs
                          /Users/corneliusromer/.conda/envs
               platform : osx-arm64
             user-agent : conda/24.5.0 requests/2.32.3 CPython/3.12.4 Darwin/23.5.0 OSX/14.5 solver/libmamba conda-libmamba-solver/24.1.0 libmambapy/1.5.8
                UID:GID : 501:20
             netrc file : None
           offline mode : False
jakirkham commented 1 month ago

Wonder if we could just save the commit used for the last download and use that to check whether it is up-to-date before downloading it again