search

apache / arrow-adbc

Database connectivity API standard and libraries for Apache Arrow
https://arrow.apache.org/adbc/
Apache License 2.0
385 stars 97 forks source link

dev/release: GitHub is rate-limiting artifact downloads more strictly #2307

Closed lidavidm closed 2 weeks ago

lidavidm commented 2 weeks ago

What happened?

Trying to verify the 15 RC: https://github.com/apache/arrow-adbc/actions/runs/11716816639/job/32637179060

It seems we now get rate-limited just trying to list the files in a release.

Stack Trace

+ python3 /var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/arrow-adbc-15.XXXXX.fUSHB88k/arrow/dev/release/download_rc_binaries.py 15 0 --dest=/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/arrow-adbc-15.XXXXX.fUSHB88k/binaries --num_parallel 4 --package_type=github --repository=apache/arrow-adbc --tag=apache-arrow-adbc-15-rc0
Fetching release from https://api.github.com/repos/apache/arrow-adbc/releases/tags/apache-arrow-adbc-15-rc0
Traceback (most recent call last):
  File "/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/arrow-adbc-15.XXXXX.fUSHB88k/arrow/dev/release/download_rc_binaries.py", line 307, in <module>
    download_rc_binaries(
    ~~~~~~~~~~~~~~~~~~~~^
        args.version,
        ^^^^^^^^^^^^^
    ...<6 lines>...
        tag=args.tag,
        ^^^^^^^^^^^^^
    )
    ^
  File "/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/arrow-adbc-15.XXXXX.fUSHB88k/arrow/dev/release/download_rc_binaries.py", line 277, in download_rc_binaries
    files = downloader.get_file_list(prefix, filter=filter)
  File "/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/arrow-adbc-15.XXXXX.fUSHB88k/arrow/dev/release/download_rc_binaries.py", line 173, in get_file_list
    raw_response = urllib.request.urlopen(request).read().decode()
                   ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", line 189, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", line 495, in open
    response = meth(req, response)
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", line 604, in http_response
    response = self.parent.error(
        'http', request, response, code, msg, hdrs)
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", line 533, in error
    return self._call_chain(*args)
           ~~~~~~~~~~~~~~~~^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", line 466, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/urllib/request.py", line 613, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: rate limit exceeded

How can we reproduce the bug?

No response

Environment/Setup

GitHub Actions