Intermittently, when running ./build_image.sh cpu, the function detect_llvm_version() does not run correctly due to exceeding Github API rate limits. This then causes a complete failure as the script attempts to use Curl to fetch the LLVM project from Github with the following syntax llvm-project-${LLVM_VERSION}.src.tar.xz. As no version is set, an invalid URL is passed to Curl and an empty file is downloaded. In the logs when building a package-cpu Docker image the following error is seen:
=> ERROR [ 7/20] RUN bash /install/centos_build_llvm.sh 16.0
------
[ 7/20] RUN bash /install/centos_build_llvm.sh 16.0:
unxz: llvm-project-.src.tar.xz: File format not recognized
Additional Details
The Github API rate limit error looks like this:
{"message":"API rate limit exceeded for <IP ADDRESS>. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
In the ./build_image.sh script we are not using an authenticated API request, so the rate is set very low (<50 in given timeout period).
Flakiness
Intermittent and more likely to impact CI running regularly, rather than an individual.
Suggested Solution
Don't use the Github API to detect the LLVM version. List the remote tags of the repo using git ls-remote instead. A Github support discussion here suggests
Git operations do not consume part of your API rate limit, as there are no API calls made to GitHub.com via the git client.
The Problem
Intermittently, when running
./build_image.sh cpu
, the functiondetect_llvm_version()
does not run correctly due to exceeding Github API rate limits. This then causes a complete failure as the script attempts to use Curl to fetch the LLVM project from Github with the following syntaxllvm-project-${LLVM_VERSION}.src.tar.xz
. As no version is set, an invalid URL is passed to Curl and an empty file is downloaded. In the logs when building apackage-cpu
Docker image the following error is seen:Additional Details
The Github API rate limit error looks like this:
The error can be replicated after the rate limit is exceeded by navigating to the LLVM URL directly: https://api.github.com/repos/llvm/llvm-project/releases?per_page=100
In the
./build_image.sh
script we are not using an authenticated API request, so the rate is set very low (<50 in given timeout period).Flakiness
Intermittent and more likely to impact CI running regularly, rather than an individual.
Suggested Solution
Don't use the Github API to detect the LLVM version. List the remote tags of the repo using
git ls-remote
instead. A Github support discussion here suggests