Installing bugsnag-cli fails when it is called many times within a short period of time

[Frank3K]

Describe the bug

We're trying to integrate bugsnag-cli as a post-publish script on codemagic, such that we can use it to upload debug symbols. The installation fails often, but not always. At first, we assumed it was an issue at codemagic, but they investigated the issue (https://github.com/orgs/codemagic-ci-cd/discussions/2023) and came to the conclusion it was not within their infrastructure.

Steps to reproduce

Consider this script:

#!/bin/sh
set -e

BUGSNAG_CLI_PATH=$HOME/.local/bugsnag/bin/bugsnag-cli

echo '--- Installing bugsnag-cli for:' $(uname)-$(uname -m) '---'
curl -o- https://raw.githubusercontent.com/bugsnag/bugsnag-cli/main/install.sh | bash

Example succesful run:

--- Installing bugsnag-cli for: Darwin-arm64 ---
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  4029  100  4029    0     0  35187      0 --:--:-- --:--:-- --:--:-- 36627
==> This script will install:
/Users/builder/.local/bugsnag/bin/bugsnag-cli
==> The following new directories will be created:
/Users/builder/.local/bugsnag/bin
==> Downloading and installing Bugsnag CLI...
#=#=#                                                                         
##O#-#                                                                        

                                                                           0.2%
                                                                           1.3%
########################################                                  55.8%
##########################################################                80.8%
######################################################################## 100.0%
Warning: /Users/builder/.local/bugsnag/bin is not in your PATH.
  Instructions on how to configure your shell for Bugsnag CLI
  can be found in the 'Next steps' section below.
==> Installation successful!

Example unsuccessful run:

--- Installing bugsnag-cli for: Darwin-arm64 ---
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  4029  100  4029    0     0  28944      0 --:--:-- --:--:-- --:--:-- 30067
==> This script will install:
/Users/builder/.local/bugsnag/bin/bugsnag-cli
==> The following new directories will be created:
/Users/builder/.local/bugsnag/bin
==> Downloading and installing Bugsnag CLI...
Failed to get download URL from https://api.github.com/repos/bugsnag/bugsnag-cli/releases/latest

Build failed :|
Step 9 script `Post-publish script` exited with status code 1

Environment

• bugsnag-cli version: latest (install.sh)
• Platform: codemagic (reproducible locally)

Example code snippet

Included above.

[clr182]

Hi @Frank3K When you say reproducible locally, do you mean you have tested the cURL command in isolation outside of the codemagic platform? I have tried to reproduce this issue and have been unable to do so. Is this still a consistently flakey issue for you? If so, is this failing consistently at this line ==> Downloading and installing Bugsnag CLI... Failed to get download URL from <https://api.github.com/repos/bugsnag/bugsnag-cli/releases/latest> ?

It's also possible that the script is falling over sometimes is due to an unexpected uname in the install.sh script. The uname variable looks to see if the operating system is either Linex or Darwin, some deviations on this may cause the script to fail inconsistently.

If you are aware of the uname of the failing device, please let us know and we could provide you with an updated install.sh script

[Frank3K]

The uname is consistent, also on codemagic. We've further investigated the issue and we've found that it fails when the rate limit is reached. For unauthenticated users it's 60 request per hour (source). On a local machine it's not that easy to reach, but on CI platforms like codemagic it is since the limit is per IP.

With some more verbose logging:

> GET /repos/bugsnag/bugsnag-cli/releases/latest HTTP/2
> Host: api.github.com
> User-Agent: curl/8.1.2
> Accept: */*
>
< HTTP/2 403
< date: Fri, 22 Sep 2023 06:51:48 GMT
< server: Varnish
< strict-transport-security: max-age=31536000; includeSubdomains; preload
< x-content-type-options: nosniff
< x-frame-options: deny
< x-xss-protection: 1; mode=block
< content-security-policy: default-src 'none'; style-src 'unsafe-inline'
< access-control-allow-origin: *
< access-control-expose-headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-RateLimit-Used, X-RateLimit-Resource, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset
< content-type: application/json; charset=utf-8
< referrer-policy: origin-when-cross-origin, strict-origin-when-cross-origin
< x-github-media-type: github.v3; format=json
< x-ratelimit-limit: 60
< x-ratelimit-remaining: 0
< x-ratelimit-reset: **********
< x-ratelimit-resource: core
< x-ratelimit-used: 60
< content-length: 281
< x-github-request-id: ****:*****:********:*******:*********
<
{"message":"API rate limit exceeded for ***.***.***.***. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)","documentation_url":"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"}
* Connection #0 to host api.github.com left intact
Failed to get download URL from https://api.github.com/repos/bugsnag/bugsnag-cli/releases/latest

[mclack]

Hi @Frank3K

Thanks for the additional analysis. We are currently considering some options to improve the script, and will let you know of any updates around this.

In the meantime, the short-term solution we’d recommend is to curl a fixed version of the CLI on the appropriate architecture for your build machine: https://github.com/bugsnag/bugsnag-cli/blob/next/install.sh#L150

Please let us know if you have any further questions, or need any clarification here.

[clr182]

Hi @Frank3K

We have a new release for our BugSnag CLI Tool (v2.0.0) which has improvements to the install script. This improvement is aimed at providing better exceptions and more specific information.

If you are still experiencing these issues then we recommend upgrading to this new release and sharing any exception you see with us here via this thread.

[Frank3K]

Thanks for fixing. We have integrated the new script in our codemagic flow and it works fine now 👍 .