API rate limit exceeded for unauthenticated API calls

Describe the bug cloud-pak-deployer installation fails if the API rate limit is exceeded.

For unauthenticated api calls agains github.com the rate limit is limited to 60 api calls per hour and its attached to the IP address. Sending authenticated API calls the rate limit is increased to 5000 api calls per hour.

Hit the following problem

fatal: [localhost]: FAILED! => {"access_control_allow_origin": "*", "access_control_expose_headers": "ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-RateLimit-Used, X-RateLimit-Resource, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, Deprecation, Sunset", "changed": false, "connection": "close", "content": "{\"message\":\"API rate limit exceeded for 129.41.87.0. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)\",\"documentation_url\":\"https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting\"}\n", "content_length": "277", "content_security_policy": "default-src 'none'; style-src 'unsafe-inline'", "content_type": "application/json; charset=utf-8", "date": "Wed, 18 Oct 2023 20:26:43 GMT", "elapsed": 0, "json": {"documentation_url": "https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting", "message": "API rate limit exceeded for 129.41.87.0. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)"}, "msg": "Status code was 403 and not [200]: HTTP Error 403: rate limit exceeded", "redirected": false, "referrer_policy": "origin-when-cross-origin, strict-origin-when-cross-origin", "server": "Varnish", "status": 403, "strict_transport_security": "max-age=31536000; includeSubdomains; preload", "url": "https://api.github.com/repos/IBM/cloud-pak-cli/releases/latest", "x_content_type_options": "nosniff", "x_frame_options": "deny", "x_github_media_type": "github.v3; format=json", "x_github_request_id": "63C5:0D38:1E5D63:3D989A:65303F83", "x_ratelimit_limit": "60", "x_ratelimit_remaining": "0", "x_ratelimit_reset": "1697660987", "x_ratelimit_resource": "core", "x_ratelimit_used": "60", "x_xss_protection": "1; mode=block"}

Root cause

Unauthenticated API calls to github are limited to a maximum of 60 per hour. This limitation can be bypassed by authenticating the API calls , by doing that the limit is increased to 5000 per hour.

Check this link : https://docs.github.com/en/rest/overview/resources-in-the-rest-api?apiVersion=2022-11-28#rate-limiting

User access token requests are limited to 5,000 requests per hour and per authenticated user. All requests from OAuth apps authorized by a user or a personal access token owned by the user, and requests authenticated with any of the user's authentication credentials, share the same quota of 5,000 requests per hour for that user.

Workaround

Navigate to your User Settings.
Create a personal Access token
Modify the following files:
- automation-roles/99-generic/ibm-pak/ibm-pak-download/tasks/main.yml [Get current version number of ibm-pak]
- automation-roles/99-generic/cpd-cli/cpd-cli-download/tasks/main.yml [Get current version number of cpd-cli]
- automation-roles/99-generic/cloudctl/cloudctl-download/tasks/main.yml [Get current version number of clouctl]

Add the Authorization Bearer header to the api calls with the token you have created

- block:

  - name: Get current version number of clouctl
    uri:
      url: https://api.github.com/repos/IBM/cloud-pak-cli/releases/latest
      return_content: yes
      method: GET
      status_code: 200
      headers:
        Authorization: "Bearer <your personal access token>
    register: _cloudctl_version

Rebuild the cloud-pak-deployer after this modification

./cp-deploy.sh build

IBM / cloud-pak-deployer

API rate limit exceeded for unauthenticated API calls #562

Root cause

Workaround