Cache terragrunt versions json

[applevladko]

Is your feature request related to a problem? Please describe. When using tenv in CI/CD pipelines to check constraints, if the required version is not installed locally in the Docker container, tenv attempts to fetch the Terragrunt release list from the GitHub API (https://api.github.com/repos/gruntwork-io/terragrunt/releases). This can lead to issues when running multiple pipelines concurrently, as it may trigger GitHub's anti-DDoS protection, resulting in temporary bans and pipeline failures.

Describe the solution you'd like Would be great implementing a caching mechanism for the tenv tg list command:

Cache the current Terragrunt releases list locally (not only installed but all available) When checking constraints: a. First, attempt to satisfy the constraint using the cached version list. b. If a newer version is required, only then make a request to the GitHub API (e.g., tenv tg remote-list). This approach would significantly reduce the number of API calls to GitHub, mitigating the risk of temporary bans and improving pipeline performance.

Describe alternatives you've considered An alternative solution could involve using a cached version of the GitHub API release data, possibly updated at regular intervals.

Additional context Thanks!

[kvendingoldo]

Good idea!

[dvaumoron]

A cache will not work out of the box, as each pipeline use different container, however you can use TENV_GITHUB_TOKEN to increase drastically Github rate limit

[applevladko]

I thought implementing the following logic for version checking in tenv:

Current Approach:

Check installed versions for constraints If not satisfied, fetch from remote JSON (GitHub API)

Proposed Approach:

Check installed versions for constraints If not satisfied, check a locally cached JSON file (updated periodically with new versions) If still not satisfied, fetch from remote JSON (GitHub API)

thanks for suggestion with token, will try it

[dvaumoron]

As already said a prebuild cache will not work, in your example, when the cache does not contains a matching version (likely since terragrunt seem to release more often than tenv), the problem causing too much call will persist (updating the cache after a successful call will only impact the current container).

[kvendingoldo]

@dvaumoron you're right, but github/gitlab allow to use cache between jobs; In such case, the cached file can make sense