actions / action-versions

MIT License
17 stars 6 forks source link

Documentation for using action-versions on self-hosted runners #10

Closed alec-drw closed 5 months ago

alec-drw commented 1 year ago

Hello! This project helps address many issues I have seen regarding rate limiting on self-hosted runners when checking out certain actions on GHES, even when using the "local" actions organization that has the default actions. However, there is no documentation on how to make use of this project - can the README or some documentation provide a high level "how to" set this up on runners built and maintained externally from the hosted runners?

TingluoHuang commented 1 year ago

Depends on where the rate limiting coming from, this cache might not work for you. Based on my understanding, there are 2 potential API calls.

I think it might be better to figure out which case you are seeing before adopting the cache in this repository.

alec-drw commented 1 year ago

@TingluoHuang it seems some jobs get secondary rate limited, for example we have a workflow that spawns ~500 jobs and makes at least 468 API calls to our GHES and we have a secondary rate limit at 500 rps

Error: API rate limit exceeded while resolving action `actions/checkout@v4`.. API rate limit exceeded while resolving action `actions/download-artifact@v3`.. API rate limit exceeded while resolving action `actions/upload-artifact@v3`.. API rate limit exceeded while resolving action `aws-actions/configure-aws-credentials@v4`.
##[debug]GitHub.DistributedTask.WebApi.UnresolvableActionDownloadInfoException: API rate limit exceeded while resolving action `actions/checkout@v4`.. API rate limit exceeded while resolving action `actions/download-artifact@v3`.. API rate limit exceeded while resolving action `actions/upload-artifact@v3`.. API rate limit exceeded while resolving action `aws-actions/configure-aws-credentials@v4`.
##[debug]   at GitHub.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
##[debug]   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
##[debug]   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
##[debug]   at GitHub.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable`1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable`1 queryParameters, Object userState, CancellationToken cancellationToken)
##[debug]   at GitHub.Runner.Worker.ActionManager.GetDownloadInfoAsync(IExecutionContext executionContext, List`1 actions)
##[debug]   at GitHub.Runner.Worker.ActionManager.GetDownloadInfoAsync(IExecutionContext executionContext, List`1 actions)
##[debug]   at GitHub.Runner.Worker.ActionManager.PrepareActionsRecursiveAsync(IExecutionContext executionContext, PrepareActionsState state, IEnumerable`1 actions, Int32 depth, Guid parentStepId)
##[debug]   at GitHub.Runner.Worker.ActionManager.PrepareActionsAsync(IExecutionContext executionContext, IEnumerable`1 steps, Guid rootStepId)
##[debug]   at GitHub.Runner.Worker.JobExtension.InitializeJob(IExecutionContext jobContext, AgentJobRequestMessage message)

So putting the official actions on the runner may get us around this

alec-drw commented 1 year ago

Found the commit that adds support for this in runners (https://github.com/actions/runner/commit/f57ecd8e3c618e1723cdf02565e5f4da188776a4). My approach is to create a PV, mount it at /opt/hostedactioncache and populate that directory with the actions using the scripts here, and set ACTIONS_RUNNER_ACTION_ARCHIVE_CACHE to that location.

Going to give this a try Monday, if this sounds way off please let me know! Happy to write something for the ARC repo or similar based on the outcome.

TingluoHuang commented 1 year ago

@alec-drw you are getting API rate limit exceeded while resolving action, the cache will help you skip the actual tarball download, but it won't skip the resolving action part, which means you will still have chance to hit rate limiting. 😢

alec-drw commented 11 months ago

@TingluoHuang Thanks for clarifying - regarding the ticket, is providing usage documentation something you still may do? I've noticed that even with my changes noted above the logs do not point to it checking the cache before downloading the action

mildmojo commented 8 months ago

I think I have the same question; I can't tell what this repository is, or how it's used for a self-hosted runner installation.

The promise of caching common actions sounds great, though, if it's not something built into the runner already!

TingluoHuang commented 5 months ago

https://github.com/actions/action-versions/blob/main/README.md#how-to-use-this-in-the-self-hosted-runner