search

projectsyn / modulesync-control

Archived. We now manage component templates via https://github.com/projectsyn/commodore-component-template. ModuleSync configuration and template repository for Commodore components
0 stars 0 forks source link

Pipeline runs into rate limits #48

Closed mhutter closed 2 years ago

mhutter commented 2 years ago 
Syncing 'appuio/component-openshift4-monitoring'
Cloning repository fresh from 'git@github.com:appuio/component-openshift4-monitoring.git'
bundler: failed to load command: msync (/opt/hostedtoolcache/Ruby/3.0.2/x64/bin/msync)
/opt/hostedtoolcache/Ruby/3.0.2/x64/lib/ruby/gems/3.0.0/gems/octokit-4.21.0/lib/octokit/response/raise_error.rb:14:
in `on_complete': 
POST https://api.github.com/repos/appuio/component-openshift4-monitoring/pulls: 403 
- You have exceeded a secondary rate limit and have been temporarily blocked from content creation. Please retry your request again later. 
// See: https://docs.github.com/rest/overview/resources-in-the-rest-api#secondary-rate-limits (Octokit::Forbidden)
ccremer commented 2 years ago

I'm not really sure what we can do here. The user executing the sync is already using a PAT to get more API requests, but I doubt that's really the problem here. This guide here explains how to deal with secondary rate limiting. This would need to be implemented in modulesync itself (e.g. no concurrent requests, wait 1sec between requests, respect Retry-After header etc). Given the low maintenance effort in modulesync we could wait forever...

What we could do is to not run the sync on master branch for every repository. One would have to trigger a run manually via workflow dispatch and limit the range of repositories by setting a filter/pattern.

simu commented 2 years ago

Closing this since we're migrating away from modulesync. This should be addressed by the new setup which runs commodore component sync, since that command waits 1-2 seconds between creating PRs.