Closed danieldelcore closed 1 year ago
Hm, I'm not sure if this is the rate limit of your org or the rate limit of the changesets action bot, @Andarist do you happen to know?
I don't really think it's possible for me to go over my personal rate limit, so I assume it's due to changesets action bot. I'm also experiencing this error, for the second time now and I only have like 4 repos and barely ran the action in like days.
Maybe related to this? https://github.com/semantic-release/semantic-release/issues/2204
Hm, I'm not sure if this is the rate limit of your org or the rate limit of the changesets action bot, @Andarist do you happen to know?
To the best of my understanding it's related to this specific token. Looking into the GitHub docs we can see that the primary rate limit for this kind of a token is 1000 requests / hour: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#requests-from-github-actions
However, the report here is about the "secondary rate limit" and we can read here that:
Secondary rate limits are not intended to interfere with legitimate use of the API. Your normal rate limits should be the only limit you target.
So my line of thinking is that one should always first receive an error about the primary rate limit being exceeded and only if they continue to make requests then the secondary rate limit can kick in. This doesn't quite match the provided information about this issue.
I'm not sure what to do here because I lack the required information. Perhaps somebody could create a GitHub support ticket about this to get a better understanding of what could have caused that?
I'm seeing this too. Might make sense to include retries, respecting the Retry-After header?
That's probably a good idea - it would be great to add some logs for this error first and log the value returned in this retry-after
header. I wonder what's the best way to handle this because if this value is too big then it might not be a feasible solution for the action to wait for too long, remember that an idle, but running, action could still cost somebody money.
We are hitting a secondary rate limit for this action as well, and it looks related to search API usage.
We have a theory that it relates to this part of the code: https://github.com/changesets/action/blob/main/src/run.ts#L295-L297
The query it builds on L294 looks like the same one that triggers the request hit by secondary rate limits:
repo:${repo}+state:open+head:${versionBranch}+base:${branch}
https://api.github.com/search/issues?q=
repo:[REDACTED]/[REDACTED]+state:open+head:changeset-release/master+base:master
The full log is attached before for reference[1].
Absolutely looks like the same issue mentioned above[2].
Haven't looked into how to avoid using the search API to figure out if there is an existing PR to update or if a new one needs to be created, and doesn't look like semantic-release has worked out a solution either.
[1] Full error log:
[HttpError]: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.
at /home/runner/work/_actions/changesets/action/v1/dist/index.js:718:926
at processTicksAndRejections (internal/process/task_queues.js:97:5) {
status: 403,
headers: {
'access-control-allow-origin': '*',
'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
connection: 'close',
'content-encoding': 'gzip',
'content-security-policy': "default-src 'none'",
'content-type': 'application/json; charset=utf-8',
date: 'Thu, 29 Sep 2022 12:55:19 GMT',
'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
server: 'GitHub.com',
'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
'transfer-encoding': 'chunked',
vary: 'Accept-Encoding, Accept, X-Requested-With',
'x-content-type-options': 'nosniff',
'x-frame-options': 'deny',
'x-github-media-type': 'github.v3; format=json',
'x-github-request-id': '0780:1463:1903E5B:3347AE5:633595B7',
'x-ratelimit-limit': '30',
'x-ratelimit-remaining': '30',
'x-ratelimit-reset': '1664456179',
'x-ratelimit-resource': 'search',
'x-ratelimit-used': '1',
'x-xss-protection': '0'
},
request: {
method: 'GET',
url: 'https://api.github.com/search/issues?q=repo%3A[REDACTED]%2F[REDACTED]+state%3Aopen+head%3Achangeset-release%2Fmaster+base%3Amaster',
headers: {
accept: 'application/vnd.github.v3+json',
'user-agent': 'octokit-core.js/3.1.0 <environment undetectable>',
authorization: 'token [REDACTED]'
},
request: { agent: [Agent], hook: [Function: bound bound r] }
},
documentation_url: 'https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits'
}
[2] https://github.com/changesets/action/issues/192#issuecomment-1204825165
Given the action uses Octokit already, could this be solved by just using https://github.com/octokit/plugin-retry.js/?
Was anyone able to figure out the underlying cause?
It seems that most* people facing this issue encounter the following headers:
{
'x-ratelimit-limit': '30',
'x-ratelimit-remaining': '29',
'x-ratelimit-resource': 'search',
'x-ratelimit-used': '1',
}
These headers imply that we are far from reaching the api rate limit. Although, the error message does say that it is a secondary rate limit we're exceeding. I'm not sure what might be causing that either, since this issue happens randomly, sometimes even when invoking the api for the first time in days.
Reading through the docs and linked comments leads me to believe that this is a bug in the Github search api.
*sample 403 responses when encountering this issue:
I'm seeing many instances of this error as well.
@DanRigby maybe can shed some light on this?
Yeah, I've also seen it more often lately - and also at very suspicious times (when there was almost no other activity in the whole organization).
I'd very much like to do something about it - but quite frankly, I'm not sure how to handle this in Changesets because it mostly seems to be out of our control.
@Andarist What if we simply retry the search query multiple times whenever we encounter a 403
? This will make us more resilient against the flakiness of what seems like a bug (on github's side). I'm not even sure we have to wait the full retry-after
duration. In the 403
responses I've encountered, the retry-after
header wasn't even included.
Admittedly, this isn't a great solution. We don't really know if it will solve the underlying issue, one which does not lie in this repo at that. Maybe I'm just desperate because the publishing workflow is my most vulnerable ci pipeline.
Octokit provides a "throttle and retry" plugin[1] that:
Implements all recommended best practices to prevent hitting secondary rate limits.
It would be interesting if it is possible to wrap the Octokit client[2] with the plugin using the method outlined in the @actions/github
toolkit[3].
[1] https://github.com/octokit/plugin-throttling.js [2] https://github.com/changesets/action/blob/main/src/run.ts#L272 [3] https://github.com/actions/toolkit/tree/main/packages/github#extending-the-octokit-instance
edit: I saw a comment discussing plugin-retry.js
, but that one explicitly does not retry 403
errors, which is why the throttling plugin looks promising.
We reached the secondary limit as well three times in aws-sdk-js-codemod repo. A manual retry of the action fixes it though.
Example run: https://github.com/awslabs/aws-sdk-js-codemod/actions/runs/3858163665/jobs/6576401193
@Andarist do you want to try out using https://github.com/octokit/plugin-throttling.js/ and https://github.com/octokit/plugin-retry.js/ ? I can help with that
Any updates regarding this issue? I'm seeing this as well.
I've been seeing this recently too - is there an update on a potential fix? Or has anyone found a workaround (other than a manual retry of the action)?
Any updates on this, it is very troublesome for our release schedule.
@Andarist I took a stab at this issue. Mind looking at the PR #286 and see if it is worth exploring further ?
I just hit this same issue today. PS hi @danieldelcore !
I just hit this same issue today. PS hi @danieldelcore !
Can confirm I updated to new version, and the limit was hit again unfortunately 😢 @Andarist
I just hit this same issue today. PS hi @danieldelcore !
Can confirm I updated to new version, and the limit was hit again unfortunately cry @Andarist
Please provide the CI logs for the run, this is a hard issue to reproduce at will and it would help immensely to get the logs for a failure.
I just hit this same issue today. PS hi @danieldelcore !
Can confirm I updated to new version, and the limit was hit again unfortunately cry @Andarist
Please provide the CI logs for the run, this is a hard issue to reproduce at will and it would help immensely to get the logs for a failure.
@varl I have a public repo where I test this stuff. This is the job where it failed again https://github.com/blissful-group/math-lib/actions/runs/4913661788/jobs/8774090637
edit added step configuration
- name: Batch changesets to pull request 📫...
id: changesets
uses: changesets/action@v1.4.2
Hm, weirdly there are no logs at all coming from the throttling plugin that we added - as if that logic was never called.
Note that it is still possible to hit secondary rate limits since this is setup so that it only retries twice before failing -- this is what I was "hoping" to see in the logs, that it retried and then failed.
But I don't see any of the console-based logging from the changesets action in the job.
@Andarist, I think we'd need to hook up logging from core1 to see if the throttling plugin is doing what it should. Should I start doing that or would you prefer to do it ? Either way is cool with me.
edit: First pass could be to do it for the throttling-plugin specifically and one other place that is always run just to be able to verify that logging is functioning as expected, and then if logging behaves as intended, we'd implement it throughout in a second pass.
@Andarist, I think we'd need to hook up logging from corehttps://github.com/changesets/action/pull/286#discussion_r1185942161 to see if the throttling plugin is doing what it should. Should I start doing that or would you prefer to do it ? Either way is cool with me.
If you could do this that would definitely help us to ship this faster.
edit: First pass could be to do it for the throttling-plugin specifically and one other place that is always run just to be able to verify that logging is functioning as expected, and then if logging behaves as intended, we'd implement it throughout in a second pass.
Sounds fine, although we don't have a lot of logs really so I think it's doable in a single pass without much more time put into it.
Thanks for looking into this again @Andarist / @varl 👍
@Andarist alright, I did it in one pass. I skipped scripts/release.js
as I wasn't sure about pulling @actions/core
into that file. It looked like it's not run from within the action, but rather as part of the workflow itself for releasing the action.
@Blissful89 If you manage to spot another error with 1.4.3 of the action, I'd love to see the logs.
@Blissful89 If you manage to spot another error with 1.4.3 of the action, I'd love to see the logs.
Will try it out soon again!
@Blissful89 If you manage to spot another error with 1.4.3 of the action, I'd love to see the logs.
@varl I've included a snippet, cannot link it like last time because this is a failing prod release on a private repo. Hopefully it helps
2023-05-09T09:53:10.1646859Z ##[group]Run changesets/action@v1.4.3
2023-05-09T09:53:10.1647109Z with:
2023-05-09T09:53:10.1647300Z setupGitUser: true
2023-05-09T09:53:10.1647543Z createGithubReleases: true
2023-05-09T09:53:10.1647781Z env:
2023-05-09T09:53:10.1648134Z GITHUB_TOKEN: ***
2023-05-09T09:53:10.1648417Z PNPM_HOME: /home/runner/setup-pnpm/node_modules/.bin
2023-05-09T09:53:10.1648741Z NPM_CONFIG_USERCONFIG: /home/runner/work/_temp/.npmrc
2023-05-09T09:53:10.1649032Z NODE_AUTH_TOKEN: XXXXX-XXXXX-XXXXX-XXXXX
2023-05-09T09:53:10.1649288Z ##[endgroup]
2023-05-09T09:53:10.3271629Z setting git user
2023-05-09T09:53:10.3334860Z [command]/usr/bin/git config user.name "github-actions[bot]"
2023-05-09T09:53:10.3454790Z [command]/usr/bin/git config user.email "github-actions[bot]@users.noreply.github.com"
2023-05-09T09:53:10.3499147Z setting GitHub credentials
2023-05-09T09:53:10.3787013Z [command]/usr/bin/git checkout changeset-release/main
2023-05-09T09:53:10.3828054Z error: pathspec 'changeset-release/main' did not match any file(s) known to git
2023-05-09T09:53:10.3858769Z [command]/usr/bin/git checkout -b changeset-release/main
2023-05-09T09:53:10.3899878Z Switched to a new branch 'changeset-release/main'
2023-05-09T09:53:10.3929912Z [command]/usr/bin/git reset --hard 2a86a1c69f09c36ef53c3e5d7400d57217c7f42c
2023-05-09T09:53:10.3990384Z HEAD is now at 2a86a1c Merge pull request #103 from ChargePoint/develop
2023-05-09T09:53:10.4093203Z [command]/opt/hostedtoolcache/node/18.16.0/x64/bin/node /home/runner/work/cp-kit/cp-kit/node_modules/.pnpm/@changesets+cli@2.26.1/node_modules/@changesets/cli/bin.js version
2023-05-09T09:53:11.1444176Z 🦋 All files have been updated. Review them and commit at your leisure
2023-05-09T09:53:11.2156609Z [command]/usr/bin/git status --porcelain
2023-05-09T09:53:11.2221853Z D .changeset/hungry-monkeys-dress.md
2023-05-09T09:53:11.2222559Z D .changeset/neat-chairs-beam.md
2023-05-09T09:53:11.2223375Z D .changeset/ninety-eels-invite.md
2023-05-09T09:53:11.2223966Z D .changeset/olive-eagles-speak.md
2023-05-09T09:53:11.2224744Z D .changeset/perfect-yaks-marry.md
2023-05-09T09:53:11.2225511Z D .changeset/tiny-dolphins-try.md
2023-05-09T09:53:11.2227470Z D .changeset/unlucky-berries-brush.md
2023-05-09T09:53:11.2229089Z M CHANGELOG.md
2023-05-09T09:53:11.2230184Z M package.json
2023-05-09T09:53:11.2247856Z [command]/usr/bin/git add .
2023-05-09T09:53:11.2332950Z [command]/usr/bin/git commit -m Version Packages
2023-05-09T09:53:11.2418864Z [changeset-release/main f86cf9e] Version Packages
2023-05-09T09:53:11.2419639Z 9 files changed, 16 insertions(+), 36 deletions(-)
2023-05-09T09:53:11.2420589Z delete mode 100644 .changeset/hungry-monkeys-dress.md
2023-05-09T09:53:11.2421257Z delete mode 100644 .changeset/neat-chairs-beam.md
2023-05-09T09:53:11.2422119Z delete mode 100644 .changeset/ninety-eels-invite.md
2023-05-09T09:53:11.2422776Z delete mode 100644 .changeset/olive-eagles-speak.md
2023-05-09T09:53:11.2423624Z delete mode 100644 .changeset/perfect-yaks-marry.md
2023-05-09T09:53:11.2425006Z delete mode 100644 .changeset/tiny-dolphins-try.md
2023-05-09T09:53:11.2426033Z delete mode 100644 .changeset/unlucky-berries-brush.md
2023-05-09T09:53:11.2449302Z [command]/usr/bin/git push origin HEAD:changeset-release/main --force
2023-05-09T09:53:11.3377565Z /home/runner/work/_actions/changesets/action/v1.4.3/dist/index.js:186
2023-05-09T09:53:11.3393153Z "use strict";Object.defineProperty(exports,"__esModule",{value:!0}),exports.request=void 0;var e=require("@octokit/endpoint"),t=require("universal-user-agent"),r=require("is-plain-object"),s=n(require("node-fetch")),o=require("@octokit/request-error");function n(e){return e&&e.__esModule?e:{default:e}}const u="6.2.3";function i(e){return e.arrayBuffer()}function a(e){const t=e.request&&e.request.log?e.request.log:console;((0,r.isPlainObject)(e.body)||Array.isArray(e.body))&&(e.body=JSON.stringify(e.body));let n,u,i={};return(e.request&&e.request.fetch||globalThis.fetch||s.default)(e.url,Object.assign({method:e.method,body:e.body,headers:e.headers,redirect:e.redirect},e.request)).then(async r=>{u=r.url,n=r.status;for(const e of r.headers)i[e[0]]=e[1];if("deprecation"in i){const r=i.link&&i.link.match(/<([^>]+)>; rel="deprecation"/),s=r&&r.pop();t.warn(`[@octokit/request] "${e.method} ${e.url}" is deprecated. It is scheduled to be removed on ${i.sunset}${s?`. See ${s}`:""}`)}if(204!==n&&205!==n){if("HEAD"===e.method){if(n<400)return;throw new o.RequestError(r.statusText,n,{response:{url:u,status:n,headers:i,data:void 0},request:e})}if(304===n)throw new o.RequestError("Not modified",n,{response:{url:u,status:n,headers:i,data:await d(r)},request:e});if(n>=400){const t=await d(r);throw new o.RequestError(c(t),n,{response:{url:u,status:n,headers:i,data:t},request:e})}return d(r)}}).then(e=>({status:n,url:u,headers:i,data:e})).catch(t=>{if(t instanceof o.RequestError)throw t;if("AbortError"===t.name)throw t;throw new o.RequestError(t.message,500,{request:e})})}async function d(e){const t=e.headers.get("content-type");return/application\/json/.test(t)?e.json():!t||/^text\/|charset=utf-8$/.test(t)?e.text():i(e)}function c(e){return"string"==typeof e?e:"message"in e?Array.isArray(e.errors)?`${e.message}: ${e.errors.map(JSON.stringify).join(", ")}`:e.message:`Unknown error: ${JSON.stringify(e)}`}function f(e,t){const r=e.defaults(t);return Object.assign(function(e,t){const s=r.merge(e,t);if(!s.request||!s.request.hook)return a(r.parse(s));const o=(e,t)=>a(r.parse(r.merge(e,t)));return Object.assign(o,{endpoint:r,defaults:f.bind(null,r)}),s.request.hook(o,s)},{endpoint:r,defaults:f.bind(null,r)})}const l=f(e.endpoint,{headers:{"user-agent":`octokit-request.js/6.2.3 ${(0,t.getUserAgent)()}`}});exports.request=l;
2023-05-09T09:53:11.3398890Z
2023-05-09T09:53:11.3402276Z
2023-05-09T09:53:11.3402701Z a [HttpError]: You have exceeded a secondary rate limit. Please wait a few minutes before you try again.
2023-05-09T09:53:11.3403201Z at /home/runner/work/_actions/changesets/action/v1.4.3/dist/index.js:186:1303
2023-05-09T09:53:11.3404339Z at processTicksAndRejections (node:internal/process/task_queues:96:5) {
2023-05-09T09:53:11.3404762Z status: 403,
2023-05-09T09:53:11.3405007Z response: {
2023-05-09T09:53:11.3405767Z url: 'https://api.github.com/search/issues?q=repo%3AChargePoint%2Fcp-kit+state%3Aopen+head%3Achangeset-release%2Fmain+base%3Amain+is%3Apull-request',
2023-05-09T09:53:11.3406238Z status: 403,
2023-05-09T09:53:11.3406562Z headers: {
2023-05-09T09:53:11.3407144Z 'access-control-allow-origin': '*',
2023-05-09T09:53:11.3408598Z 'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
2023-05-09T09:53:11.3409365Z connection: 'close',
2023-05-09T09:53:11.3409733Z 'content-encoding': 'gzip',
2023-05-09T09:53:11.3410167Z 'content-security-policy': "default-src 'none'",
2023-05-09T09:53:11.3410612Z 'content-type': 'application/json; charset=utf-8',
2023-05-09T09:53:11.3411019Z date: 'Tue, 09 May 2023 09:53:11 GMT',
2023-05-09T09:53:11.3411573Z 'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
2023-05-09T09:53:11.3411960Z server: 'GitHub.com',
2023-05-09T09:53:11.3412435Z 'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
2023-05-09T09:53:11.3412862Z 'transfer-encoding': 'chunked',
2023-05-09T09:53:11.3413285Z vary: 'Accept-Encoding, Accept, X-Requested-With',
2023-05-09T09:53:11.3415273Z 'x-content-type-options': 'nosniff',
2023-05-09T09:53:11.3415696Z 'x-frame-options': 'deny',
2023-05-09T09:53:11.3416091Z 'x-github-api-version-selected': '2022-11-28',
2023-05-09T09:53:11.3416787Z 'x-github-media-type': 'github.v3; format=json',
2023-05-09T09:53:11.3417273Z 'x-github-request-id': '0800:2267:865FC2:1136761:645A1807',
2023-05-09T09:53:11.3417646Z 'x-ratelimit-limit': '30',
2023-05-09T09:53:11.3418006Z 'x-ratelimit-remaining': '30',
2023-05-09T09:53:11.3418417Z 'x-ratelimit-reset': '1683626051',
2023-05-09T09:53:11.3418771Z 'x-ratelimit-resource': 'search',
2023-05-09T09:53:11.3419126Z 'x-ratelimit-used': '1',
2023-05-09T09:53:11.3419460Z 'x-xss-protection': '0'
2023-05-09T09:53:11.3419723Z },
2023-05-09T09:53:11.3420008Z data: {
2023-05-09T09:53:11.3420575Z documentation_url: 'https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits',
2023-05-09T09:53:11.3421248Z message: 'You have exceeded a secondary rate limit. Please wait a few minutes before you try again.'
2023-05-09T09:53:11.3421586Z }
2023-05-09T09:53:11.3421862Z },
2023-05-09T09:53:11.3422101Z request: {
2023-05-09T09:53:11.3422388Z method: 'GET',
2023-05-09T09:53:11.3422972Z url: 'https://api.github.com/search/issues?q=repo%3AChargePoint%2Fcp-kit+state%3Aopen+head%3Achangeset-release%2Fmain+base%3Amain+is%3Apull-request',
2023-05-09T09:53:11.3423389Z headers: {
2023-05-09T09:53:11.3424100Z accept: 'application/vnd.github.v3+json',
2023-05-09T09:53:11.3424850Z 'user-agent': 'octokit-core.js/4.2.0 Node.js/16.16.0 (linux; x64)',
2023-05-09T09:53:11.3425256Z authorization: 'token [REDACTED]'
2023-05-09T09:53:11.3425543Z },
2023-05-09T09:53:11.3425801Z request: {
2023-05-09T09:53:11.3426053Z agent: Agent {
2023-05-09T09:53:11.3426362Z _events: [Object: null prototype] {
2023-05-09T09:53:11.3426695Z free: [Function (anonymous)],
2023-05-09T09:53:11.3427079Z newListener: [Function: maybeEnableKeylog]
2023-05-09T09:53:11.3427328Z },
2023-05-09T09:53:11.3427577Z _eventsCount: 2,
2023-05-09T09:53:11.3427877Z _maxListeners: undefined,
2023-05-09T09:53:11.3428156Z defaultPort: 443,
2023-05-09T09:53:11.3428513Z protocol: 'https:',
2023-05-09T09:53:11.3428857Z options: [Object: null prototype] { path: null },
2023-05-09T09:53:11.3429163Z requests: [Object: null prototype] {},
2023-05-09T09:53:11.3429550Z sockets: [Object: null prototype] {},
2023-05-09T09:53:11.3429953Z freeSockets: [Object: null prototype] {},
2023-05-09T09:53:11.3430255Z keepAliveMsecs: 1000,
2023-05-09T09:53:11.3430541Z keepAlive: false,
2023-05-09T09:53:11.3430767Z maxSockets: Infinity,
2023-05-09T09:53:11.3431045Z maxFreeSockets: 256,
2023-05-09T09:53:11.3431417Z scheduling: 'lifo',
2023-05-09T09:53:11.3431718Z maxTotalSockets: Infinity,
2023-05-09T09:53:11.3432304Z totalSocketCount: 0,
2023-05-09T09:53:11.3432905Z maxCachedSessions: 100,
2023-05-09T09:53:11.3433200Z _sessionCache: {
2023-05-09T09:53:11.3433485Z map: {
2023-05-09T09:53:11.3434823Z 'api.github.com:443:::::::::::::::::::::': [Buffer [Uint8Array]]
2023-05-09T09:53:11.3435156Z },
2023-05-09T09:53:11.3435859Z list: [ 'api.github.com:443:::::::::::::::::::::' ]
2023-05-09T09:53:11.3436261Z },
2023-05-09T09:53:11.3436556Z [Symbol(kCapture)]: false
2023-05-09T09:53:11.3436831Z },
2023-05-09T09:53:11.3437108Z hook: [Function: bound bound r]
2023-05-09T09:53:11.3437420Z }
2023-05-09T09:53:11.3437644Z }
2023-05-09T09:53:11.3437839Z }
Great, I see the problem. Thanks !
There were additional uses of octokit that I missed the first time around (:disappointed:), I've fixed it in #291.
There were additional uses of octokit that I missed the first time around (😞), I've fixed it in #291.
Nice. Unfortunately github is having issues right now see status page... so we can't really check it right now I guess?
Nice. Unfortunately github is having issues right now see status page... so we can't really check it right now I guess?
Yeah, and I broke a test on CI. Looking into it.
Nice. Unfortunately github is having issues right now see status page... so we can't really check it right now I guess?
Yeah, and I broke a test on CI. Looking into it.
Seems to be coming back online, you can probably just rerun the failed jobs
@marcovanharten-cpi @Blissful89 1.4.4 is live, if you want to try it again. :)
@marcovanharten-cpi @Blissful89 1.4.4 is live, if you want to try it again. :)
Did and so far so good! Maybe you can finally put this one to rest 😉
Hitting this in 2024. The relevant line:
Warning: SecondaryRateLimit detected for request GET /search/issues
What does this action do with issues?
Hitting this in 2024. The relevant line:
Warning: SecondaryRateLimit detected for request GET /search/issues
What does this action do with issues?
Of course, pull requests and issues are the same thing 😊
@WoodyWoodsta did you note if it retried the action in the logs ?
It should log according to this:
console.log(`Retrying after ${retryAfter} seconds!`);
It will only make 3 attempts (initial, then two retries) before failing. If it fails after 3 attempts, you need to manually re-run the job and it should go through (unless it's still secondary rate limited by github .. in that case it's a wait and play situation).
The fixes applied to this issue are mitigation strategies to make the issue less likely to appear, but it's not a perfect fix and as more people rely on this action, the more common it becomes.
Changing the retry count is another mitigation however it needs to be balanced, and we should respect the API responses for rate limiting as well1:
Continuing to make requests while you are rate limited may result in the banning of your integration.
@varl Yep, it's doing 3 retries and then failing.
Do I understand correctly that your API limits are used, not mine? The use of this action in our organisation is low such that if it were contributing to our API request limits we shouldn't be rate limited.
If so, is there a way the requests can be made via our limits?
@varl Yep, it's doing 3 retries and then failing.
Alright, then it's working as expected. That is good, at least.
Do I understand correctly that your API limits are used, not mine? The use of this action in our organisation is low such that if it were contributing to our API request limits we shouldn't be rate limited.
I just contributed with some work to help mitigate this specific issue.
To your point, the rate limits on the GitHub API are strict by design. If this was a "primary rate limit", using a different GITHUB_TOKEN
would (probably) help1.
However, the issue at hand is the "secondary rate limit", and from their docs I don't see mentions of secondary rate limits behaving differently in that case2.
If so, is there a way the requests can be made via our limits?
If you are on GitHub Enterprise the API limits are different, I presume that includes the secondary rate limits.
You could try forking the action to your organisation, and running it from there, e.g. uses: my-org/changesets-actions
in your workflows and see if that helps. Remember to update your fork from time-to-time.
cc: @WoodyWoodsta
as more people rely on this action, the more common it becomes
This was the basis for my question. But looking at the secondary rate limit docs, it's a bit hazy and I can see how we'd possibly be hitting the limit if this action is performing a lot of api requests.
@WoodyWoodsta Yeah, I'm just not sure if it's actually shared, or if it's based on something other than the GITHUB_TOKEN
, or if the API limits are so strict that even a single repo (or small organisation) can trip over the secondary rate limits. Maybe the API for searching is so slow that it takes more than 90s of CPU time per 60s real-time ...
It seems to me that it is probable is that we observe a usage effect: more people use it, so more people trip over the issue.
Thinking about it some more, if the GITHUB_TOKEN
and resources are shared, that doesn't make sense as that would grant the shared GITHUB_TOKEN
used by everyone access to all repos it's used in. The action creates commits and posts comments, so based on that it must use repo/org-local tokens to make the requests.
I dunno, man. I just hit that re-run button in the failed workflow. :sweat_smile:
You could try forking the action to your organisation, and running it from there, e.g.
uses: my-org/changesets-actions
in your workflows and see if that helps. Remember to update your fork from time-to-time.
This is what solved secondary rate limit
issue for me, thanks 🤝
Hey folks, I seem to be running into some rate limits when running
Create Release PR or Publish to npm
It looks my Org/Account hasn't hit a rate limit so I'm wondering if it has done so on the changesets org.
My action config looks like so: https://github.com/CodeshiftCommunity/CodeshiftCommunity/blob/main/.github/workflows/release.yml
Currently using changesets/cli 2.6.2
Failing build: https://github.com/CodeshiftCommunity/CodeshiftCommunity/runs/6961550448?check_suite_focus=true