babblebey
commented
1 month ago We are already paginating and in fact we split requests into chunks, the issue here is that we've got a query that possibly break the GitHub GraphQL API Node Limit... this one's my bad 🫣
But, 1,010,000 possible nodes! How did we get here??? 🤔
Examine our Query and Calculate
query getAssociatedPRs {
repository([name], [owner]) {
commit1: object(oid: [sha]) {
...Fields
}
// ....other commits minimized
commit100: object(oid: [sha]) {
...Fields
}
}
}
fragment Fields on Commit {
oid
associatedPullRequests(first: 100) {
pageInfo {
endCursor
hasNextPage
}
nodes {
//...otherBaseFields minimized
labels(first: 100) {
nodes {
id
url
name
color
}
}
mergeable
canBeRebased
changedFiles
mergedAt
isDraft
mergedBy {
login
avatarUrl
url
}
commits {
totalCount
}
}
}
}100 = 100 Commits Nodes + 100 x 100 = 10,000 AssociatedPRs Nodes + 100 x 100 x 100 = 1,000,000 AssociatedPRs Label Nodes
= 1,010,000 Total Possible Nodes
Explainer
- We are making request to 100 commits in one call, this mean the API connects to a total of each of the 100 commits and that calculate: 100 Commits Nodes
- We are getting 100 associatedPRs per commits so this means for all the 100 commits, this means the API is requesting a total of: 10,000 AssociatedPRs Nodes
- We are also getting 100 labels per associatedPRs (on each commits), the API is requesting a total of: 1,000,000 AssocatedPRs Label Nodes
- Total Requested Nodes: 1,010,000
Proposed Fix
We currently split the context.commit into chunks of 100 and make a parallel call to the graphQL API; this process involves building the graphql query (writing the commit1 - commit100 fragment in one call) for each chunks and making the call.
In order to accommodate the <= 500,000 node limits, I will reduce the label requested first value to 40 this will help bring the requested nodes per call to 410,000.
😉
AntonyF-Andreani
github-actions[bot]
i just hit this error when releasing a package that hasnt had activity in quite some time other than renovate dependency update PRs. the last release of this package was in 2021, so i imagine there are lots of dependency update commits and associated PRs since then. i imagine this is another scenario where pagination could be necessary.
error output: