Open samrocketman opened 5 years ago
query {
repository(owner: "endless-sky", name: "endless-sky") {
pullRequest(number: 6669) {
changedFiles
files(first: 100) {
file: nodes {
name: path
}
}
}
}
}
query {
repository: repository(owner: "samrocketman", name: "blog") {
commit: object(expression: "48419771766c593d26492d1f0bb9889d940ace18") {
... on Commit {
changedFilesIfAvailable
relatedPRs: associatedPullRequests(first: 100) {
totalCount
pr: nodes {
files(first: 100) {
file: nodes {
name: path
}
}
}
}
}
}
}
}
Query a repository for pull requests, branches, and tags. Retrieve associated contributor metadata and some Git metadata.
import static net.gleske.jervis.tools.AutoRelease.getScriptFromTemplate
import static net.gleske.jervis.tools.SecurityIO.avoidTimingAttack as delayForMillis
import static net.gleske.jervis.tools.YamlOperator.writeObjToYaml
import net.gleske.jervis.remotes.creds.EphemeralTokenCache
import net.gleske.jervis.remotes.creds.GitHubAppCredential
import net.gleske.jervis.remotes.creds.GitHubAppRsaCredentialImpl
import net.gleske.jervis.remotes.GitHubGraphQL
String githubOwner = 'samrocketman'
String githubRepo = 'jervis'
EphemeralTokenCache tokenCred = new EphemeralTokenCache('src/test/resources/rsa_keys/good_id_rsa_4096')
GitHubAppRsaCredentialImpl rsaCred = new GitHubAppRsaCredentialImpl(
'173962',
{-> new File('../jervis-jenkins-as-a-service.2023-06-29.private-key.pem').text }
)
rsaCred.owner = 'samrocketman'
GitHubAppCredential github_app = new GitHubAppCredential(rsaCred, tokenCred)
github_app.scope = [permissions: [contents: 'read']]
github_app.ownerIsUser = true
GitHubGraphQL github = new GitHubGraphQL()
github.credential = github_app
String query_template = '''\
query PrsBranchesTags(
<% if(pullsHasNextPage) { %>\\$pullsEndCursor: String = null,
<% } %><% if(headsHasNextPage) { %>\\$headsEndCursor: String = null,
<% } %><% if(tagsHasNextPage) { %>\\$tagsEndCursor: String = null,
<% } %>\\$owner: String = "",
\\$repo: String = "",
\\$first: Int = 100) {
repository(owner: \\$owner, name: \\$repo) {
<% if(pullsHasNextPage) { %>pulls: pullRequests(
after: \\$pullsEndCursor,
first: \\$first,
states: OPEN,
orderBy: {field: UPDATED_AT, direction: DESC}) {
pageInfo {
...paginate
}
totalCount
ref: nodes {
name: number
author {
login
}
baseRef {
...ref
}
headRef {
...ref
}
}
}
<% } %><% if(headsHasNextPage) { %>heads: refs(
after: \\$headsEndCursor,
first: \\$first,
refPrefix: "refs/heads/") {
pageInfo {
...paginate
}
totalCount
ref: nodes {
...ref
}
}
<% } %><% if(tagsHasNextPage) { %>tags: refs(
after: \\$tagsEndCursor,
first: \\$first,
refPrefix: "refs/tags/") {
<% /*orderBy: {field: TAG_COMMIT_DATE, direction: DESC}*/ %>
pageInfo {
...paginate
}
totalCount
ref: nodes {
...ref
}
}<% } %>
}
}
fragment paginate on PageInfo {
hasNextPage
endCursor
}
fragment ref on Ref {
prefix
name
target {
...commit
}
}
fragment commit on Commit {
author {
date
email
name
user {
login
}
}
committer {
date
email
name
user {
login
}
}
sha: oid
}
'''.trim()
Map binding = [
pullsHasNextPage: true,
headsHasNextPage: true,
tagsHasNextPage: true
]
Map variables = [owner: githubOwner, repo: githubRepo, first: 100]
Map data = [pullsCount: 0, headsCount: 0, tagsCount: 0].withDefault { [] }
List errors = []
Integer queryCount = 0
Integer retryCount = 0
Integer retryLimit = 30
println "Discover PRs, Branches, and Tags on:\n${variables.owner}/${variables.repo}"
// do-while loop in Groovy
while({->
queryCount++
Map response
try {
response = github.sendGQL(getScriptFromTemplate(query_template, binding), variables)
// max throttle
variables.first = 100
} catch(Exception httpError) {
if(retryCount > retryLimit) {
throw httpError
}
// back off object count because we failed
variables.first = Math.max(10, variables.first / 2 as Integer)
// wait at least 200ms but not more than 3000ms (random in-between)
// before attempting to retry
delayForMillis(200) {
delayForMillis(-3000, {->})
}
// retry the last query since an HTTP error occurred
return true
}
if('errors' in response.keySet()) {
errors = response.errors
return false
}
if(!response?.data?.repository) {
return false
}
Map refs = response.data.repository
['pulls', 'heads', 'tags'].findAll { String ref ->
ref in refs.keySet()
}.each { String ref ->
if(!data["${ref}Count".toString()]) {
data["${ref}Count".toString()] = refs[ref].totalCount
}
binding["${ref}HasNextPage".toString()] = refs[ref]?.pageInfo.hasNextPage ?: false
if(binding["${ref}HasNextPage".toString()]) {
variables["${ref}EndCursor".toString()] = refs[ref].pageInfo.endCursor
} else {
variables.remove("${ref}EndCursor".toString())
}
if(!data["${ref}Count".toString()]) {
return
}
data[ref] += refs[ref]?.ref ?: []
}
// if any refs have a next page return true to query again
binding.findAll { k, v ->
v in Boolean
}.collect { k, v -> v }.any { it }
}()) continue
println "Query count: ${queryCount}"
if(errors) {
throw new Exception(github.objToJson(errors: errors))
}
Map output = [
'Total pull requests': data.pullsCount,
'Total branches': data.headsCount,
'Total tags': data.tagsCount,
'First pull request': data.pulls.find(), // get first item or null
'First branch': data.heads.find(),
'First tag': data.tags.find()
]
println writeObjToYaml(output)
GitHub app authentication:
EphemeralTokenCache
GitHubAppRsaCredentialImpl
GitHubAppCredential
(both credential and an API client)API client
Utility classes
AutoRelease
SecurityIO
security featuresYamlOperator
Smarter branch detection
Use case: find only branches which have a
.jervis.yml
or.travis.yml
at their root.GitHub.getJervisBranches()
should only return a listing of known good branches that match.Search all branches and get a listing of their root file tree. Technically, you can get the contents of each file in the root folder but you can't limit it to
.jervis.yml
or.travis.yml
. Because of this, I feel it's a bit too expensive of a call.Smarter YAML retrieval
Use case: Get both
.travis.yml
and.jervis.yml
in a single API call for a branch.GitHub.getJervisYaml(String branch)
will attempt to get.jervis.yml
, fall back to.travis.yml
, and if it finds neither then return an empty string.