Please find attached the PR which is a rework of the gather_github_repositories.rb task. There are several enhancements to the task which include:
Rather than using Octokit, the task now directly interacts with Github's REST API by firing off HTTP requests. This was done because the task supports both authenticated & unauthenticated use-cases. In the case of unauthenticated, you are unable to use Octokit thus increasing the 'complexity' of the script by having to implement additional methods that would interact with the API by firing off HTTP requests. By directly interacting with Github's REST API for both authenticated and unauthenticated requests, this simplifies a lot of the logic as the only difference is passing the access token in the header thus allowing the behavior to be abstracted to only use one method .
In the case where using Octokit to return repositories belonging to a GithubAccount entity (basically in most scenarios this means that the access token provided is not owned by the account else the String entity would be passed), this would only return the user's public repositories. This becomes a problem in cases where the access token which was used to initialize the Octokit client may have access to a private repository owned by the (other) account. To combat this, we originally invoked two API calls and did a diff as shown in the following snippet:
# when calling client.repos and passing in a name, it will return only public repositories
# even if the client is associated with the user's token
# since we have a valid access token; we will call client.repos
# also call client.repos with the name of the github account provided
# then extract out all the repos which belong to the github account provided
(client.repos + client.repos(name)).uniq
Not only did this result in the task taking longer it also wasted the precious API Request quota limit... By directly interacting with the REST API, we are now able to avoid taking this 'hacky' route as the route returns all the repositories belonging to the user including ones that the key has access to.
More documentation...
Bug Fixes
All the regex associated with GithubProject entities have been updated to fix a bug where repository names which start with a . (period) would fail to be parsed.
Other
The GithubAccount and GithubProject entity tasks had their pretty names updated to Enrich Github XXX to follow the naming convention of the other enrichment tasks.
Hi team,
Please find attached the PR which is a rework of the
gather_github_repositories.rbtask. There are several enhancements to the task which include:Rather than using Octokit, the task now directly interacts with Github's REST API by firing off HTTP requests. This was done because the task supports both authenticated & unauthenticated use-cases. In the case of unauthenticated, you are unable to use Octokit thus increasing the 'complexity' of the script by having to implement additional methods that would interact with the API by firing off HTTP requests. By directly interacting with Github's REST API for both authenticated and unauthenticated requests, this simplifies a lot of the logic as the only difference is passing the access token in the header thus allowing the behavior to be abstracted to only use one method .
In the case where using Octokit to return repositories belonging to a
GithubAccountentity (basically in most scenarios this means that the access token provided is not owned by the account else theStringentity would be passed), this would only return the user's public repositories. This becomes a problem in cases where the access token which was used to initialize the Octokit client may have access to a private repository owned by the (other) account. To combat this, we originally invoked two API calls and did a diff as shown in the following snippet:Not only did this result in the task taking longer it also wasted the precious API Request quota limit... By directly interacting with the REST API, we are now able to avoid taking this 'hacky' route as the route returns all the repositories belonging to the user including ones that the key has access to.
Bug Fixes
GithubProjectentities have been updated to fix a bug where repository names which start with a.(period) would fail to be parsed.Other
GithubAccountandGithubProjectentity tasks had their pretty names updated toEnrich Github XXXto follow the naming convention of the other enrichment tasks.Pending specs
Thank you.
Best regards, Maxim