search

JuliaWeb / GitHub.jl

A Julia package for interfacing with GitHub
MIT License
172 stars 60 forks source link

readme

GitHub.jl

Coverage Status

GitHub.jl provides a Julia interface to the GitHub API v3. Using GitHub.jl, you can do things like:

Here's a table of contents for this rather lengthy README:

1. Response Types

2. REST Methods

3. Authentication

4. Pagination

5. Handling Webhook Events

6. GitHub Enterprise

Response Types

GitHub's JSON responses are parsed and returned to the caller as types of the form G<:GitHub.GitHubType. Here's some useful information about these types:

Here's a table that matches up the provided GitHubTypes with their corresponding API documentation, as well as alternative identifying values:

type alternative identifying property link(s) to documentation
Owner login, e.g. "octocat" organizations, users
Repo full_name, e.g. "JuliaWeb/GitHub.jl" repositories
Commit sha, e.g. "d069993b320c57b2ba27336406f6ec3a9ae39375" repository commits
GitCommit sha, e.g. "d069993b320c57b2ba27336406f6ec3a9ae39375" raw git commits
Branch name, e.g. master repository branches
Content path, e.g. "src/owners/owners.jl" repository contents
Comment id, e.g. 162224613 commit comments, issue comments, PR review comments
Label name, e.g. bug issue labels
Status id, e.g. 366961773 commit statuses
PullRequest number, e.g. 44 pull requests
PullRequestFile filename, e.g. file1.txt pull request files
Issue number, e.g. 31 issues
Team id, e.g. 1 teams
Gist id, e.g. 0bace7cc774df4b3a4b0ee9aaa271ef6 gists
Review id, e.g. 1 reviews
Blob sha, e.g. "95c8d1aa2a7b1e6d672e15b67e0df4abbe57dcbe" raw git blobs
Tree sha, e.g. "78e524d5e979e326a7c144ce195bf94ca9b04fa0" raw git trees
Tag tag name, e.g. v1.0 git tags
References reference name, e.g. heads/master (note: omits leading refs/) references
Secrets secret name, e.g. TAGBOT_SECRET secrets
DeployKeys id, e.g., 12345 deploy keys

You can inspect which fields are available for a type G<:GitHubType by calling fieldnames(G).

REST Methods

GitHub.jl implements a bunch of methods that make REST requests to GitHub's API. The below sections list these methods (note that a return type of Tuple{Vector{T}, Dict} means the result is paginated).

Users and Organizations

method return type documentation
whoami() Owner get currently authenticated user as a user
owner(owner[, isorg = false]) Owner get owner as a user or organization
orgs(owner) Tuple{Vector{Owner}, Dict} get the owner's organizations
followers(owner) Tuple{Vector{Owner}, Dict} get the owner's followers
following(owner) Tuple{Vector{Owner}, Dict} get the users followed by owner
repos(owner[, isorg = false]) Tuple{Vector{Repo}, Dict} get the owner's repositories/get an organization's repositories
teams(owner) Tuple{Vector{Team}, Dict} get the organizations's teams repositories
sshkeys(owner) Tuple{Vector{Dict}, Dict} get the owner's public ssh keys
gpgkeys(owner) Tuple{Vector{Dict}, Dict} get the owner's public gpg keys

Teams

method return type documentation
members(team) Tuple{Vector{Owner}, Dict} get team members as users
repos(owner, team) Tuple{Vector{Repo}, Dict} get team repositories as users

Repositories

method return type documentation
repo(repo) Repo get repo
create_repo(owner, name) Repo create a repository of the given name in the given owner's account
create_fork(repo) Repo create a fork of repo
forks(repo) Tuple{Vector{Repo}, Dict} get repo's forks
contributors(repo) Dict get repo's contributors
collaborators(repo) Tuple{Vector{Owner}, Dict} get repo's collaborators
iscollaborator(repo, user) Bool check if user is a collaborator on repo
add_collaborator(repo, user) HTTP.Response add user as a collaborator to repo
remove_collaborator(repo, user) HTTP.Response remove user as a collaborator from repo
collaborator_permission(repo, user) HTTP.Response get the repo permission of a collaborator
stats(repo, stat[, attempts = 3]) HTTP.Response get information on stat (e.g. "contributors", "code_frequency", "commit_activity", etc.)
topics(repo) Vector{String} get the list of topics of a repository.)
set_topics(repo, topics) Vector{String} set the list of topics of a repository.)
commit(repo, sha) Commit get the commit specified by sha
commits(repo) Tuple{Vector{Commit}, Dict} get repo's commits
commits(repo, pr) Tuple{Vector{Commit}, Dict} get pr's commits for repo
compare(repo, base, head) Comparison compare repo's commits
branch(repo, branch) Branch get the branch specified by branch
branches(repo) Tuple{Vector{Branch}, Dict} get repo's branches
file(repo, path) Content get the file specified by path
directory(repo, path) Tuple{Vector{Content}, Dict} get the contents of the directory specified by path
create_file(repo, path) Dict create a file at path in repo
update_file(repo, path) Dict update a file at path in repo
delete_file(repo, path) Dict delete a file at path in repo
permalink(content::Content, commit) URIs.URI get a permalink for content at the SHA specified by commit
readme(repo) Content get repo's README
create_status(repo, sha) Status create a status for the commit specified by sha
statuses(repo, ref) Tuple{Vector{Status}, Dict} get the statuses posted to ref
status(repo, ref) Status get the combined status for ref
create_webhook(owner, repo) Webhook create a webhook for repo
secrets(repo; auth) Tuple{Vector{Secret}, Dict} get names of all secrets for repo
secret(repo, name; auth) Secret get status of secret in repo
create_secret(repo, name; value, auth) nothing create a secret for repo
delete_secret(repo, name; auth) nothing delete a secret for repo
deploykeys(repo; auth) Tuple{Vector{DeployKey}, Dict} get all deploy keys for repo
deploykey(repo, key; auth) DeployKey get the deploy key in repo
create_deploykey(repo; params=..., auth) nothing create a deploy key for repo
delete_deploykey(repo, key; auth) nothing delete a deploy key for repo
releases(repo, key; auth) nothing get the releases for repo

Pull Requests and Issues

method return type documentation
pull_request(repo, pr) PullRequest get the pull request specified by pr
pull_requests(repo) Tuple{Vector{PullRequest}, Dict} get repo's pull requests
pull_request_files(repo, pr) Tuple{Vector{PullRequestFiles}, Dict} get this repo's pr's file changes
create_pull_request(repo) PullRequest create pull request in repo
update_pull_request(repo, pr) PullRequest update the given pr in repo
close_pull_request(repo, pr) PullRequest close the given pr in repo
issue(repo, issue) Issue get the issue specified by issue
issues(repo) Tuple{Vector{Issue}, Dict} get repo's issues
create_issue(repo) Issue create an issue in repo
edit_issue(repo, issue) Issue edit issue in repo
reviews(repo, pr) Tuple{Vector{PullRequest}, Dict} get a pr's reviews
dismiss_review(repo, review) HTTP.Response dismiss review in repo

Comments

method return type documentation
comment(repo, comment, :issue) Comment get an issue comment from repo
comment(repo, comment, :pr) Comment get a PR comment from repo
comment(repo, comment, :review) Comment get an review comment from repo
comment(repo, comment, :commit) Comment get a commit comment from repo
comments(repo, issue, :issue) Tuple{Vector{Comment}, Dict} get the comments on issue in repo
comments(repo, pr, :pr) Tuple{Vector{Comment}, Dict} get the comments on pr in repo
comments(repo, pr, :review) Tuple{Vector{Comment}, Dict} get the review comments on pr in repo
comments(repo, commit, :commit) Tuple{Vector{Comment}, Dict} get the comments on commit in repo
create_comment(repo, issue, :issue) Comment create a comment on issue in repo
create_comment(repo, pr, :pr) Comment create a comment on pr in repo
create_comment(repo, pr, :review) Comment create a review comment on pr in repo
create_comment(repo, commit, :commit) Comment create a comment on commit in repo
edit_comment(repo, comment, :issue) Comment edit the issue comment in repo
edit_comment(repo, comment, :pr) Comment edit the PR comment in repo
edit_comment(repo, comment, :review) Comment edit the review comment in repo
edit_comment(repo, comment, :commit) Comment edit the commit comment in repo
delete_comment(repo, comment, :issue) HTTP.Response delete the issue comment from repo
delete_comment(repo, comment, :pr) HTTP.Response delete the PR comment from repo
delete_comment(repo, comment, :review) HTTP.Response delete the review comment from repo
delete_comment(repo, comment, :commit) HTTP.Response delete the commitcomment from repo
delete_comment(repo, comment, :commit) HTTP.Response delete the commitcomment from repo
reply_to(repo, review, comment, body) HTTP.Response reply to the comment (of review in repo) creating a new comment with the specified body

Labels

method return type documentation
labels(repo,issue) Vector{Label} list labels from issue
add_labels(repo, issue, labels) Vector{Label} add labels to an issue
set_labels(repo, issue, labels) Vector{Label} set the labels for an issue
remove_all_labels(repo, issue) HTTP.Response remove all labels from an issue
remove_label(repo, issue, label) HTTP.Response remove a label from an issue

Social Activity

method return type documentation
star(repo) HTTP.Response star repo
unstar(repo) HTTP.Response unstar repo
stargazers(repo) Tuple{Vector{Owner}, Dict} get repo's stargazers
starred(user) Tuple{Vector{Repo}, Dict} get repositories starred by user
watchers(repo) Tuple{Vector{Owner}, Dict} get repo's watchers
watched(user) Tuple{Vector{Repo}, Dict} get repositories watched by user
watch(repo) HTTP.Response watch repo
unwatch(repo) HTTP.Response unwatch repo

Gists

method return type documentation
gist(id) Gist get the gist specified by id
gist(id, revision) Gist get the gist specified by id and revision
gists() Tuple{Vector{Gist}, Dict} get all public gists
gists(owner) Tuple{Vector{Gist}, Dict} get all gists for owner
create_gist() Gist create a gist
edit_gist(gist) Gist edit a gist
delete_gist(gist) HTTP.Response delete a gist
create_gist_fork(gist) Gist fork a gist
gist_forks(gist) Tuple{Vector{Gist}, Dict} list the forks of a gist
star_gist(gist) HTTP.Response star gist
starred_gists() Tuple{Vector{Gist}, Dict} get the starred gists
unstar_gist(gist) HTTP.Response unstar gist

Git Data

method return type documentation
blob(repo, sha) Blob Look up a blob in the repo by its SHA
create_blob(repo) Blob Create a blob in the repo
gitcommit(repo, sha) GitCommit Look up a commit in the repo by its SHA
create_gitcommit(repo) GitCommit Create a commit in the repo
tree(repo, sha) Tree Look up a tree in the repo by its SHA
create_tree(repo) Tree Create a tree in the repo
tag(repo, sha) Tag Look up a tag in the repo by its name
create_tag(repo) Tag Create a tag in the repo
reference(repo, name) Reference Look up a ref in the repo by its name
references(repo) Vector{Reference} Get all refs of the repo
create_reference(repo) Reference Create a reference in the repo
update_reference(repo) Reference Update a reference in the repo
delete_reference(repo) GitCommit Delete a the repo
tag(repo) Reference Update a reference in the repo
delete_reference(repo) GitCommit Delete a the repo

GitHub Apps

method return type documentation
app(id) App get the GitHub app with the specified id
app(slug) App get the GitHub app with the specified slug
app(;auth=auth) App get the GitHub app authenticated by the corresponding auth
installations(auth) Vector{Installation} get the installations for the GitHub app authenticated by the corresponding auth
repos(i::Installation) Tuple{Vector{Repo}, Dict} get the active repositories for this installation

GitHub Check Runs

method return type documentation
create_check_run(repo; params=...) CheckRun Create a new check run
update_check_run(repo, id::Int; params=...) CheckRun Update the check run with the given id

Licenses

method return type documentation
licenses(; params=...) Vector{License} Get all commonly used licenses
license(license_id; params=...) License Get a license
repo_license(repo; params=...) Content Get the license for a respository

Miscellaneous

method return type documentation
rate_limit() Dict get your rate limit status
authenticate(token) OAuth2 validate token and return an authentication object

Keyword Arguments

All REST methods accept the following keyword arguments:

keyword type default value description
auth GitHub.Authorization GitHub.AnonymousAuth() The request's authorization
params Dict Dict() The request's query parameters
headers Dict Dict() The request's headers. Note that these headers will be mutated by GitHub.jl request methods.
handle_error Bool true If true, a Julia error will be thrown in the event that GitHub's response reports an error.
page_limit Real Inf The number of pages to return (only applies to paginated results, obviously)

Authentication

To authenticate your requests to GitHub, you'll need to generate an appropriate access token. Then, you can do stuff like the following (this example assumes that you set an environmental variable GITHUB_AUTH containing the access token):

import GitHub
myauth = GitHub.authenticate(ENV["GITHUB_AUTH"]) # don't hardcode your access tokens!
GitHub.star("JuliaWeb/GitHub.jl"; auth = myauth)  # star the GitHub.jl repo as the user identified by myauth

As you can see, you can propagate the identity/permissions of the myauth token to GitHub.jl's methods by passing auth = myauth as a keyword argument.

Note that if authentication is not provided, they'll be subject to the restrictions GitHub imposes on unauthenticated requests (such as stricter rate limiting)

Authenticating as a GitHub app

GitHub apps (formerly called integrations) have their own authentication format based on JSON Web Tokens. When creating a GitHub app, you will be prompted to download your app's private key. You can use this private key to authenticate as a Github App using the JWTAuth type:

appauth = JWTAuth(1234, "privkey.pem") # Replace with your app id/privkey file

The following shows a complete example that opens an issue on every repository on which your application gets installed:

listener = GitHub.EventListener() do event
    # On installation, open an issue on every repository we got installed in
    if event.kind == "installation"
        # Authenticate as the application
        appauth = GitHub.JWTAuth(1234, "privkey.pem")
        # Now, get permissions for this particular installation
        installation = Installation(event.payload["installation"])
        auth = create_access_token(installation, appauth)
        for repo in event.payload["repositories"]
            create_issue(GitHub.Repo(repo), auth=auth,
                params = Dict(
                    :title => "Hello World",
                    :body => "Thank you for installing me - I needed that"
            ))
        end
    end
    return HTTP.Response(200)
end
GitHub.run(listener, host=IPv4(0,0,0,0), port=8888)

Pagination

GitHub will often paginate results for requests that return multiple items. On the GitHub.jl side of things, it's pretty easy to see which methods return paginated results by referring to the REST Methods documentation; if a method returns a Tuple{Vector{T}, Dict}, that means its results are paginated.

Paginated methods return both the response values, and some pagination metadata. You can use the per_page/page query parameters and the page_limit keyword argument to configure result pagination.

For example, let's request a couple pages of GitHub.jl's PRs, and configure our result pagination to see how it works:

# show all PRs (both open and closed), and give me 3 items per page starting at page 2
julia> myparams = Dict("state" => "all", "per_page" => 3, "page" => 2);

julia> prs, page_data = pull_requests("JuliaWeb/GitHub.jl"; params = myparams, page_limit = 2);

julia> prs # 3 items per page * 2 page limit == 6 items, as expected
6-element Array{GitHub.PullRequest,1}:
 GitHub.PullRequest(44)
 GitHub.PullRequest(43)
 GitHub.PullRequest(42)
 GitHub.PullRequest(41)
 GitHub.PullRequest(39)
 GitHub.PullRequest(38)

julia> page_data
Dict{String,String} with 4 entries:
  "prev"  => "https://api.github.com/repositories/16635105/pulls?page=2&per_page=3&state=all"
  "next"  => "https://api.github.com/repositories/16635105/pulls?page=4&per_page=3&state=all"
  "first" => "https://api.github.com/repositories/16635105/pulls?page=1&per_page=3&state=all"
  "last"  => "https://api.github.com/repositories/16635105/pulls?page=7&per_page=3&state=all"

In the above, prs contains the results from page 2 and 3. We know this because we specified page 2 as our starting page ("page" => 2), and limited the response to 2 pages max (page_limit = 2). In addition, we know that exactly 2 pages were actually retrieved, since there are 6 items and we said each page should only contain 3 items ("per_page" => 3).

The values provided by page_data are the same values that are included in the Link header of the last requested item. You can continue paginating by starting a new paginated request at one of these links using the start_page keyword argument:

# Continue paging, starting with `page_data["next"]`.
# Note that the `params` kwarg can't be used here because
# the link passed to `start_page` has its own parameters
julia> prs2, page_data2 = pull_requests("JuliaWeb/GitHub.jl"; page_limit = 2, start_page = page_data["next"]);

julia> prs2
6-element Array{GitHub.PullRequest,1}:
 GitHub.PullRequest(37)
 GitHub.PullRequest(34)
 GitHub.PullRequest(32)
 GitHub.PullRequest(30)
 GitHub.PullRequest(24)
 GitHub.PullRequest(22)

julia> page_data2
Dict{String,String} with 4 entries:
  "prev"  => "https://api.github.com/repositories/16635105/pulls?page=4&per_page=3&state=all"
  "next"  => "https://api.github.com/repositories/16635105/pulls?page=6&per_page=3&state=all"
  "first" => "https://api.github.com/repositories/16635105/pulls?page=1&per_page=3&state=all"
  "last"  => "https://api.github.com/repositories/16635105/pulls?page=7&per_page=3&state=all"

Handling Webhook Events

GitHub.jl comes with configurable EventListener and CommentListener types that can be used as basic servers for parsing and responding to events delivered by GitHub's repository Webhooks.

EventListener

When an EventListener receives an event, it performs some basic validation and wraps the event payload (and some other data) in a WebhookEvent type. This WebhookEvent instance, along with the provided Authorization, is then fed to the server's handler function, which the user defines to determine the server's response behavior. The handler function is expected to return an HTTP.Response that is then sent back to GitHub.

The EventListener constructor takes the following keyword arguments:

Here's an example that demonstrates how to construct and run an EventListener that does benchmarking on every commit and PR:

import GitHub
import URIs
# EventListener settings
myauth = GitHub.authenticate(ENV["GITHUB_AUTH"])
mysecret = ENV["MY_SECRET"]
myevents = ["pull_request", "push"]
myrepos = [GitHub.Repo("owner1/repo1"), "owner2/repo2"] # can be Repos or repo names
myforwards = [URIs.URI("http://myforward1.com"), "http://myforward2.com"] # can be URIs.URIs or URI strings

# Set up Status parameters
pending_params = Dict(
    "state" => "pending",
    "context" => "Benchmarker",
    "description" => "Running benchmarks..."
)

success_params = Dict(
    "state" => "success",
    "context" => "Benchmarker",
    "description" => "Benchmarks complete!"
)

error_params(err) = Dict(
    "state" => "error",
    "context" => "Benchmarker",
    "description" => "Error: $err"
)

# We can use Julia's `do` notation to set up the listener's handler function
listener = GitHub.EventListener(auth = myauth,
                                secret = mysecret,
                                repos = myrepos,
                                events = myevents,
                                forwards = myforwards) do event
    kind, payload, repo = event.kind, event.payload, event.repository

    if kind == "pull_request" && payload["action"] == "closed"
        return HTTP.Response(200)
    end

    if event.kind == "push"
        sha = event.payload["after"]
    elseif event.kind == "pull_request"
        sha = event.payload["pull_request"]["head"]["sha"]
    end

    GitHub.create_status(repo, sha; auth = myauth, params = pending_params)

    try
        # run_and_log_benchmarks isn't actually a defined function, but you get the point
        run_and_log_benchmarks(event, "\$(sha)-benchmarks.csv")
    catch err
        GitHub.create_status(repo, sha; auth = myauth, params = error_params(err))
        return HTTP.Response(500)
    end

    GitHub.create_status(repo, sha; auth = myauth, params = success_params)

    return HTTP.Response(200)
end

# Start the listener on localhost at port 8000
GitHub.run(listener, IPv4(127,0,0,1), 8000)

CommentListener

A CommentListener is a special kind of EventListener that allows users to pass data to the listener's handler function via commenting. This is useful for triggering events on repositories that require configuration settings.

A CommentListener automatically filters out all non-comment events, and then checks the body of each comment event against a trigger Regex supplied by the user. If a match is found in the comment, then the CommentListener calls its handler function, passing it the event and the corresponding RegexMatch.

The CommentListener constructor takes the following keyword arguments:

For example, let's set up a silly CommentListener that responds to the commenter with a greeting. To give a demonstration of the desired behavior, if a collaborator makes a comment like:

Man, I really would like to be greeted today.

`sayhello("Bob", "outgoing")`

We want the CommentLister to reply:

Hello, Bob, you look very outgoing today!

Here's the code that will make this happen:

import GitHub

# CommentListener settings
trigger = r"`sayhello\(.*?\)`"
myauth = GitHub.authenticate(ENV["GITHUB_AUTH"])
mysecret = ENV["MY_SECRET"]

# We can use Julia's `do` notation to set up the listener's handler function.
# Note that, in our example case, `phrase` will be "`sayhello(\"Bob\", \"outgoing\")`"
listener = GitHub.CommentListener(trigger; auth = myauth, secret = mysecret) do event, phrase
    # In our example case, this code sets name to "Bob" and adjective to "outgoing"
    name, adjective = matchall(r"\".*?\"", phrase)
    comment_params = Dict("body" => "Hello, $name, you look very $adjective today!")

    # Parse the original comment event for all the necessary reply info
    comment = GitHub.Comment(event.payload["comment"])

    if event.kind == "issue_comment"
        comment_kind = :issue
        reply_to = event.payload["issue"]["number"]
    elseif event.kind == "commit_comment"
        comment_kind = :commit
        reply_to = comment.commit_id
    elseif event.kind == "pull_request_review_comment"
        comment_kind = :review
        reply_to = event.payload["pull_request"]["number"]
        # load required query params for review comment creation
        comment_params["commit_id"] = comment.commit_id
        comment_params["path"] = comment.path
        comment_params["position"] = comment.position
    end

    # send the comment creation request to GitHub
    GitHub.create_comment(event.repository, reply_to, comment_kind; auth = myauth, params = comment_params)

    return HTTP.Response(200)
end

# Start the listener on localhost at port 8000
GitHub.run(listener, IPv4(127,0,0,1), 8000)

GitHub Enterprise

This library work with github.com, and also with self-hosted github, a.k.a. GitHub Enterprise.

To use it with self-hosted github, you need to create GitHubWebAPI structure and pass it to functions when needed. Following example shows obtaining repository info private/Package.jl on github instance with API https://git.company.com/api/v3.

import GitHub
import URIs

api = GitHub.GitHubWebAPI(URIs.URI("https://git.company.com/api/v3"))
myauth = GitHub.authenticate(api, ENV["GITHUB_AUTH"])
myrepo = GitHub.repo(api, "private/Package.jl", auth=myauth)

SSH keys

You can generate public-private key pairs with GitHub.genkeys. Here's an example adding a deploy key and secret, in this case to deploy documentation via GitHub Actions:

pubkey, privkey = GitHub.genkeys()
create_deploykey(repo; auth, params=Dict("key"=>pubkey, "title"=>"Documenter", "read_only"=>false))
create_secret(repo, "DOCUMENTER_KEY"; auth, value=privkey)

privkey is sent in encrypted form to GitHub. Do not share privkey with others or post it publicly; doing so breaches the security of your repository. You can read more about the meaning of SSH keys and their security implications.