GitHub App Installation APIs missing

[carterbuce]

I'm using a GitHub App with two goals: get the (private) repositories in an organization, and get the (private) members in the organization.

The way I'm using this API is by creating an installation of the app on my organization. My App is set to have read access to checks, members and metadata. Then I use the private key to generate a JWT token and authenticate with the API, which gives me a GHApp object. From there I'm using listInstallations to create a new GitHub object with the installation token.

So now I'm authenticated as my installation on the organization, which is where I expect to be. However from here I don't see any installation-specific methods I can use with this Java API. The getRepositories seems to use the /user/{org}/repositories endpoint, which means I only get the public repositories on my organization, whereas I'm looking for the GET /installation/repositories endpoint. In addition to this, the listMembers method is also only showing me the public members, but not the ones marked as private in the organization. Am I missing something? I'm new to the GitHub API so I could just be not using this API correctly.

[bitwiseman]

@carterbuce Most of this project's APIs are meant for the end-user endpoints, not GitHub App installations. So, what you're looking for is probably not added yet.

https://github.com/github-api/github-api/blob/master/src/main/java/org/kohsuke/github/GHAppInstallation.java would be where we'd want to add the methods you're requesting.

[PauloMigAlmeida]

sorry for my absence guys, I have been snowed under a bit lately.

@carterbuce everything that you wrote led me to believe that you are confusing "Logged in as App" with "Logged in as AppInstalation". (There is the possibility that I got you wrong...but I'm sure we will eventually find this out).

Would you mind sharing some code snippets and errors so that I can assist you with that?

Another useful bit would be if you could fully read this documentation:

First: http://github-api.kohsuke.org/githubappjwtauth.html Then: http://github-api.kohsuke.org/githubappappinsttokenauth.html

[PauloMigAlmeida]

Nevermind, I understood what you meant now.

Yep, that endpoint is missing.. as soon as I finish the thing I'm doing now I will take a look at it (unless someone else creates a PR before that)

Good catch @carterbuce

[carterbuce]

Regarding my issues getting the private members of an organization: I think the issue has something to do with how the installation token is created. I created two access tokens, one through curl and one through the Java api.

curl:

curl -v -i -X POST -H "Authorization: Bearer JWT_TOKEN"
-H "Accept: application/vnd.github.machine-man-preview+json" 
https://api.github.com/app/installations/INSTALLATION_ID/access_tokens

java api:

GHApp gh = new GitHubBuilder().withJwtToken(getJwtToken()).build().getApp();
GHAppInstallation installation = gh.listInstallations().iterator().next();
Map<String, GHPermissionType> permissions = installation.getPermissions();
GHAppInstallationToken token = installation.createToken(permissions).create().build();

For both of these methods I'm using the same JWT token. I then use the token from each in the curl request for getting organization members:

curl -v -i -H "Authorization: token TOKEN" 
-H "Accept: application/vnd.github.machine-man-preview+json"
https://api.github.com/orgs/TeamCodeScanner/members

The request using the token made in curl gives me all members of the organization, both public and private. However the token created through the Java API gives me only the public members of the organization.

I did some further investigating and saw that the "permissions" map has the READ GHPermissionType for members, checks, contents, and metadata. However the token that gets created has "read" for only checks, contents, and metadata. However I'm not exactly sure what's causing this, or if it's causing the issue with getting members that I'm seeing.

[PauloMigAlmeida]

Hi @carterbuce,

I finally had a chance to debug this issue... here is what I found so far:

the getRepositories seems to use the /user/{org}/repositories endpoint, which means I only get the public repositories on my organization, whereas I'm looking for the GET /installation/repositories endpoint.

While you're right about the fact that /user/{org}/repositories returns only public repositories and that GET /installation/repositories endpoint is yet to be implemented, there is one or two meta-considerations to be said here:

Both GHUser and GHOrganization extends GHPerson (which has been a contributing factor of several issues we've had to distinguish one from the other such as https://github.com/github-api/github-api/issues/126). However, if you read the source you will find that listRepositories(final int pageSize) is overridden which then points to the /org/{org}/repositories.

The key point here is that /org/{org}/repositories can list both private and public repositories with a caveat though: Only private repositories in which the user has given the app permission will be listed, otherwise, only public repositories will return.

Given your use-case, make sure that this is what you have configured (if you haven't already done so)

I ran a few tests and could list all 200+ private repos we have internally this way:

GitHub gitHubApp = new GitHubBuilder().withJwtToken(jwtToken).build();
GHApp app = gitHubApp.getApp();

for (GHAppInstallation appInstallation : app.listInstallations()) {
    Map<String, GHPermissionType> permissions = new HashMap<>();
    app.getPermissions().forEach((k, v) -> {
        permissions.put(k, GHPermissionType.valueOf(v.toUpperCase()));
    });

    GHAppInstallationToken appInstallationToken = appInstallation.createToken(permissions)
                    .create();
    GitHub githubAuthAsInst = new GitHubBuilder().withAppInstallationToken(appInstallationToken.getToken()).build();
    githubAuthAsInst.getOrganization("MyOrgPlaceholder").listRepositories().asList().stream().filter(GHRepository::isPrivate).count()
}

In addition to this, the listMembers method is also only showing me the public members, but not the ones marked as private in the organization. Am I missing something?

If you take a look at the GitHub App permissions webpage, you will see that /orgs/:org/members is listed as part of the metadata permission which is mandatory for all apps so we can rule out the possibility that any permission is missing when issuing the App installation token.

GitHub's members-list endpoint says:

List all users who are members of an organization. If the authenticated user is also a member of this organization then both concealed and public members will be returned.

GitHub treats users and bots quite differently (which is the classification for GitHub Apps)... for instance: You can't add a bot to the list of users that can push to a protected branch but you can do exactly the same with a regular user.

I did several tests and what I got was that whenever I accessed that endpoint via a GitHub App, I only got the public members. On the other hand, when I access the same endpoint using a Personal Access Token (with very limited permissons: read:org) then I would get the whole list of members. Maybe this is the missing piece of the puzzle for you?

Last but not least:

The request using the token made in curl gives me all members of the organization, both public and private. However the token created through the Java API gives me only the public members of the organization.

This isn't (unfortunately) what I was able to reproduce locally and all of my tests were consistent with the behaviour I described above. Would you mind double-checking it?

Happy coding ;)

[carterbuce]

Okay, thanks for the research into private repositories. I don't have any on the organization I'm testing on so I was unable to verify if that would work or not.

I'm definitely still seeing the issue with private members not showing up using the token created through the Java API. On the left window in this screenshot I created the token using a curl request, and on the right side I used a token created the same way as your previous comment creates it in Java.

[PauloMigAlmeida]

Hi @carterbuce,

I tried obtaining the same behaviour with:

• GitHub App (Target Type Org) -> Curl and API
• GitHub App (Target Type User) -> Curl and API

And in all cases, I obtained the same result I mentioned in my previous comment. Here is what I propose: Let's do a 1:1 video conf (Skype or any other tool) so that we can debug it a bit further and potentially find out what's different between your env/app and the ones I'm using.

PS.: You can find my email on my profile and please bear in mind I'm in NZTD (GMT +13:00).

[PauloMigAlmeida]

@carterbuce Nice talking to you earlier today.

For documenting purposes, here is the root cause of this issue:

POST /app/installations/:installation_id/access_tokens endpoint can requested with an empty JSON object or even no payload. This isn't (unfortunately) very clear on [GitHub's documentation] (https://developer.github.com/v3/apps/#create-a-new-installation-token).

It turns out that GitHub generates a token with certain default permissions when a body isn't specified which differ from the permissions obtained from the list installation endpoint....and that's why Carter and I couldn't reproduce each other's results.

I will open a PR that will enable users to choose if they want to send an empty JSON object or to specify either permissions or repositoryIds (both is fine too)