Cannot set custom Github API URL on remote source create

[MedicineYeh]

What would you like to be added: Customized Github API URL Currently it only supports github.com as remote source. Is it possible to set the API URL like gitea ?

docker run --rm sorintlab/agola --token "xxxxx" --gateway-url http://xxxxx:8000 remotesource create \
--name github \
--type github \
--api-url https://{Customized GitHub API URL}/ \
--auth-type oauth2 \
...

Why is this needed: Private LAN has its own GitHub server.

[sgotti]

@MedicineYeh thanks for reporting this. I noticed there's a bug in the remotesource create command logic since it's going to always force the public github url, will open a PR to fix this. In the meantime you should be able to fix this issuing a remotesource update command to update the api url to a different one.

[sgotti]

created PR #205

[MedicineYeh]

Thanks for the work around. I guess it's still not getting the right value on some functions. here is the log I got. At the beginning, it redirect to private github successfully but later it shows this error. Seems like the variables are still not updated when accessing the Github API.

agola-ci-components_1  | 2020-02-10T08:54:50.751Z       ERROR   api/oauth2.go:44        err: failed to retrieve remote user info for remote source "[[[REMOVED_FOR_PRIVACY]]]":
agola-ci-components_1  |     agola.io/agola/internal/services/gateway/action.(*ActionHandler).LoginUser
agola-ci-components_1  |         /agola/internal/services/gateway/action/user.go:393
agola-ci-components_1  |   - Get https://api.github.com/user: dial tcp 140.82.113.6:443: i/o timeout

[sgotti]

@MedicineYeh Thanks, also the github remote source needs to be fixed. Can you please try building agola after applying PR #206?

[MedicineYeh]

@sgotti I've tried the patches of #205 and #206. My guess is that #205 is not the call path because it doesn't print anything when I add log as followed.

 87    // for github remote source type, set defaults for github.com
 88    if remoteSourceCreateOpts.rsType == "github" {
 89         if !flags.Changed("api-url") {
 90          remoteSourceCreateOpts.apiURL = github.GitHubAPIURL
 91       }
 92       if remoteSourceCreateOpts.apiURL == github.GitHubAPIURL && !flags.Changed("ssh-host-key") {
 93          remoteSourceCreateOpts.sshHostKey = github.GitHubSSHHostKey
 94       }
 95    }
 96 +   log.Infof(remoteSourceCreateOpts.apiURL)

The web page redirect me to this URL after clicking Register with ... button. https://github.com/login/oauth/authorize?client_id=xxxxxxxxxxxxxxxxx&redirect_uri=http%3A%2F%2Fxxxxxxxxxxxxxxxxx%3A8000%2Foauth2%2Fcallback&response_type=code&scope=repo&state=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1ODEzNzE1MjQsInJlbW90ZV9zb3VyY2VfbmFtZSI6ImdpdGh1YjUiLCJyZXF1ZXN0Ijoie1wiUmVtb3RlU291cmNlTmFtZVwiOlwiZ2l0aHViNVwiLFwiVXNlckFjY2Vzc1Rva2VuXCI6XCJcIixcIk9hdXRoMkFjY2Vzc1Rva2VuXCI6XCJcIixcIk9hdXRoMlJlZnJlc2hUb2tlblwiOlwiXCIsXCJPYXV0aDJBY2Nlc3NUb2tlbkV4cGlyZXNBdFwiOlwiMDAwMS0wMS0wMVQwMDowMDowMFpcIn0iLCJyZXF1ZXN0X3R5cGUiOiJhdXRob3JpemUifQ.V2K4EjOW5czf5Pa1epVFx-1cMs7wleFTI3AieRxTvr8

P.S. This can be fixed by running a remotesource update command.

[MedicineYeh]

Thanks for your quick supports. Also, there is another topic that might need some efforts and worth another issue tracking. I found that our private GitHub server does not enable api.github... URL. I've tried DroneCI on it. I don't know what API calls it relies on but it works without any issue. Maybe it's worth an feature update to remove the requirements for such api URL since some so called private GitHub might disable that interface.

[sgotti]

@MedicineYeh Sorry but I can't understand from your previous posts if it's working or not. We don't have a private github to test so we are relying on your reports. #205 is only for the remotesource create while #206 is for fixing the issue you reported here https://github.com/agola-io/agola/issues/204#issuecomment-584020491

I found that our private GitHub server does not enable api.github... URL.

You should provide the "api url" of your local github instance that should be that same as the web url (the different api and web urls api.github.com vs github.com are only a public github thing).

[MedicineYeh]

Sorry for the confusion. I'd patched two fixes manually. I guess that the #205 did not fix the bug. I still need to run a remotesource update command to make it work.

As for the "api url", it's a blocker for my environment because our private GitHub server disables that function.

[sgotti]

I've updated #206 with more fixes, please try it.

As for the "api url", it's a blocker for my environment because our private GitHub server disables that function.

I think you're not understanding what is the api url, in your case you should provide the url you use to access via web (the same you are using with drone, i.e. https://yourlocalgithub.yourdomain.com). it's not possible they are disabled or also drone won't work.

[MedicineYeh]

I've tested the updated commits. It's still not redirecting to the specified URL. I found it was this line setting the values internal/services/gateway/action/remotesource.go:102 and the #205 is not in the actual call path.

As for the update of #206, I've commented directly on the PR.

The latest error after fixing the #206 PR and #205 by myself is as follow.

2020-02-11T07:52:08.249Z       ERROR   api/oauth2.go:44        err: cannot get oauth2 token:
     agola.io/agola/internal/gitsources/github.(*Client).RequestOauth2Token
         /agola/internal/gitsources/github/github.go:181
   - oauth2: server response missing access_token

[sgotti]

@MedicineYeh Please try again with the updated #206.

and the #205 is not in the actual call path.

I don't understand what you mean by actual call path. #205 fixes the agola remotesource create command to correctly set the provided api url. So it's not part of the server side.

[sgotti]

@MedicineYeh I've setup a trial github enterprise server and managed to make everything to work fixing the last issue you had in https://github.com/agola-io/agola/issues/204#issuecomment-584516356 with #207.

Tested a full workflow by creating a repo, pushing to it etc... and everything works.

Thanks for your report. Closing as fixed by #205 #206 #207.