Add support for specifying Endpoint in Github Provider in the Material page

ghost commented 7 years ago

On the read me there is the below quote:

You can create a file ~/.github with the following contents: (Note: ~/.github needs to be available on Go Server and on all Go Agent machines)

Where does ~ refer to in this case? Would that be the "go" users home directory (/var/go)? I ask because I am trying to get this plugin working with our enterprise github for a new project but whenever I add a material and "Check Connection" the failure message indicates https://api.github.com/repos/... which is making me think the setting is not being picked up. Could I get some assistance here to point out what I am doing wrong?

Thanks: Frito

ashwanthkumar commented 7 years ago

Yes it refers to the home directory of the go user - which by default is /var/go.

Assuming you already have the file in there. Can you try these steps?

Restart your server once just to be sure the Github client is able to pick your properties.
Can you share the full plugin logs - it might have some stacktrace which might help us understand what is going wrong.

ghost commented 7 years ago

@ashwanthkumar I did restart the go-server and go-agent before posting the question so I believe we're covered on item 1. For the giggles I ended up unpausing the new pipeline and it appears to have pulled from the enterprise URL so that's promising. I'm going to assume this might just be a bug in the "Check Connections" portion of the plugin, possibly not reading that .github file.

Please excuse my ignorance here but where can I grab the plugin log file from? I'm happy to provide it but I'm not sure where to grab that from. Also, huge thanks on the quick reply!

ashwanthkumar commented 7 years ago

The plugin logs are available in the same directory as the go server logs. In our setup they're available under /var/log/go-server. The file name would be plugin-github.pr.log.

ghost commented 7 years ago

Sadly there's not much available in the logs...

From plugin-github.pr.log: go@ubuntu-gocd:/var/log/go-server$ cat plugin-github.pr.log 2016-10-03 13:02:38,902 INFO [92@MessageListener for MaterialUpdateListener] GitHubPRBuildPlugin:52 - Flyweight: /var/lib/go-server/pipelines/flyweight/c3c00f7f-8cd0-4652-ac3b-7472d64f63a5 2016-10-03 13:02:40,895 INFO [92@MessageListener for MaterialUpdateListener] GitHubPRBuildPlugin:52 - Triggered build for master with head at 8f0203f83e62c5504dfa1ede793b4fceba87d591 2016-10-03 13:48:56,673 INFO [91@MessageListener for MaterialUpdateListener] GitHubPRBuildPlugin:52 - new commits: 1

From plugin-github.pr.status.log (repeated over and over): 2016-10-03 13:49:58,482 ERROR [72@MessageListener for PluginNotificationListener] BuildStatusNotifierPlugin:77 - Error occurred. Could not update build status - URL: https://enterprisegithub.enterprise.com/Organization/Repo.git Revision: e851aa106456447f1432c78ecda19e23836b8ad9 Build: PR-Build/2/Package/1 Result: Passed java.io.IOException: Failed to resolve credentials from ~/.github or the environment. at org.kohsuke.github.GitHubBuilder.fromCredentials(GitHubBuilder.java:66) at org.kohsuke.github.GitHub.connect(GitHub.java:147) at com.tw.go.plugin.provider.github.GitHubProvider.createGitHubClient(GitHubProvider.java:89) at com.tw.go.plugin.provider.github.GitHubProvider.updateCommitStatus(GitHubProvider.java:67) at com.tw.go.plugin.provider.github.GitHubProvider.updateStatus(GitHubProvider.java:57) at com.tw.go.plugin.BuildStatusNotifierPlugin.handleStageNotification(BuildStatusNotifierPlugin.java:160) at com.tw.go.plugin.BuildStatusNotifierPlugin.handle(BuildStatusNotifierPlugin.java:84) at com.thoughtworks.go.plugin.infra.DefaultPluginManager$2.execute(DefaultPluginManager.java:186) at com.thoughtworks.go.plugin.infra.DefaultPluginManager$2.execute(DefaultPluginManager.java:181) at com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.executeActionOnTheService(FelixGoPluginOSGiFramework.java:315) at com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.doOn(FelixGoPluginOSGiFramework.java:245) at com.thoughtworks.go.plugin.infra.DefaultPluginManager.submitTo(DefaultPluginManager.java:181) at com.thoughtworks.go.plugin.access.PluginRequestHelper.submitRequest(PluginRequestHelper.java:32) at com.thoughtworks.go.plugin.access.notification.NotificationExtension.notify(NotificationExtension.java:66) at com.thoughtworks.go.server.messaging.plugin.PluginNotificationService.notifyPlugin(PluginNotificationService.java:61) at com.thoughtworks.go.server.messaging.plugin.PluginNotificationService.notifyPlugins(PluginNotificationService.java:53) at com.thoughtworks.go.server.messaging.plugin.PluginNotificationListener.onMessage(PluginNotificationListener.java:30) at com.thoughtworks.go.server.messaging.plugin.PluginNotificationListener.onMessage(PluginNotificationListener.java:21) at com.thoughtworks.go.server.messaging.activemq.JMSMessageListenerAdapter.runImpl(JMSMessageListenerAdapter.java:69) at com.thoughtworks.go.server.messaging.activemq.JMSMessageListenerAdapter.run(JMSMessageListenerAdapter.java:50) at java.lang.Thread.run(Thread.java:745)

I'm off to a meeting but I can write some steps to reproduce the issue pretty easily even though I have it working now. I'm thinking it's because I entered credentials in the dialog but did not populate credentials in the ~/.github file.

ashwanthkumar commented 7 years ago

I'm glad you got it working :) Sure - please do share the steps to re-produce the issue. I'll look into it as soon as I get a chance.

ghost commented 7 years ago

Steps to Reproduce:

1.) Install Go CD Agent and Server on a single machine (for testing) 2.) Downloaded / Installed github-pr-poller.jar from release page 3.) Created /var/go/.github file with contents a.) endpoint=https://internalgithub.corporate.com (NOTE: also tried with "/api/v3" appended to uri) 4.) Created new pipeline / stage / job / task as stub through the wizard. Used "git" material temporarily 5.) Edited new pipeline, deleted "git" material. 6.) Added material "Github" with settings from attached screen shot. Sorry for the black boxes but I'm working on a tool here that isn't public knowledge yet. 7.) Click "CHECK CONNECTION" button

dialog-error

NOTE: If I ignore the error displayed the plugin does still function properly.

gunnarx commented 7 years ago

I was about to ask a separate question but since this is open and it seems near enough:

Why does the README require ~/.github to be on all agents? I seem to have success with it being on the server only. @ashwanthkumar, could you explain what it is used for on the agent side? Thanks.

ashwanthkumar commented 7 years ago

@gunnarx Sorry for the delay in getting back. Looks like README needs to be updated. IIRC, Earlier we used to query GithubAPI using the auth token for populating the PR fields which are injected as environment variables in the task. I guess we moved away from that model a long time ago, but didn't change the README to reflect it. Would you be kind enough to send a PR with the fixes?

gavingmiller commented 5 years ago

I am seeing this same behaviour on our installation too (master using code from my previous PR). Can you advise whether this is bring actively looked at? If not, I can pickup the work as this implies serious security implications for the plugin. Thanks.

ashwanthkumar commented 5 years ago

@gavingmiller I'm sorry, what's the issue?

gavingmiller commented 5 years ago

@ashwanthkumar have you done any work on this? If not, I will pickup the issue and try to fix it. 😄

ashwanthkumar commented 5 years ago

@gavingmiller Are you not able to connect to a Enterprise Github instance via the plugin? I'm sorry, there're many things in this issue I don't quite follow which issue you're talking about. Forgive my ignorance.

gavingmiller commented 5 years ago

@ashwanthkumar sorry about clarity. The issue I'm seeing is detailed in this comment: https://github.com/ashwanthkumar/gocd-build-github-pull-requests/issues/75#issuecomment-251384231

Rephrased, the username/password set in the Edit Material don't account for an endpoint set in the .github file. And so instead of querying an enterprise instance, they query github.com.

This is a security issue since a user's enterprise credentials are sent to github.com instead of their enterprise server.

ashwanthkumar commented 5 years ago

Ah, I now understand. The relevant section of code is in here. While GitHub.connect() resorts to using the ~/.github file we don't directly consume it. If you specify credentials directly in the material, we don't end up using that file at all. Also if we specify the credentails in the material, we can't specify the end-point there.

@gavingmiller Would you like to pick up, adding the end-point text-field for Github Provider in the plugin?

ashwanthkumar / gocd-build-github-pull-requests

Add support for specifying Endpoint in Github Provider in the Material page #75