Not obeys the Github Branch protection settings

[vinoth241]

Hi Team, This is the exact app which i looked for auto merge which should happen after all required branch protection settings are getting passed. I just did the setup by following the 'deployment' steps. I started the server and it listened on port 8080. I tried to have setup a branch protection settings for my 'master' branch in Github repo. Once I raised a PR request against that 'master' branch, it is quickly getting auto merged even before all 'branch protection settings/status updates happen'.

This is the config i am using against my Github Repo.

version: 1

merge:
  method: squash

  options:
    squash:
      body: "summarize_commits"

  required_statuses:
    - "default"

  delete_after_merge: true 

My questions are

1. Am i missing something?
2. Where can i find the logs? [I can see the server log with debug option where i cannot find a message other than server started and listening]

It would be great if anybody can help me out as i am struck it here.

[vinoth241]

I tried to hit the server once hosted inside an instance curl http://127.0.0.1:8080/api/github/hook It returns 2019-01-04T11:20:57.037607971Z |INFO| http_request elapsed=0.091949 method=GET path=/api/github/hook rid=bgnk36em7cor6kv9660g size=19 status=404

[bluekeyes]

All logs should be printed to standard output. If running with a Docker container, you should be able to see them by running docker logs. Unfortunately, I don't think I'll be able to help debug more without logs, so let's try to get that working first. If you still can't find any logs, can you share:

1. The commands you are using to start the server
2. The server configuration (remove any secret values)
3. The configuration of the application in GitHub (what permissions are granted, etc.)

[vinoth241]

Thanks for the response @bluekeyes .

1. The commands you are using to start the server . -- I've downloaded the binary from https://bintray.com/palantir/releases/bulldozer and started using the command "./bulldozer server"

2. The server configuration (remove any secret values)

github:
  # The URL of the GitHub homepage
  web_url: "https://github.com"
  # The base URL for v3 (REST) API requests
  v3_api_url: "https://api.github.com"
  app:
    # The integration ID of the GitHub app
    integration_id: 556891
    # A random string used to validate webhooks
    webhook_secret: "app_secret"
    # The private key of the GitHub app (pem file download from GitHub)
    private_key:
         -----BEGIN RSA PRIVATE KEY-----
          xxxxxxxxxxxxxx

# Options for application behavior
options:
  # The path within repositories to find the bulldozer.yml config file
  configuration_path: .bulldozer.yml
  # The name of the application. This will affect the User-Agent header
  # when making requests to Github.
  app_name: bulldozer

Would like to know what is that integration Id? I have taken integration ID from the applications configure page(556891) GitHub Settings -> Applications -> Installed Github Apps --> bulldozer https://github.com/settings/installations/556891

3. Permissions Granted Write access to code Read access to administration, commit statuses, and metadata Read and write access to issues and pull requests

Hitting the server via curl responds this. curl http://127.0.0.1:8080/api/github/hook It returns 2019-01-04T11:20:57.037607971Z |INFO| http_request elapsed=0.091949 method=GET path=/api/github/hook rid=bgnk36em7cor6kv9660g size=19 status=404

Actually I am seeing the application [bulldozer] itself does not do any action on Pull Requests. Kindly suggest!

[bluekeyes]

Thanks for the information. For the 404 response, this is because the /api/github/hook endpoint only accepts POST requests. If you run the following, you should see a 202 response in the output:

curl -XPOST http://127.0.0.1:8080/api/github/hook

Because you can see these request logs, it seems like logging is working correctly, so if you don't see any other messages it probably means that Bulldozer never received a request from GitHub. You should be able to check this in GitHub: from the app's settings page (where you set the URLs, etc.), click on the "Advanced" tab in the sidebar. The top section should show a list of recent webhook deliveries where you can see if the webhooks were successful or not.

Based on what you've described so far, my guess is that GitHub is not able to send requests to your local computer, where the server is running. If this is the case, take a look at this section from the GitHub App guide. It describes how to create a tunnel back to your local machine so your server can receive GitHub hooks.

[vinoth241]

Yes @bluekeyes . Thanks for the detail. I just whitelisted Github IP to reach the hosted instance. Now I can see the pull request event get reach to server end. But I am getting the below error. Can you help me where i am getting wrong. It says the private key which generated is invalid

2019-01-09T07:20:40.706871610Z |ERROR| Unexpected error handling webhook request error="failed to instantiate github client: could not parse private key: Invalid Key: Key must be PEM encoded PKCS1 or PKCS8 private key\ngithub.com/palantir/bulldozer/server/handler.(*PullRequest).Handle\n\t/go/src/github.com/palantir/bulldozer/server/handler/pull_request.go:56\ngithub.com/palantir/bulldozer/vendor/github.com/palantir/go-githubapp/githubapp.(*eventDispatcher).ServeHTTP\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/palantir/go-githubapp/githubapp/dispatcher.go:118\ngithub.com/palantir/bulldozer/vendor/goji%2eio.dispatch.ServeHTTP\n\t/go/src/github.com/palantir/bulldozer/vendor/goji.io/dispatch.go:17\ngithub.com/palantir/bulldozer/vendor/goji%2eio.(*dispatch).ServeHTTP\n\t<autogenerated>:1\ngithub.com/palantir/bulldozer/vendor/github.com/bluekeyes/hatpear.Recover.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/bluekeyes/hatpear/hatpear.go:107\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/bluekeyes/hatpear.Catch.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/bluekeyes/hatpear/hatpear.go:60\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog.AccessHandler.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog/hlog.go:180\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog.RequestIDHandler.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog/hlog.go:169\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/palantir/go-baseapp/baseapp.NewMetricsHandler.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/palantir/go-baseapp/baseapp/middleware.go:42\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog.NewHandler.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog/hlog.go:30\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/goji%2eio.(*Mux).ServeHTTP\n\t/go/src/github.com/palantir/bulldozer/vendor/goji.io/mux.go:74\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2741\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1847\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1333" github_delivery_id=117f7030-13df-11e9-86cc-dda81a10b0f4 github_event_type=pull_request rid=bgqq1i6m7coo7fd5886g
2019-01-09T07:20:40.707027969Z |INFO| http_request elapsed=1.84232 method=POST path=/api/github/hook rid=bgqq1i6m7coo7fd5886g size=22 status=500

[bluekeyes]

That probably means that the formatting of your configuration file is incorrect. Here's an example with some fake key data:

github:
  # ...
  app:
    # ...
    private_key: |
      -----BEGIN RSA PRIVATE KEY-----
      xxxxxxxx
      xxxxxxxx
      xxxxxxxx
      -----END RSA PRIVATE KEY----- 

In particular, make sure that:

1. The private_key property line ends with a |, which indicates the start of a multiline literal
2. All the lines of the private key value (the xxxxxxxx lines from the example) have the same indentation

[vinoth241]

@bluekeyes Happy New Year!!. Sorry for late response from my end. I've corrected the keys as you mentioned above

1. Included the property line ends with a |
2. Indentation also corrected. Still the same issue persist. Any idea on where may I be wrong. I can see the webhook payload request and there responses be 500(Internal Server Error) Server debug log 2019-01-22T07:26:06.463900141Z |ERROR| Unexpected error handling webhook request error="failed to validate webhook payload: payload signature check failed\ngithub.com/palantir/bulldozer/vendor/github.com/palantir/go-githubapp/githubapp.(eventDispatcher).ServeHTTP\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/palantir/go-githubapp/githubapp/dispatcher.go:109\ngithub.com/palantir/bulldozer/vendor/goji%2eio.dispatch.ServeHTTP\n\t/go/src/github.com/palantir/bulldozer/vendor/goji.io/dispatch.go:17\ngithub.com/palantir/bulldozer/vendor/goji%2eio.(dispatch).ServeHTTP\n\t:1\ngithub.com/palantir/bulldozer/vendor/github.com/bluekeyes/hatpear.Recover.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/bluekeyes/hatpear/hatpear.go:107\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/bluekeyes/hatpear.Catch.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/bluekeyes/hatpear/hatpear.go:60\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog.AccessHandler.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog/hlog.go:180\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog.RequestIDHandler.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog/hlog.go:169\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/palantir/go-baseapp/baseapp.NewMetricsHandler.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/palantir/go-baseapp/baseapp/middleware.go:42\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog.NewHandler.func1.1\n\t/go/src/github.com/palantir/bulldozer/vendor/github.com/rs/zerolog/hlog/hlog.go:30\nnet/http.HandlerFunc.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:1964\ngithub.com/palantir/bulldozer/vendor/goji%2eio.(Mux).ServeHTTP\n\t/go/src/github.com/palantir/bulldozer/vendor/goji.io/mux.go:74\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2741\nnet/http.(conn).serve\n\t/usr/local/go/src/net/http/server.go:1847\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1333" github_delivery_id=fb26bb62-1e16-11e9-9223-a3007823bace github_event_type=status rid=bh3cb3mm7cot15vs39i0 2019-01-22T07:26:06.464108759Z |INFO| http_request elapsed=0.561091 method=POST path=/api/github/hook rid=bh3cb3mm7cot15vs39i0 size=22 status=500

[bluekeyes]

This latest error can happen if the webhook_secret value in the Bulldozer configuration file does not match the value configured in the GitHub App. You can set the value used by GitHub on the settings page for your app. The actual value is arbitrary, so I recommend generating a random 32 character string.

[vinoth241]

@bluekeyes Thank a lot for patiently answering all my queries. It is working now as expected!!.

[bluekeyes]

Great, I'll close this out, but let us know if you run into any other issues.