Feature Request: kodiak label should run github actions with secrets

[styfle]

I'm not sure if this is possible, but I would like to utilize Kodiak to solve a very annoying issue for public repos.

The issue is that PRs from someone outside the org come from a forked repo instead of a branch inside the main repo, so GitHub won't provide the secrets to GitHub Actions, and thus CI fails.

Since we already use a label, automerge, to tell Kodiak to merge when all GitHub checks pass, I think it would be great if it could detect a fork and run the last commit with elevated privileges.

I think this is possible for a couple reasons:

1. Kodiak already has elevated permissions because it can merge, users outside the org cannot.
2. I was able to make forked PR run CI with secrets by checking out the forked branch, then creating a new PR with my account. At that point, CI ran for both my new PR and the original PR (likely because the commits were the same).

[chdsbd]

Hi @styfle,

Thanks for opening this issue.

I did some testing and I was able use a GitHub App's authorization token to push commits from a fork to an upstream repository. So with that branch the GitHub App could create a PR and then the commits would get tested.

Here's the flow I envision:

1. User adds label "test-pr" to PR from fork
2. GitHub App clones the forked repository and pushes the fork's branch to the main repository.
3. GitHub App creates a PR using the newly pushed branch. This triggers CI jobs to run with secrets.
4. GitHub App closes the PR when it's no longer needed

I think if this functionality were to be added to Kodiak we'd probably want a separate service to handle cloning repositories and pushing commits. Maybe this could be a completely separate GitHub App?

I'm curious to hear your thoughts.

[styfle]

Hi @chdsbd

Thanks for the quick response!

Yes, those steps sound correct. The more I think about it, the more it does seem to be a different GitHub App because I forgot about the use case where Kodiak can be used to merge without labels. Another thing to consider is the case when you add a label, CI runs, but fails. How do you restart once new commits are pushed? Probably remove and add the label? Or perhaps this is the job for a /test command in a comment?

Ideally GitHub Actions would have a button to authorize CI when secrets are found but I don't think this is coming anytime soon.

[chdsbd]

I saw this new post from GitHub and it reminded me of this issue.

https://github.blog/changelog/2021-05-06-github-actions-beta-api-to-approve-actions-from-forks/

[styfle]

That might work if it adds secrets upon approval 👍

One thing to also consider is that a malicious user might wait for "approval" and then push another commit while CI is still running. So that subsequent push should probably still require approval before adding secrets.