[Explore]: Test PRs securely on Concourse

[gcapizzi]

Background

Testing PRs on Concourse poses security risks, as it would allow anyone to run arbitrary code on our workers. We want to set up a system that allows us to manually approve the PRs that we want to be tested, so that we can vet the changes before they run.

One way of doing this would be:

• enable the Dismiss stale pull request approvals when new commits are pushed GitHub setting
• use the required_review_approvals param on the github-pr-resource

This way only approved PRs would be tested, and the code couldn't be changed without requiring a new approval. Having to approve every commit on a PR could be inconvenient though, so let's see if we can find something better.

Acceptance Criteria

A proposed Concourse/GitHub configuration that allows us to test PRs on Concourse with confidence.

[georgethebeatle]

This is partially implemented by #904. Instead of depending on at least one approval (which can be tedious for foundation team members) we made the github pr resource trigger only if a special e2e-allowed label is set on the PR. The label can be set in one of two ways:

• Out korifi-bot is automatically setting it on PRs which has an author who is on the team (a concourse job does that), because team members are trusted
• A team member can set it on a PR filed by an external contributor after looking at the code if the code looks safe to run on the test cluster.

The slight problem with the current implementation is that we are depending on assignableUsers as returned by the gh cli. We are not totally sure what is the lifecycle of that group and how people go in and out of it (it looks like a superset of the team). Maybe a better approach will be to list the github handles of @cloudfoundry/cf-k8s using the Github API (as the gh cli does not seem to be able to do that).

Here is the current script: https://github.com/cloudfoundry/cf-k8s-ci/blob/main/pipelines/scripts/allow-e2e.sh

[kieron-dev]

I've made the change:

• use GET https://api.github.com/orgs/cloudfoundry/teams/cf-k8s-members to list the team members
• use POST https://api.github.com/repos/cloudfoundry/korifi/issues/PR_NUMBER/labels to add the label.

Unfortunately, the korifi-bot needs to be in the cloudfoundry org, otherwise the first request returns a 404. So I've switched over to one of my tokens in the vault config for now. Once we get the korifi-bot added to cloudfoundry, we can put its values back in concourse/main/github/pr-label-setter in the username and token fields.

[kieron-dev]

Now korifi-bot is part of the cloudfoundry org, it can list the cf-k8s team members. I've switched vault concourse/main/github/pr-label-setter to the korifi-bot credentials and the label setter still works. So this issue looks done to me.