Add pipeline name as prefix to context

[tgolsson]

I'm trying out concourse and setting up for a monorepo/multi-project workflow, and it's easy to end up having test/validate/build jobs in multiple pipelines for the same repository, etc. I'm wondering what your stance is on prefixing the pipeline name to the context, if the information is available? As an example, instead of just test for the test job it'd be for example publish-docker-image/test if this test is part of the publishing pipeline.

[marco-m-pix4d]

Hello @tgolsson, we were using as GH status context the format PIPELINENAME/JOBNAME, but this was making impossible our workflow of using branch protection rules, see #11. On the other hand, I can see the usefulness of prepending the pipeline name.

I would suggest to think how you would use the branch protection rules in your case and, if it would work, I am open to a PR. We would have to introduce this feature as an optional attribute in the source configuration.

[tgolsson]

I see the problem! I've not thought too much about branch protection rules and interaction with concourse. From the description I take it you run a multi-repo setup with per-branch pipelines? I've not gotten that deep yet but it seems like the branching story is more of a concourse issue. For me, I'd have the problem that I'd not be able to see all my "test" statuses so I also can't add branch protection rules.

I'm wondering whether maybe this should be a prefix instead, i.e, in the resource declaration we could declare a status_context_prefix: feature-branch? That decouples it from branch-based pipelines while still preventing collisions.

[marco-m-pix4d]

Currently Concourse doesn't have a out-of-the-box solution for what I call "feature branch pipelines" vs "main branch pipeline". Forget for a moment wether you have a monorepo or not, this is for the most part independent. Think about a repo with one project, to make things simple to reason about.

Everything works fine if you use only one pipeline, to represent the status of the main branch, and maybe one pipeline (or job) to represent pull requests.

What we didn't like about that approach is that it goes back to Jenkins: each build in the history of the PR pipeline represents a different, almost random PR. We wanted instead a 1:1 mapping between a feature branch and a pipeline, and that pipeline must be identical (modulo maybe deploy steps) to the main pipeline.

So for example for project banana, branch feat-1 takes pipeline name banana-feat-1 and branch feat-2 takes pipeline name banana-feat-2.

Now if we go to the branch protection rules, since the pipeline names are ephemeral (the feat branches will be merged and the pipelines be destroyed), we do not want to add a branch protection rule per pipeline, we want to specify only the job names, for example build, test, foobar.

If we were to add the pipeline name, we would have to use rules like

banana/test banana/build

this would mean that the rules would not be applied to our feature branch pipelines, while with the current situation, with context simply build and test, the rules apply to ALL branches.

BUT, if you plan to do the "classic" Concourse way (main pipeline + PR job), then you can get away with a context of PIPELINE/JOB.

Lastly, I am not sure how one would use the status_context_prefix thing. Is that part of the GH API ?

[tgolsson]

Thank you for the thorough reply! I understand your use-case more clearly, and I understand why - Buildkite works the way you have it set up; and I like that you get a per-branch history view. However, Buildkite handles that on their side, and uses the same status check across all pipelines - giving you a collated view of builds happening on a single pipeline, which solves both problems.

I think my amended proposal solves that, though I wasn't maybe clear. We currently declare a resource block like so:

# This is the status object that will be updated in each job
- name: gh-status
  type: cogito
  check_every: 1h
  source:
    owner: tgolsson
    repo: concourse-ci-test
    access_token: ((status-token))

I'd simply amend one line to this configuration; which maybe is clearer named as context_namespace. This is a user-provided string which if non-zero gets inserted into the context like so: $context_namespace/$job_name, otherwise use $job_name. That should work for your use-case (stable statuses) and for mine (multiple pipelines on a single commit with same job names).

[tgolsson]

Hello again @marco-m-pix4d,

I went ahead and implemented the above in a fork: https://github.com/tgolsson/cogito/commit/1b2db8c0ba750df831cf7a1090aeb6d75e728d4f. If you want to try it before you buy I've made it available as a public image here: harbor.sbg.dev/library/cogito@sha256:09991c679e0737abae5ea6272e2af4f4314f2e2f8f832e69a09157dfe6b6899e.

The final status check looks like this on a "regular" commit (not a PR), with context_namespace: test: