leverage tekton SCM trigger support

gabemontero commented 4 years ago

today the https://github.com/tektoncd/triggers project provides github/gitlab styled webhook trigger support for creating tekton API object to trigger their CI/CD pipelines.

This is akin to the webhook support in openshift build v1

In theory build v2 could deploy tekton trigger event sinks in conjunction with their controller and define event triggers that will run the same underlying tekton objects which buildv2 maps too.

zhangtbj commented 4 years ago

We used the tekton triggers in our source-to-image PoC. It works well. We defined two kinds of triggers: Github webhook triggers (source code change and rebuild) dockerhub webhook triggers (builder or baseimage change and rebuild)

for each webhook triggers, there is a trigger event listener in the cluster that keep monitor the event from each webhook then trigger the Tekton trigger template to do the source-to-image build task.

If you interest, I can show your our PoC demo and what we did by using it.

gabemontero commented 3 years ago

Jotting down some notes/updates from some of my recent upstream Tekton Trigger WG activities

These notes should be consider complimentary or building upon to @zhangtbj 's comments here back in March, as his description there lines up with my improved understanding/interpretation since the initial description in this issue of what is available with tekton triggers.

1) https://github.com/tektoncd/community/pull/229/files contains a proposal that will improve the experience around plugging in interceptors

2) while technically SCM based triggers like github, etc. exist in triggers today, I'm currently of the opinion that we could proscribe replacing the built in versions of those for any deployed EventListeners that are intended to serve an associated shipwright deployment, plugging in our own implementations of those github etc. interceptors, as the new TEP would allow

3) then as @zhangtbj mentioned, our new interceptor (a webhook interceptor that would replace the built in interceptor) would construct trigger templates with the underlying tekton tasks/taskruns etc. created by the shipwright controllers

you can either inline those trigger templates def, or use a reference approach, where the event listener sink then reads it it (see sink.go's processTrigger method, which calls ExecuteInterceptor followed by ResolveTrigger ... fwiw the trigger code looks at the in line fields first, and if it sees nothing there, tries the ref)

4) mult tenent event listeners (https://github.com/tektoncd/community/blob/master/teps/0009-trigger-crd.md) from pipelines team here at redhat have also gained some traction upstream

I think we'll want to consider the pros/cons cluster scope vs. per namespace EL's as we go down this path

adambkaplan commented 3 years ago

Moving this to the post GA backlog. Triggers are an important feature, but not absolutely required.

sbose78 commented 3 years ago

I was exploring the possibility of creating the following resources when the user expresses the intent to use webhooks with GitHub(for example):

TriggerBinding
TriggerTemplate ( to create a BuildRun )
EventListener

This experiment didn't succeed because the admission webhook doesn't let TriggerTemplate specify non-Tekton resources for creation.

Potential next steps:

Propose upstream that "Tekton Triggers" as a project be used for creation of arbitrary resource types ( and not just Tekton types ). This would need the design to address security
Propose that we make the "supported TriggerTemplate types" configurable, per installation of Tekton.

Questions that need to be answered in future:

Should we have an EventListener per Build
Should we have an EventListener per namespace.
Should we have an EventListener per cluster, for handling builds ( definitely not keen on this ).

gabemontero commented 3 years ago

I was exploring the possibility of creating the following resources when the user expresses the intent to use webhooks with GitHub(for example):
* `TriggerBinding`

* `TriggerTemplate` ( to create a `BuildRun` )

* `EventListener`
This experiment didn't succeed because the admission webhook doesn't let TriggerTemplate specify non-Tekton resources for creation.

Yes a subject that has come up before in the upstream triggers WG.

Main reason it has not happened yet is security concerns, as you noted below.

Potential next steps:

* Propose upstream that "Tekton Triggers" as a project be used for creation of arbitrary resource types ( and not just Tekton types ). This would need the design to address security

@ImJasonH and I had a recent slack discussion on this as well. I would sign off on the upstream proposal if

it is opt it
can be enabled / disabled on a per namespace basis
which precise objects that could be added could be specified

rbac around this, akin to our csi driver stuff, would be nice too

* Make the "supported types" configurable per installation of Tekton.

gabemontero commented 3 years ago

The impersonation piece I placed into triggers could also help facilitate expanding the list of objects triggers can create.

gabemontero commented 2 years ago

UPDATE: so @sbose78 's https://github.com/shipwright-io/build/pull/912 is a take at SCM based triggering

This issue was originally just SCM based triggering

For image based triggering, upstream would either start or build off of https://github.com/shipwright-io/community/blob/main/ships/0020-shipwright-image-api.md

For openshift imagestreams, feels very much downstream only at the moment, unless a shipwright image api implementation sheds further light on that

shipwright-io / build

leverage tekton SCM trigger support #51