search

tektoncd / pipeline

A cloud-native Pipeline resource.
https://tekton.dev
Apache License 2.0
8.51k stars 1.78k forks source link

Pipelinerun should be able to read from Secret and Configmap #8347

Open pombaer opened 3 weeks ago

pombaer commented 3 weeks ago

I want to reopen the following Feature Request: https://github.com/tektoncd/pipeline/issues/2688

Feature request

We are using a current version of Openshift/Tekton

Openshift/Kubernetes: Client Version: 4.16.13 Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3 Server Version: 4.16.13 Kubernetes Version: v1.29.8+f10c92d Openshift Version: 4.16.13

Tekton: Client version: 0.37.0 Chains version: v0.22.2 Pipeline version: v0.62.3 Triggers version: v0.29.1 Operator version: v0.73.1 Openshift Operator: 1.16.0

I'am looking for a feature to read Pipelinerun Parameters from a Configmap and Secret. I know it is possible within a task to read Settings from Secrets and Configmaps into a Environment Variable, but this is no option when using Openshift ClusterTasks.

Use case

In my Case i am using "buildah" ClusterTask which expects the Variable "IMAGE" for the image to build, since i am using an external Image Registry the Name must include the Hostname in order the image can be pushed, so it should be in a dynamic form. Example Pipeline:

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: imagebuilder
spec:
  description: "build image based on dockerfile"
...
  tasks:
  - name: buildah
    timeout: "0h15m00s"
    params:
      name: IMAGE
      value: "<myserver>:<port>/$(context.pipelineRun.namespace)/myimage:latest"
...

I need a "Variable" that represents my ":" value. I can use some workarrounds like Including the Servername in the Webhook Payload, create a previous tasks that reads the Secret/Configmap Value and set's a Task Result Variable which can be used in buildah value Parameter, but all of them are not satisfying.

I am not sure where would be the right place to read the Secret/Configmap value, for me it would be ok if i could do it in the Pipelinerun definition like:

...
  resourcetemplates:
  - apiVersion: tekton.dev/v1beta1
    kind: PipelineRun
    metadata:
      generateName: run-build-
    spec:
      serviceAccountName: pipeline
      pipelineRef:
        name: build
      params:
      - name: dockerregistry
        valueFrom:
          secretKeyRef:
            name: <mysecret>
            key: <mykey>
...
afrittoli commented 3 weeks ago

@pombaer Thanks for this feature request. Generally speaking, I would say that setting up the PipelineRun is the responsibility of something external to the PipelineRun. If you're using Tekton triggers, you could use a TriggerBinding to inject any configuration info you may need into the PipelineRun parameters.

Another way to achieve this with existing Tekton features is to mount your ConfigMap as a workspace in your Pipeline. You can have a Task or StepAction that reads the required values and exposes them as results, for following Tasks to consume. This approach is more verbose and worse performing, so it should only be used as a workaround.

That said, I think it's reasonable to consider pulling parameters from a ConfigMap as it would enable use cases similar to those that can be achieved via TriggerBindings for users who do not use Tekton Triggers and would like to avoid having some component that reads the config map and creates the PipelineRun. If that was implemented, we would need to make sure that the content of the ConfigMap and PipelineRun start time is added to the PipelineRun status/provenance and that Tasks, throughout the execution of the PipelineRun, only read values from said provenance.

I would be more hesitant about supporting secrets though: secrets should never be passed as parameters as that would be a security problem. We could include the secret namespace and name in the provenance, instead of the secret itself, to avoid exposing the secret, and than it would be up to the user to make sure they only include actual secrets in there, and not other configuration values that would become hidden and impoverish the provenance.

If this was implemented, it would require support in the CLI and dashboard, too.

Is this something you'd be interested in contributing?

@tektoncd/core-maintainers wdyt?

pombaer commented 3 weeks ago

You are right, Secrets could be a security issue since at least in Openshift they are visible as parameters in the WebUI, i did not thought about that, since i only needed an uncitical Value (hostname) and the value is part of the Secret which also holds Username and Password, but they are injected by the annotation "tekton.dev/docker-0: ". Nevertheless i think it should be possible to read from botz, secrets and configmaps since not every secret is critical.

I did a lot of research before posting and thind there is need for such a feature, for simpler inplementation it may is enough to make a hint in documentation of the security considerations.

I am sorry, but i cannot do contributing of the feature since i am not a developer, i am a Systems Engineer wich very basic coding skills, i do not think that you would be happy with my code ;)

But one more question, you wrote "If you are using Tekton triggers you could use a TriggerBinding to inject any configuration into", i did some research again but found no hints how to use secrets/configmaps in TriggerBinding, i only can use the payload from my Webhook and in my case some "overlays" from the "cel" interceptor, but that would not help me, do you have an working example for me?

afrittoli commented 3 weeks ago

You are right, Secrets could be a security issue since at least in Openshift they are visible as parameters in the WebUI, i did not thought about that, since i only needed an uncitical Value (hostname) and the value is part of the Secret which also holds Username and Password, but they are injected by the annotation "tekton.dev/docker-0: ". Nevertheless i think it should be possible to read from botz, secrets and configmaps since not every secret is critical.

I did a lot of research before posting and thind there is need for such a feature, for simpler inplementation it may is enough to make a hint in documentation of the security considerations.

Thank you for taking the time to research and create this feature request!

I am sorry, but i cannot do contributing of the feature since i am not a developer, i am a Systems Engineer wich very basic coding skills, i do not think that you would be happy with my code ;)

I will mark this as "help-wanted" then. Maybe there are developers within your company who might be interested in contributing to this? @vdemeester @pritidesai is this of interest to you?

But one more question, you wrote "If you are using Tekton triggers you could use a TriggerBinding to inject any configuration into", i did some research again but found no hints how to use secrets/configmaps in TriggerBinding, i only can use the payload from my Webhook and in my case some "overlays" from the "cel" interceptor, but that would not help me, do you have an working example for me?

You are right, you cannot use secrets or configmaps directly, what I meant is that your trigger bindings can basically act as configmaps since you can provide fixed values within for the parameters you want to inject. See this example - depending on filters in the trigger a different binding will be used and different inputs will be passed to the underlying pipeline run - and the actual values are defined in the bindings directly.