Closed michaelsauter closed 1 year ago
A few more thoughts from me on my proposal above:
stage
) from environment
. This would need to be replaced by another mechanism (e.g. default or mandatory parameter)scope
field is not part of the Tekton Param typebranchParams
is not ideal as we'd likely want the same concept to work for tags as wellThe alternative could be rewritten like this:
branchParams:
- branch: develop
params:
- task: deploy # name of task in pipeline run spec
param:
name: target-namespace
value: foo-dev
Potentially, task
could be optional, and if it isn't set, the param could be a pipeline-level param ... but it looks a bit funny then.
From the description:
- branchToEnvironmentMapping overlaps with the recently introduced pipeline trigger functionality. Using triggers, it would be possible to have a separate pipeline for each environment, and hardcode the target environment in each pipeline.
At first glance having pipelines per environment and then hardcoding or better configuring target environment related information outside of the pipeline makes sense to me. Your proposal does not pursue this it seems? Am I wrong or what are the reasons not to do that?
Right, I think having a separate pipeline per environment is not ideal. Let's use an example. Assume we have 3 environments, dev, test, prod. Currently one could achieve it like this, assuming the deploy task learns additional parameters:
pipeline:
- trigger:
event: ["repo:refs_changed"]
branches: ["develop"]
tasks:
- name: deploy
taskRef:
kind: Task
name: ods-deploy-helm
params:
- name: target-namespace
value: foo-dev
workspaces:
- name: source
workspace: shared-workspace
- trigger:
event: ["repo:refs_changed"]
branches: ["release/*"]
tasks:
- name: deploy
taskRef:
kind: Task
name: ods-deploy-helm
params:
- name: target-namespace
value: foo-test
workspaces:
- name: source
workspace: shared-workspace
- trigger:
event: ["repo:refs_changed"]
branches: ["master"]
tasks:
- name: deploy
taskRef:
kind: Task
name: ods-deploy-helm
params:
- name: api-server
value: https://api.example.com
- name: api-credentials-secret
value: secret-for-api-example-com
- name: registry-host
value: registry.example.com
- name: target-namespace
value: foo-prod
workspaces:
- name: source
workspace: shared-workspace
I think this is a lot of duplication and not desirable. I believe we need to find a way to easily share pipeline definition across environments.
I recently was considering something like this:
pipelineTriggers:
- event: ["repo:refs_changed"]
branches: ["develop"]
pipeline: default
params:
- name: deploy.target-namespace
value: foo-dev
- event: ["repo:refs_changed"]
branches: ["release/*"]
pipeline: default
params:
- name: deploy.target-namespace
value: foo-test
- event: ["repo:refs_changed"]
branches: ["master"]
pipeline: default
params:
- name: deploy.api-server
value: https://api.example.com
- name: deploy.api-credentials-secret
value: secret-for-api-example-com
- name: deploy.registry-host
value: registry.example.com
- name: deploy.target-namespace
value: foo-prod
pipelines:
- name: default
tasks:
- name: deploy
taskRef:
kind: Task
name: ods-deploy-helm
workspaces:
- name: source
workspace: shared-workspace
This would avoid duplication of the pipeline. Further, it would repurpose the existing pipeline triggers spec to also define params, without the need of a separate config option like the originally proposed branchBasedParams
field. As a downside, pipelines need a name
field now so that they can be referenced from a pipeline trigger. The params given in pipeline trigger could either be pipeline-level params (for usage in multiple tasks), or, as shown above, params for a specific tasks by adding the task name as a prefix followed by a dot.
@henrjk Did that clarify your question? What do you think about the new evolution of the idea?
Yet another iteration of this is now ready for review in https://github.com/opendevstack/ods-pipeline/pull/685.
Background
ods.yaml
is the central point to configure a pipeline run. One common configuration need is to deploy to different environments based on the branch. Inline with the Jenkins shared library,ods.yaml
allows this via a combination ofbranchToEnvironmentMapping
andenvironments
. The pipeline manager selects an environment to deploy to based on this configuration, then injects this as ODS context into the pipeline, which theods-deploy-helm
task uses to know where to deploy to.While this works for now, the solution has several issues:
ods-deploy-helm
with theods.yaml
file - changes inods-deploy-helm
may require changes toods.yaml
which could potentially lead to issues for other tasks relying on that information.ods-deploy-helm
. If other tasks have similar needs (e.g. a task deploying a Terraform module) the configuration options available inods.yaml
may not be a good fit or may not even be enough informationbranchToEnvironmentMapping
overlaps with the recently introduced pipeline trigger functionality. Using triggers, it would be possible to have a separate pipeline for each environment, and hardcode the target environment in each pipeline.Proposal
Instead of
branchToEnvironmentMapping
andenvironments
, we could have a new field named e.g.branchParams
. This field would allow to set parameters depending on the current branch. The pipeline manager would then use this field to inject the params into the pipeline.For example, the configuration could look like this:
A few notes:
branchToEnvironmentMapping
. However, we would likely want to allow multiple matches so that one can have generic params (branch: *
) and branch-specific params. If a param is specified in multiple matching branch patterns, the most specific one would win.params
list would be of type Tekton Param so that they can easily be used in Tekton tasks as-is. Therefore, those the param values could also be arrays or, following the latest changes in Tekton, objects.ods-deploy-helm
task would need to change to accept all fields of the currentenvironment
struct as params So instead of knowing an environment name, and then reading from ods.yaml, the task would have params for api-server, registry, target-namespace etc.There could also be a variation of this configuration, allowing to also set task parameters directly. This would reduce verbosity, but may be more likely to break. Technically, the parameters would be set on the pipeline run spec in the pipeline manager.
Of course there are many different ways how the configuration could exactly look like e.g. instead of having a
scope
field one could use a map of keys (pipeline / task names) to params.Alright, at this point my goal is to just throw this out there and get some feedback on the idea. What do you think @henrjk @kuebler @henninggross @gerardcl?