search

tektoncd / pipeline

A cloud-native Pipeline resource.
https://tekton.dev
Apache License 2.0
8.48k stars 1.78k forks source link

Pipelines in Kubernetes separated by folder structure #7333

Open briankurilko opened 1 year ago

briankurilko commented 1 year ago

Feature request

Currently Kubernetes lists all the pipelines you've added to a given namespace in a flat list. This is manageable until you have too many unique pipelines (30+) in a single namespace/openshift project.

It would be amazing if I was able to group pipelines based on some sort of file structure. For example, having a folder structure like this:

│   ├── pipelines
│   │   ├── pipeline_group_A
│   │   │   ├── pipeline1.yml
│   │   │   ├── pipeline2.yml
│   │   ├── pipeline_group_B
│   │   │   ├── pipeline3.yml
│   │   │   ├── pipeline4.yml

Use case

I have 3 apps (appA, appB, and appC) that are all deployed in a single Kubernetes namespace through Tekton. Each app has a "deploy to dev", "deploy to qa", "deploy to stage", "deploy to perf", and "deploy to prod" pipeline. Each app also has a "build" pipeline. The apps also have a "smoke test" pipeline and a "load test" pipeline for each environment. In total, this means that there are 48 pipelines in a single Kubernetes namespace.

48 pipelines is too large to parse through for most users. I'd like to be able to organize the pipelines with something like the following structure, so that its easier to parse:

│   ├── pipelines
│   │   ├── appA_pipeline_group
│   │   │   ├── deployment_pipelines
│   │   │   │   ├── deploy_to_dev.yml
│   │   │   │   ├── deploy_to_qa.yml
│   │   │   │   ├── deploy_to_stage.yml
│   │   │   │   ├── deploy_to_perf.yml
│   │   │   │   ├── deploy_to_prod.yml
│   │   │   ├── deployment_test_pipelines
│   │   │   │   ├── dev
│   │   │   │   │   ├── dev_smoke_test.yml
│   │   │   │   │   ├── dev_load_test.yml
│   │   │   │   ├── qa
│   │   │   │   │   ├── qa_smoke_test.yml
│   │   │   │   │   ├── qa_load_test.yml
│   │   │   │   ├── stage
│   │   │   │   │   ├── stage_smoke_test.yml
│   │   │   │   │   ├── stage_load_test.yml
│   │   │   │   ├── perf
│   │   │   │   │   ├── perf_smoke_test.yml
│   │   │   │   │   ├── perf_load_test.yml
│   │   │   │   ├── prod
│   │   │   │   │   ├── prod_smoke_test.yml
│   │   │   │   │   ├── prod_load_test.yml
│   │   │   ├── artifact_build_pipelines
│   │   │   │   ├── build_dev.yml
│   │   │   │   ├── build_qa.yml
│   │   │   │   ├── build_stage.yml
│   │   │   │   ├── build_perf.yml
│   │   │   │   ├── build_prod.yml
│   │   ├── appB_pipeline_group
│   │   │   ├── deployment_pipelines
│   │   │   │   ├── deploy_to_dev.yml
│   │   │   │   ├── deploy_to_qa.yml
.....
vdemeester commented 11 months ago

@briankurilko this is something that cannot really implemented in Tekton / Kubernetes. Kubernetes CRDs offer a "flat" structure, and we cannot change this. The only way we could do that would be with a custom cli (like tkn) and some convention on how to group objects.

One thing to note though, is that with resolvers you don't really need to have any Tekton objects in your namespaces/project.