Multiple Pipeline Configs

[ecrupper]

Description

What is your idea?

It'd be nice to have multiple separate pipeline files. It would help to separate and organize pipelines. This would be similar to how Github Actions and Concourse works.

Value

Currently with Vela we have a 2k+ lines in .vela.yml which is hard to manage due to:

• careful ordering of steps
• hard to find related steps in such big file
• increased chance that change will impact other "pipelines"

We tried to do separation of pipelines using templates in vela. However, this means we would not be able to leverage stages as you can not define stages inside a template and can only use a template in a step. Plus, all templates/pipelines still would need to be tied together by the single vela.yml.

[ecrupper]

Note on this: https://github.com/go-vela/server/pull/597 has addressed the lack of stages in templates

[cognifloyd]

I would like to see this. What would the UX be?

I think we should use subdirectories similar to Github Actions.

root.git/
├── .github/
│   │
│   ├── actions/
│   │  └── my-action/
│   │     └── action.yaml
│   │
│   └── workflows/
│      ├── build.yaml
│      ├── test.yaml
│      └── deploy.yaml
│
... other repo contents

I think we can do something similar with a .vela directory:

root.git/
├── .vela/
│   │
│   ├── plugins/
│   │  └── my-plugin/
│   │     └── ... whatever an in-repo plugin might look like in the future
│   │
│   ├── templates/
│   │  └── my-template.yaml
│   │
│   └── pipelines/
│      ├── build.yaml
│      ├── test.yaml
│      └── deploy.yaml
│
... other repo contents

Aside: I prefer .yaml over .yml, so whatever we use should support that (I hope).