search

kata-containers / kata-containers

Kata Containers is an open source project and community working to build a standard implementation of lightweight Virtual Machines (VMs) that feel and perform like containers, but provide the workload isolation and security advantages of VMs. https://katacontainers.io/
Apache License 2.0
5.34k stars 1.04k forks source link

Ease the management of CI jobs #1298

Open cpmeadors opened 3 years ago

cpmeadors commented 3 years ago

User story:

As a contributor to the project, I want to add or modify the CI jobs run for a PR in order to cover new code changes, preferably in the same PR.

Additional information:

This is part of the effort to improve CI in general that has been discussed multiple times in the Architecture meeting. It most recently was brought up in light of merging the tests repo in the the kata-containers repo and how that would require changes to CI. Below are some use cases that are problematic:

If I want to change a job because I, for example, need to export a new variable in the build script then I need to change the job XML found in [2]. Those XMLs aren't meant for humans to edit, they are just Jenkins internal representation of jobs. Once I changed the XML (actually several xmls, one per job), I opened a PR to [2]. Once merged, someone needs to manually update the job in the Jenkins instance. I've not idea who is going to do it. This is completely inefficient and error prone.

Now suppose I want to create a new job then I need to somehow write several XMLs. I can copy-and-edit from existing XMLs. And I won't waste time explaining how insane it is.

Because this manual handling of the jobs, what is running in the instance may not be exactly the code found in the repo [2].

It has a complex combination of tests vs configuration vs OSes being mapped to a lot of jobs, and this is all managed manually (and editing the XMLs!). Again completely error prone and inefficient.

Acceptance criteria:

CI jobs are managed as code CI jobs are clearly documented CI jobs can be updated with code changes in one PR

wainersm commented 3 years ago

There are a few technologies which can be used to achieve the goal of this use case. On https://github.com/kata-containers/ci/pull/359 I've done a PoC of the Jenkins Job Builder (JJB) tool.

fidencio commented 3 years ago

I got to this issue while going through the open issues with the "needs-review" label. As this is a valid enhancement request, I'm removing the "needs-review" label, and I'm adding the "area/ci" one.

@cpmeadors , feel free to adjust the labels accordingly, if you notice that something is wrong.

wainersm commented 2 years ago

IMO, the first acceptance criteria "CI jobs are managed as code" was met with the jobs-builder (https://github.com/kata-containers/ci/tree/main/jobs-builder) although just a few jobs are effectively using that tool.