Improve OCI upload time

With the current approach a complete run to push all release artifacts takes ~2h. This is mostly because the runs for each target run sequentially. Each run takes ~5min currently we have 28 targets.

My idea would be to split the current program (which does manifest & index in one step) into two programs: One for the creation of the manifests create_manifest.py (called 28 times). And another one for the creation of the index (called exactly once). The outputs of the 28 manifest runs should be collected and handed over to the index creation program create_index.py by GitHub Actions.

The pushing of the OCI has 4 relevant steps:

push Blobs [oras]
create manifests [create_manifest.py]
sign manifests [cosign]
create index [create_index.py]

Data between the steps could be passed using GA outputs:

https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/passing-information-between-jobs#using-job-outputs-in-a-matrix-job

I am not 100% sure whether this approach is actually feasible but I am happy to discuss, at stake is a smaller, easier approach with a better separation of concerns between the tools. This approach makes it also possible to push blobs in parallel.

1. push blobs

this step can be spread across multiple steps, for example building and later testing of artifacts

inputs: list of files

outputs: list of digests of files

2. create manifests

inputs: list of digests of files

outputs: manifest digest

3. sign manifests

inputs: manifest digest

outputs: -

4. create index

inputs: list of manifest digests

outputs: -

@5kt this is just a very very rough sketch of the idea but maybe we can have a discussion about it next week.

gardenlinux / python-gardenlinux-cli