Implement a mechanism that build and push containers to a registry (quay.io atm)

[gianarb]

Currently, we have a CI/CD mechanism to create and publish the YAML manifests used by ArtifactHub

https://github.com/tinkerbell/hub/blob/main/cmd/gen/cmd/gen.go

It works fine and it will work even better when #5 will be implemented.

We do not have a way to figure out changes in the code that require a docker image to be built (or re built) and published to quay.io (right now we use https://quay.io/organization/tinkerbell-actions/ ).

This process is something I do manually. Before merging a PR that requires an image to be built I run:

docker buildx build ....

And I push the image to Quay. I would like to automate myself with a few lines of Go that use buildkit directly for example. The image has to be multi-arch.

[nicklasfrahm]

I might be understanding it wrong, but you want to automate building the hub images in this repository right? Would it be possible to do this via GitHub actions? It is possible to filter by files and folders changed. I built something similar there: https://github.com/nicklasfrahm/labman/blob/main/.github/workflows/api-server.yml

[gianarb]

I think you got the overall desire but let's make it clear.

Ideally, this repo will grow over time, and I would like to automatically figure out which Tinkerbell Actions changed and recompile them (not all of them, but only the one that needs to be re-built).

Right not for example we have two actions:

/actions/rootio/v1 /actions/img2disk/v1

If somebody opens a PR and changes

/actions/img2disk/v1

I expect to have CI building and releasing /actions/img2disk/v1.

At some point in time, we will have:

/actions/rootio/v1 /actions/rootio/v2 /actions/rootio/v3 /actions/img2disk/v1 /actions/foo/v1 /actions/foo/v2 /actions/bar/v1

If somebody in a PR changes:

/actions/rootio/v3

I want CI/CD to detect that only /actions/rootio/v3 changed, rebuild and push it when the PR is merged to master.

GitHub Actions are a mechanism that GitHub exposes to run code packaged as a container image. We use Github Action for this project, and yes, at some point, we will need a GitHub Action. Personally, I like to have those workflows as code as I did in this repository for the logic that from a Tinkerbell Action README.md generates an ArtifactHub manifest https://github.com/tinkerbell/hub/blob/main/cmd/gen/cmd/gen.go. That logic runs as GitHub Action here https://github.com/tinkerbell/hub/blob/main/.github/workflows/ci.yaml#L18.

Looking at your example, I think you are using a feature exposed by GitHub Action that runs a job only if a specific path changes. I am not sure it gives us enough granularity to identify the single Tinkerbell Actions that changed in a PR.

[nicklasfrahm]

Okay, I see. My proposal does not scale well. Which mechanism would you preferrably use to detect changes? Git or anything else as you mentioned manifest hashes?

[gianarb]

The hash can work but I think it is easier to write something in Go that inspects the file changed in the pull request and figure out what has to be built. You can do it with bash as well, but Go will make testing a bit easier, at least for my experience

[nicklasfrahm]

Go sounds great. Last question, would you like a second, seperate binary or rather a subcommand? Something like hub gen could replace the current gen command and then hub build could be used for the new command?

[nicklasfrahm]

Waiting for https://github.com/moby/buildkit/issues/1973.