search

coreos / fedora-coreos-tracker

Issue tracker for Fedora CoreOS
https://fedoraproject.org/coreos/
263 stars 59 forks source link

RFC: A new `continuous` mechanical FCOS stream #910

Open jlebon opened 3 years ago

jlebon commented 3 years ago

Describe the enhancement

In the Fedora Atomic days, we had continuous builds from git-main of all the important upstream repos via rpmdistro-gitoverlay. I've been trying on and off to replicate something similar for FCOS using Packit, though that stalled on https://pagure.io/releng/issue/9801.

In a discussion with @bgilbert, we realized a simpler approach is to just use the fcos-buildroot image to make && make install from all the repos we care about into overrides/rootfs and go from there. This should naturally work on all arches too. So with not too much effort, we could have a mechanical stream of FCOS tracking git-main of all our components. This would be useful for CI purposes only. We'd never release/promote the content there.

The primary negative is simply the added overhead of tracking failures in yet another stream. It's also not exactly the same as using RPMs because removed files won't actually be removed, which could lead to false positives. Also, the rpmdb would be incorrect.

Still, it's worth considering I think given the effort it'd require to set up.

jlebon commented 3 years ago

I guess maybe to clarify the framing, there are two separate questions:

  1. Should we pursue the goal of a continuous FCOS stream tracking git-main of all our projects?
  2. If so, what is the best way to achieve this goal? Should we just use overrides/rootfs, or try to push forward on the Packit path (or something else)?
travier commented 3 years ago

I like this. Some thoughts:

cgwalters commented 3 years ago

Heh I had missed this issue. OK so I had worked on https://github.com/coreos/rpm-ostree/pull/3032 which will hook into copr for this. My immediate motivation actually in this case is I want to create quay.io/coreos-assembler:continuous so I have my updated error prefixing PRs. In the interim I spun a custom image which was fine, but having a continually regenerated image would be clearly useful.

I think it's fine if short term a continuous build is only x86_64, which COPR supports fine.

cgwalters commented 3 years ago

One argument for using COPR is other Fedora editions using rpm-ostree would find it useful.

jlebon commented 3 years ago

One intermediate approach to this which doesn't involve a new stream and the overhead it involves is to target rawhide only and have the rawhide stream pick them up.

Even if we do a separate continuous stream which is based on testing-devel's releasever we should still do that IMO.

Obviously if we go the COPR/Packit route, we can still also build the RPMs for the current releasever too. That would be useful for FCOS developers to drop into overrides/rpm locally.

lucab commented 3 years ago

We briefly touched this topic in the last meeting. Lots of questions still open, it will be brought back in the next meeting.

jlebon commented 2 years ago

https://github.com/ostreedev/ostree/pull/2534 https://github.com/coreos/ignition/pull/1317 https://github.com/coreos/butane/pull/323 https://github.com/coreos/coreos-installer/pull/782 https://github.com/coreos/afterburn/pull/704

jlebon commented 2 years ago

OK, so it turns out setting up COPR for other repos is pretty easy so it seems silly to go with the make install hack. I've now set up automated builds for ostree, butane, and ignition in https://copr.fedorainfracloud.org/coprs/g/CoreOS/continuous/packages/ (and rpm-ostree was already set up by Colin).

Will work on adding more packages to the list in the background, but my proposal for this now is:

Frequency-wise, it could just match rawhide for now. I'd like to have both of those rebuild more frequently, but we should really implement some GC first.

cgwalters commented 2 years ago

Let's not build all the artifacts for the continuous stream: maybe just QEMU, live artifacts, and AWS + single region upload? (Full builds can be requested on-demand.)

We could even just start with only pushing a container image; xref https://github.com/coreos/coreos-assembler/issues/2685

jlebon commented 2 years ago

Let's not build all the artifacts for the continuous stream: maybe just QEMU, live artifacts, and AWS + single region upload? (Full builds can be requested on-demand.)

We could even just start with only pushing a container image; xref coreos/coreos-assembler#2685

Hmm, I think having at least QEMU and AWS images would make it more useful to developers/testers so they can quickly try it out. E.g. cosa buildfetch --stream continuous && kola spawn -p aws (or --artifact qemu && cosa run).

dustymabe commented 2 years ago

I have to admit that I think this would be useful, but I'm really not excited about looking at failures from another stream.

dustymabe commented 2 years ago

We discussed this in the community meeting yesterday.

12:05:35  dustymabe | #agreed While we see value in a continuous stream we aren't
                    | wholly convinced the benefits qualify adding another stream at
                    | this point. We'd like to continue discussion in the ticket and
                    | collect examples of issues we think would have been caught by a
                    | continuous stream before landing in testing-devel.
cgwalters commented 2 years ago

(related) PR in https://github.com/coreos/fedora-coreos-config/pull/1710