Add Fedora integration CI job

[dmnks]

While testing rpm in a "sterile" environment where no distribution specific rpm configuration is present has its value, there are cases where integration with an actual distribution such as Fedora may reveal bugs in otherwise innocent looking patches, such as the just-discovered #3115.

The test-suite (mktree script) should gain an option that installs rpm into the /usr prefix (instead of the default /usr/local) so that it reflects rpm's actual deployment better, specifically so that any distribution-provided macros (like redhat-rpm-config on Fedora) are effective during the test-suite. Then, we can add another CI job that runs the test-suite against such an image (or something of that sorts). Details to be figured out.

[dmnks]

Note that it might also turn out that many of our existing tests aren't compatible with e.g. redhat-rpm-config's macros, in which case we might need to either adjust those or disable them.

[dmnks]

At that point, though, we might as well have a proper spec file and build an actual rpm package out of the sources, much like what Packit does... I need to think about this more.

[dmnks]

Here's another, even more embarrassing case that would've been discovered if Mock had been actually tested with the new stack: #3120, #3121

[pmatilai]

To summarize a private discussion from elsewhere, we should have a test-suite mode that lets us run the tests on the system rpm inside the image. That would allow testing the vanilla distro rpm at any time (could be useful for testing against regressions in other components for example), but in particular we could then in distro CI update the system rpm with the packages that just got built, and run the test-suite on the real production thing.

There is of course a whole army of Matryoshkas involved in that.

[dmnks]

Basically, allow for using a custom image. I wonder if this isn't technically the same as #2877...

[pmatilai]

I don't think it's the same, because "make install" missing from the process is a pretty drastic difference.

Another random thought on the topic (greetings from Wizard :smile: ): to allow "re-entry" to the test-suite from a basically different context, cmake itself needs to run with matching parameters to initialize the test-suite and all, even if we don't actually build rpm there. For that, a handy looking path could be using the cmake find_package module we ship for rpm. It probably doesn't have all the relevant details just now but I suppose it could have them.

[dmnks]

I don't think it's the same, because "make install" missing from the process is a pretty drastic difference.

It's not really. The tests themselves don't care about where the rpm installation comes from, they just need it to be in the image, in the default system paths.

We might as well just skip the "make install" step, keep the stock rpm installation in place, and the tests would happily test that.

That way, you could supply your own software in the image to test against, including rpm itself (that's the idea in #2877 at least).

Another random thought on the topic (greetings from Wizard 😄 ): to allow "re-entry" to the test-suite from a basically different context, cmake itself needs to run with matching parameters to initialize the test-suite and all, even if we don't actually build rpm there. For that, a handy looking path could be using the cmake find_package module we ship for rpm. It probably doesn't have all the relevant details just now but I suppose it could have them.

I'm not following :sweat_smile: Mind to rephrase?

[dmnks]

We do feed a couple of cmake variables into our test script when building it so those things could/should be decoupled, though.

[pmatilai]

Heh. Maybe being more concrete will help. Taking the Fedora infra as an example:

• a build happens in koji and produces binary rpms
• you submit an update which spins up the CI, which we want to run the test-suite on the just produced rpms
• where do we get the test-suite from? well okay, probably unpack the src.rpm and run %prep and then tell it to run using with image rpm

... which will supposedly get the same exact parameters, so ... never mind :laughing: I was basically thinking of letting an (upstream) source checkout be used by just pointing it to the configuration we get from the rpm or something. Clearly, the Matryoshkas caught me here :laughing:

[dmnks]

No worries. Wizard (Wizard, the professional rpm consultation dog, for the uninitiated :smile:) was clearly lost in the Matryoshka layers just as much as we are.

But OK:

* where do we get the test-suite from? well okay, probably unpack the src.rpm and run %prep and then tell it to run using with image rpm

This is what I was missing, thanks. I was assuming we'd be running the Fedora CI job from within the upstream sources fetched from GH, like we do when testing a local build with make check.

But that won't work because the test-suite is tied to the sources (i.e. covering the available functionality) and so we must use the version from the src.rpm, indeed.

It could go like this:

1. Unpack the src.rpm and run %prep (like you said)
2. Go to the build directory
3. Run mktree build with a switch that tells it to use a given rpm package file instead of a custom cmake build inside the Dockerfile (not implemented yet)
4. Run mktree check

Basically, a variant of what we do in our upstream CI, just with the necessary bits required to unpack the src.rpm, find the build dir and run the mktree commands.

At that point, there's nothing stopping us from having a similar job in our upstream CI as well, one that would grab the latest nightly build of rpm (from your COPR perhaps).

[dmnks]

... and of course, to tie everything together, we could have a make install-check variant that does this locally from your cmake build (without actually using the cmake build, lol, much like make ci now).

[dmnks]

The real question is, how do we layer the Matryoshkas so that it's easy to use. We could do all this logic inside of mktree build, or have yet another script (akin to that GH workflow yaml) that does this. The advantage of the former (built into mktree) is that we could then reuse the same logic in upstream CI... :sweat_smile:

[dmnks]

Except you need to obtain the mktree script in the first place :facepalm: So it can't be there. This will simply have to be handled from the outside, i.e. from the Fedora CI job (similarly to that GH yaml). It's not as much logic, though, so should be fine.

[dmnks]

Anyway :smile: I'll try to sketch something up during one of these Fridays and we'll go from there. It's easier to reason about a concrete solution at hand.