eskultety
commented
3 months ago I was able to build cachi2 on a UBI image that can be registered with subscription-manager with just one hack - fetching an RPM for createrepo_c from the Rocky repo.
And ^this was exactly the reason why we chose Rocky over UBI, because we'd ultimately end up with a complete Frankenstein of an image where besides fetching Golang/NodeJS from their official images we'd install individual RPMs from other RPM distros due to UBI being inadequate in terms of repo contents for cachi2 purposes (many package managers support => many potential direct dependencies) which,as a use case, ultimately may not fall exactly into UBI's intended purpose. Speaking in numbers, here's a rough comparison of the number of packages available in both
# UBI9
$ dnf list --all | wc -l
2261
================
# Rocky
$ dnf list --all | wc -l
6825The issue is that the rpm support is using asyncio to download RPMs instead of using dnf. This circumvents the certificate handshake the is normally handled by the dnf subscription-manager plugin. I believe that this image and the prefetch task updates I have would enable authenticated RPM prefetch..._if we can switch to using 'dnf download' for fetching RPMs.
^This is a direct result of the original design which heavily revolved around the idea that a package name wouldn't be enough to identify a package uniquely using PURL which is still a problem to this day and apart from https://redhatproductsecurity.github.io/security-data-guidelines/purl/ we don't anything in the official PURL upstream guideline (not a blocker, the project looks like it needs more active maintenance anyway).
TL;DR
We could start recognizing a new field name in the RPM lockfile and use DNF to download a package.
Regarding the hack for createrepo_C, the only alternative I could find is to get it from a repo that requires an entitlement which would make this container non-redistributable. I tried to build a createrepo_c rpm from source but the dependencies aren't available either.
See my response at the top. ...
AFAICT subscription manager scans the underlying platform before enabling repositories, so in general there isn't a single platform that would satisfy all use cases and all builds depending on the intended product against which the build is supposed to run, is there? In other words, not all RH product will always support all its active RHEL platforms, will they? For example, Satellite can be deployed on RHEL-7 and RHEL-8 (but not RHEL-9) depending on the exact Satellite version one is installing, so on UBI-9 the subscription manager would never enable the repositories, see the example below. This all means we'd have to maintain multiple concurrent cachi2 container images to satisfy just Red Hat pipeline build needs - sounds like a downstream candidate IMO rather than upstream. Then again, upstream aside, someone will still have to solve which UBI-based cachi2 image would be used in such a pipeline based on what the intended build is for IIUC.
Another minor issue is multi-arch. With RPM URLs, cachi2 doesn't need to care about non-x86 arches. With subscription manager the consumer has to make sure that all arch repos are enabled using the activation key so that dnf download --arch would work, but admittedly this problem relies outside of this project.
# UBI8
[root@64b60c84e7ef /]# dnf repolist
Updating Subscription Management repositories.
repo id repo name
rhel-8-for-x86_64-appstream-rpms Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs)
rhel-8-for-x86_64-baseos-rpms Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs)
satellite-6.15-for-rhel-8-x86_64-rpms Red Hat Satellite 6.15 for RHEL 8 x86_64 (RPMs)
satellite-6.15-for-rhel-8-x86_64-source-rpms Red Hat Satellite 6.15 for RHEL 8 x86_64 (Source RPMs)
ubi-8-appstream-rpms Red Hat Universal Base Image 8 (RPMs) - AppStream
ubi-8-baseos-rpms Red Hat Universal Base Image 8 (RPMs) - BaseOS
ubi-8-codeready-builder-rpms Red Hat Universal Base Image 8 (RPMs) - CodeReady Builder
===========================================================
# UBI9
[root@668790fa564c /]# dnf repolist
Updating Subscription Management repositories.
repo id repo name
rhel-9-for-x86_64-appstream-rpms Red Hat Enterprise Linux 9 for x86_64 - AppStream (RPMs)
rhel-9-for-x86_64-baseos-rpms Red Hat Enterprise Linux 9 for x86_64 - BaseOS (RPMs)
ubi-9-appstream-rpms Red Hat Universal Base Image 9 (RPMs) - AppStream
ubi-9-baseos-rpms Red Hat Universal Base Image 9 (RPMs) - BaseOS
ubi-9-codeready-builder Red Hat Universal Base Image 9 (RPMs) - CodeReady BuilderGoes without saying that both were registered with the same activation key.
brianwcook
ben-alkov
chmeliik
brunoapimentel
This is not ready to merge, but at this point I need a review on the approach to know whether to go ay further or not.
I was able to build cachi2 on a UBI image that can be registered with subscription-manager with just one hack - fetching an RPM for createrepo_c from the Rocky repo.
The issue is that the rpm support is using asyncio to download RPMs instead of using dnf. This circumvents the certificate handshake the is normally handled by the dnf subscription-manager plugin. I believe that this image and the prefetch task updates I have would enable authenticated RPM prefetch..._if we can switch to using 'dnf download' for fetching RPMs.
Regarding the hack for createrepo_C, the only alternative I could find is to get it from a repo that requires an entitlement which would make this container non-redistributable. I tried to build a createrepo_c rpm from source but the dependencies aren't available either.
Depends on: https://github.com/containerbuildsystem/cachi2/pull/682
Maintainers will complete the following section
Note: if the contribution is external (not from an organization member), the CI pipeline will not run automatically. After verifying that the CI is safe to run:
/ok-to-test(as is the standard for Pipelines as Code)