search

coreos / rpm-ostree

⚛📦 Hybrid image/package system with atomic upgrades and package layering
https://coreos.github.io/rpm-ostree
Other
858 stars 195 forks source link

rpm-ostree and %post scripts #1884

Open fencekicker opened 5 years ago

fencekicker commented 5 years ago

I'm working on an OS image that needs to work on 2 (let's say embedded) hardware platforms; the OS image uses CentOS atomic host and is built using rpm-ostree compose, basically. I'm trying to package the QAT driver (see https://01.org/intel-quickassist-technology) for our OS, and encountered some issues. I wanted to enquire which of the limitations I'm seeing are inherent to rpm-ostree and which are due to the ancient version we use (we have rpm-ostree 2016.4) ; also, known workarounds would be appreciated.

The QAT driver build system is quirky and not made for cross compiling, so I had to do a lot of trial and error to get it right. I was basically building the RPM, then installing it on top of our image after doing a rpm-ostree usroverlay. I want to handle some things in %post scripts - I need to install some firmware files that are owned by the linux-firmware RPM, and also put the configuration for the right QAT chip in /etc based on the platform I'm on.

Maybe you can see where this is going. I had 2 issues here when going from manually installed RPM to rpm-ostree compose flow:

So far I've gone with moving the %post logic in its own script, where I check: hey, am I running on real hardware? Thus, when installing the RPM manually during development, all the logic is in place; when installing the RPM during build time, the script isn't run at compose time; at the moment I'm running the standalone script from the treecompose script that we use - as far as I can tell, at that point changes made to files seem to persist (so solves issue 1 above). I'm still not sure how to tackle issue 2, I'll probably have the standalone script create configuration files for both supported platforms and be done with it.

I'm sorry if this is covered somewhere I've missed. I found an issue about %post scripts (#328), but I couldn't understand if it pertains to what I'm seeing, probably because my knowledge of rpm-ostree internals is limited. In general, I think it's useful for people building RPMs to know what is different when also targeting systems based on rpm-ostree or project atomic, any such resource would be welcome.

cgwalters commented 5 years ago

the %post scripts are run during build time instead of run time when doing rpm-ostree compose. Is there a way to actually have these scripts run after the image is installed on the real hardware?

No; you need to convert anything like that to run as a systemd unit.

cgwalters commented 5 years ago

Note also that as of today, rpm-ostree for RHEL7/CentOS7 is in maintenance mode. We may fix critical bugs, and don't hesistate to ask questions - but the focus of the team is on modern Fedora and RHEL8 (Fedora CoreOS/RHEL CoreOS). See also https://github.com/coreos/coreos-assembler

cgwalters commented 5 years ago

I'm trying to package the QAT driver (see https://01.org/intel-quickassist-technology) for our OS

I'm confused as to what that actually is...some of it seems to be talking about the crypto extensions that are part of the processor, why does that need a driver? Hm, OK I see https://01.org/sites/default/files/downloads//336212-006qatswgettingstarted.pdf which has a kernel driver that does...something.

I think what you want is to get an RPM package of the .ko file and just add it to your manifest. That model may require you to rebuild the kernel module as the CentOS upstream kernel changes though; see also https://01.org/sites/default/files/downloads//336212-006qatswgettingstarted.pdf

fencekicker commented 5 years ago

the %post scripts are run during build time instead of run time when doing rpm-ostree compose. Is there a way to actually have these scripts run after the image is installed on the real hardware?

No; you need to convert anything like that to run as a systemd unit.

OK, so the paradigm is to run post script if running natively, else schedule it as a systemd unit? I would expect this to be the model for Fedora RPMs that need to run in both contexts. Also, is the failure of the %post script to alter the filesystem (I was trying to overwrite firmware files owned by another package) expected?

fencekicker commented 5 years ago

I'm trying to package the QAT driver (see https://01.org/intel-quickassist-technology) for our OS

I'm confused as to what that actually is...some of it seems to be talking about the crypto extensions that are part of the processor, why does that need a driver? Hm, OK I see https://01.org/sites/default/files/downloads//336212-006qatswgettingstarted.pdf which has a kernel driver that does...something.

I think what you want is to get an RPM package of the .ko file and just add it to your manifest. That model may require you to rebuild the kernel module as the CentOS upstream kernel changes though; see also https://01.org/sites/default/files/downloads//336212-006qatswgettingstarted.pdf

In order to use the QAT chip for crypto offloading, you need the QAT driver. It's not just a bunch of kernel modules: there are some firmware files that we need in /lib/firmware (clashing with linux-firmware, yay), configuration files for the device in /etc, a startup script in /etc/init.d. The configuration file name depends on the chip (e.g. /etc/dh895xcc_devX.conf on a system with a dh8950 chip), so I have a separate folder in /etc/ where I store the config files and was planning to symlink the proper configuration files in /etc based on the hardware.

fencekicker commented 5 years ago

Note also that as of today, rpm-ostree for RHEL7/CentOS7 is in maintenance mode. We may fix critical bugs, and don't hesistate to ask questions - but the focus of the team is on modern Fedora and RHEL8 (Fedora CoreOS/RHEL CoreOS). See also https://github.com/coreos/coreos-assembler

I will definitely look at that; I had seen the announcement, but haven't read further yet. So CoreOS is the new project atomic? Is there going to be a CentOS CoreOS (hm, the name sounds... redundant :) ) when CentOS 8 gets out?

fencekicker commented 5 years ago

Sorry, I seem to have closed this by mistake, I didn't mean to.