Update to Yocto 4.2 - Mickledore

hnez commented 9 months ago

This is the long-awaited update from Langdale (4.1) to Mickledore (4.2)

Things to do before merging¹:

[x] Fix the underlying issue that makes trace-cmd fail to build and fix it instead of disabling the package. 2023-11-01: It looks like this issue fixed itself in the meantime 🎉.
[x] systemd-repart.service fails to start with "Could not determine temporary directory: No such file or directory", which needs to be fixed.
[x] barebox-state does not work. This can be observed by running e.g. rauc status mark-bad other: rauc mark: Failed marking slot rootfs.1 as bad: barebox backend: Failed to run barebox-state: Child process exited with code 1 This makes RAUC unusable, which needs to be fixed.
[x] Test on hardware

Other Pull requests that need coordination:

[ ] #61 can not happen before this is merged, so it needs to be rebased on top of this work and the target branch has to be updated.
[x] #69 backports a systemd service file. This should become obsolete in mickledore, so the backport should be removed again. 2023-11-01: added a commit that removes the file again.

¹ Due to the way meta layers are ususually handled, e.g. having a branch name matching the release name I have created a mickledore branch based on the current langdale branch content to be able to open this PR against it.

hnez commented 8 months ago

It looks like the newer barebox-state looks for the state on one of the eMMC hardware boot partitions:

$ barebox-state --version
dt-utils 2023.08.0
Configured with build-time option '--disable-state-backward-compatibility'.
                                  '--disable-lock-device-node'.
$ strace barebox-state
…
openat(AT_FDCWD, "/dev/mmcblk1boot1", O_RDONLY|O_LARGEFILE) = 4
ioctl(4, MIXER_READ(1) or MEMGETINFO, 0xbefe8ae8) = -1 EINVAL (Invalid argument)
close(4)                                = 0
openat(AT_FDCWD, "/dev/mmcblk1boot1", O_RDONLY|O_LARGEFILE) = 4
openat(AT_FDCWD, "/dev/mmcblk1boot1", O_RDONLY|O_LARGEFILE) = 5
openat(AT_FDCWD, "/dev/mmcblk1boot1", O_RDONLY|O_LARGEFILE) = 6
_llseek(4, 786432, [786432], SEEK_SET)  = 0
read(4, "\271&\3\0\r-\3\0", 8)          = 8
write(2, "state: No meta data header found"..., 33state: No meta data header found
) = 33
_llseek(5, 790528, [790528], SEEK_SET)  = 0
read(5, "\10\20\7\26\34P\0\0", 8)       = 8
write(2, "state: No meta data header found"..., 33state: No meta data header found
) = 33
_llseek(6, 794624, [794624], SEEK_SET)  = 0

While the old barebox-state finds it on the normal eMMC partition:

$ barebox-state --version
dt-utils 2021.03.0
Configured with build-time option '--disable-state-backward-compatibility'.
$ strace barebox-state
openat(AT_FDCWD, "/dev/mmcblk1", O_RDONLY|O_LARGEFILE) = 4
ioctl(4, MIXER_READ(1) or MEMGETINFO, 0xbefcaaa0) = -1 EINVAL (Invalid argument)
close(4)                                = 0
openat(AT_FDCWD, "/dev/mmcblk1", O_RDWR|O_LARGEFILE) = 4
openat(AT_FDCWD, "/dev/mmcblk1", O_RDWR|O_LARGEFILE) = 5
openat(AT_FDCWD, "/dev/mmcblk1", O_RDWR|O_LARGEFILE) = 6
_llseek(4, 786432, [786432], SEEK_SET)  = 0

In my opinion having the state on the normal partition makes more sense, as the active boot partition switches with every RAUC update. I did not yet investigate further @a3f have you seen this before?

a3f commented 8 months ago

In my opinion having the state on the normal partition makes more sense, as the active boot partition switches with every RAUC update.

Agreed.

I did not yet investigate further @a3f have you seen this before?

dt-utils iterates via libudev over all block devices in cdev_from_block_device to find the matching device. Indeed all of boot0, boot1 and the user area have type "disk", so barebox-state gets the wrong device thinking it's the disk device. I am not sure how to best differentiate between boot and user partitions. Maybe check prefer block devices that have no block devices as a parent?

hnez commented 8 months ago

I've worked around the barebox-state issue for now by stepping the meta-rauc submodule back by one commit (the one upgrading to dt-utils 2023.08.0). This is obviously not a permanent solution, but combined with another workaround for systemd-repart I can now at least pass the (not yet public - sorry potential outside readers) LXA TAC labgrid testsuite.

I think I will do some manual testing and then mark this PR as ready for review. Would be glad to take suggestions for more elegant solutions for the aforementioned workarounds.

a3f commented 8 months ago

dt-utils iterates via libudev over all block devices in cdev_from_block_device to find the matching device. Indeed all of boot0, boot1 and the user area have type "disk", so barebox-state gets the wrong device thinking it's the disk device. I am not sure how to best differentiate between boot and user partitions. Maybe check prefer block devices that have no block devices as a parent?

For now, I restored the old behavior, namely taking the first "disk" again instead of the last. Fix is here: https://lore.barebox.org/oss-tools/20231106110405.3209233-1-a.fatoum@pengutronix.de/T/#u. IMO, it's worth a bugfix release, so once tested, we can kick that off.

hnez commented 8 months ago

I've included the barebox-state patch and tested manually and using the LXA TAC testsuite. It looks to be working as intended.

With this I will mark this PR as "Ready for review".

PS: Something weird is going on with GitHubs commit list. It shows commits before cc4d0ea as part of this PR, even though they are already in the "mickledore" branch (which matches current langdale).

hnez commented 8 months ago

Now that #71 is merged we should have everything we want to have in a first stable release (not having it would have been annoying because we would have needed to keep the workaround documentation around somewhere).

This means we should leave langdale behind and concentrate on getting this PR merged. All other open PRs should then be updated to go to the mickledore branch instead.

jluebbe commented 8 months ago

PS: Something weird is going on with GitHubs commit list. It shows commits before cc4d0ea as part of this PR, even though they are already in the "mickledore" branch (which matches current langdale).

I've toggled the base branch to langdale and back to mickledore, which seems to have repaired the list of changes to review.

a3f commented 8 months ago

FYI, dt-utils v2023.11.0 incorporates the fix in https://github.com/linux-automation/meta-lxatac/pull/70/commits/62dfa038848123597e994d524f2154d2a7c921e4, so you should be able to drop the patch on your next update.

hnez commented 8 months ago

FYI, dt-utils v2023.11.0 incorporates the fix in 62dfa03, so you should be able to drop the patch on your next update.

Cool, I've added Pull-Requests in pengutronix/meta-ptx#121 and rauc/meta-rauc#292 to update dt-utils there.

linux-automation / meta-lxatac

Update to Yocto 4.2 - Mickledore #70