`/sysroot` dir and subfiles are `unlabeled_t` since version 40.20240504.3.0

HuijingHei commented 1 month ago

Describe the bug

/sysroot dir and subfiles are unlabeled_t since 40.20240504.3.0.

Bisect results:

40.20240416.3.1 is good 40.20240504.3.0 is bad

Reproduction steps

Start FCOS and run ls -alZ /sysroot

Expected behavior

/sysroot dir and subfiles are correctly labeled.

Actual behavior

/sysroot dir and subfiles are unlabeled_t.

[root@cosa-devsh ~]# ls -alZ /sysroot
total 8
drwxr-xr-x.  4 root root system_u:object_r:unlabeled_t:s0   93 Aug  1  2022 .
drwxr-xr-x. 12 root root system_u:object_r:root_t:s0      4096 Jul 29 18:54 ..
-rw-r--r--.  1 root root system_u:object_r:unlabeled_t:s0  205 Aug  1  2022 .aleph-version.json
lrwxrwxrwx.  1 root root system_u:object_r:unlabeled_t:s0   19 Aug  1  2022 .coreos-aleph-version.json -> .aleph-version.json
drwxr-xr-x.  2 root root system_u:object_r:unlabeled_t:s0    6 Jul 29 18:54 boot
drwxr-xr-x.  5 root root system_u:object_r:unlabeled_t:s0   62 Aug  1 02:32 ostree

[root@cosa-devsh ~]# ls -alZ /sysroot/ostree
total 0
drwxr-xr-x. 5 root root system_u:object_r:unlabeled_t:s0  62 Aug  1 02:32 .
drwxr-xr-x. 4 root root system_u:object_r:unlabeled_t:s0  93 Aug  1  2022 ..
lrwxrwxrwx. 1 root root system_u:object_r:unlabeled_t:s0   8 Aug  1  2022 boot.1 -> boot.1.1
drwxr-xr-x. 3 root root system_u:object_r:unlabeled_t:s0  27 Aug  1  2022 boot.1.1
drwxr-xr-x. 3 root root system_u:object_r:unlabeled_t:s0  27 Aug  1  2022 deploy
drwxr-xr-x. 7 root root system_u:object_r:unlabeled_t:s0 102 Aug  1  2022 repo

System details

N/A

Butane or Ignition config

No response

Additional information

No response

jlebon commented 2 weeks ago

When fixing https://github.com/coreos/fedora-coreos-tracker/issues/1771, we should also fix this in the same barrier code.

jlebon commented 6 days ago

OK yeah, this is a mess. Basically everything in /sysroot that's not the OSTree deployment checkout or the var stateroot or the file objects themselves are unlabeled. And... actually even the dirtree/dirmeta objects are unlabeled. Those don't affect the deployment checkouts, but it's still ugly.

root@cosa-devsh:/sysroot# find /sysroot -context '*:unlabeled_t:*'
/sysroot
/sysroot/boot
/sysroot/.aleph-version.json
/sysroot/.coreos-aleph-version.json
/sysroot/ostree
/sysroot/ostree/deploy
/sysroot/ostree/deploy/fedora-coreos
/sysroot/ostree/deploy/fedora-coreos/deploy
/sysroot/ostree/deploy/fedora-coreos/deploy/462cc2876802b1d9c8565a1d9187b05c76cb14ae3ea12898f9a321c50a3cbca5.0.origin
/sysroot/ostree/deploy/fedora-coreos/backing
/sysroot/ostree/deploy/fedora-coreos/backing/462cc2876802b1d9c8565a1d9187b05c76cb14ae3ea12898f9a321c50a3cbca5.0
/sysroot/ostree/deploy/fedora-coreos/backing/462cc2876802b1d9c8565a1d9187b05c76cb14ae3ea12898f9a321c50a3cbca5.0/root-transient
/sysroot/ostree/deploy/fedora-coreos/backing/462cc2876802b1d9c8565a1d9187b05c76cb14ae3ea12898f9a321c50a3cbca5.0/root-transient/work
/sysroot/ostree/deploy/fedora-coreos/backing/462cc2876802b1d9c8565a1d9187b05c76cb14ae3ea12898f9a321c50a3cbca5.0/root-transient/upper
/sysroot/ostree/boot.1
/sysroot/ostree/repo
/sysroot/ostree/repo/refs
/sysroot/ostree/repo/refs/heads
/sysroot/ostree/repo/refs/heads/ostree
/sysroot/ostree/repo/refs/heads/ostree/1
/sysroot/ostree/repo/refs/heads/ostree/1/1
/sysroot/ostree/repo/refs/heads/ostree/1/1/0
/sysroot/ostree/repo/refs/mirrors
/sysroot/ostree/repo/refs/remotes
/sysroot/ostree/repo/refs/remotes/fedora
/sysroot/ostree/repo/refs/remotes/fedora/fedora
/sysroot/ostree/repo/refs/remotes/fedora/fedora/x86_64
/sysroot/ostree/repo/refs/remotes/fedora/fedora/x86_64/coreos
/sysroot/ostree/repo/refs/remotes/fedora/fedora/x86_64/coreos/testing-devel
/sysroot/ostree/repo/objects
/sysroot/ostree/repo/objects/f3
/sysroot/ostree/repo/objects/f3/3c59694b64449073f6ee0f4e8b0a0ffd9c5e4666ffef9f2afb9a0c39511541.dirtree
...
/sysroot/ostree/repo/tmp
/sysroot/ostree/repo/tmp/cache
/sysroot/ostree/repo/extensions
/sysroot/ostree/repo/config
/sysroot/ostree/repo/state
/sysroot/ostree/repo/.lock
/sysroot/ostree/boot.1.1
/sysroot/ostree/boot.1.1/fedora-coreos
/sysroot/ostree/boot.1.1/fedora-coreos/f737c3f7695016455274f7b964c037c8ecbd3209e28a197476ab404785ef00c0
/sysroot/ostree/boot.1.1/fedora-coreos/f737c3f7695016455274f7b964c037c8ecbd3209e28a197476ab404785ef00c0/0

In the create_disk.sh path, all these used to have root_t, inherited from /sysroot being root_t: https://github.com/coreos/coreos-assembler/blob/472c2cf6c1f952dc337cad1aa0238aa063ffaa76/src/create_disk.sh#L296.

Some of these entries will cycle out over time. E.g. some of the dirmeta/dirtree objects, the directories with digests in them, etc... Others will linger.

A comprehensive fix for this is now trickier and riskier than I thought. We could do something like the find command above but we need to filter out:

entries below /sysroot/ostree/deploy/*/deploy
entries at and below /sysroot/ostree/deploy/*/var
all of /sysroot/ostree/repo/objects; do this in a separate invocation instead where we only target directories and .dirmeta/.dirtree files

This will need to be carefully written and tested. We should run ostree fsck at the end.

dustymabe commented 6 days ago

@jlebon - did you mean to close this?

jlebon commented 6 days ago

Whoops no! Sorry, GitHub project issue.

travier commented 2 days ago

For /sysroot in https://github.com/coreos/fedora-coreos-config/pull/3150, let's start with doing the bare minimum to get us back to a reasonable state in F41, and we'll do the risky bits later.

Let's pick a static list of files that we know are safe to fix.

jbtrystram commented 2 days ago

Experimenting a bit with a good and a bad build on rawhide, following jonathan's comment guidelines I find 90 files that are unlabeled_t instead of root_t.

I got a list of files mounting the FCOS rootfs on a loop device then sudo find /mnt/ | sudo xargs ls -dZ.

grep -v /mnt/ostree/deploy/fedora-coreos/deploy/ -> excludes files below /sysroot/ostree/deploy/*/deploy grep -v /mnt/ostree/deploy/fedora-coreos/var/ -> exclude entries below below /sysroot/ostree/deploy/*/var grep -v /ostree/repo/objects exclude all the ostree repo objects. (see below for dirmeta and dirtree files)

The remaining files are as follow

/mnt/
/mnt/.aleph-version.json
/mnt/boot
/mnt/.coreos-aleph-version.json
/mnt/ostree
/mnt/ostree/boot.1
/mnt/ostree/repo
/mnt/ostree/repo/config
/mnt/ostree/repo/extensions
/mnt/ostree/repo/.lock
/mnt/ostree/repo/refs
/mnt/ostree/repo/refs/heads
/mnt/ostree/repo/refs/heads/ostree
/mnt/ostree/repo/refs/heads/ostree/1
/mnt/ostree/repo/refs/heads/ostree/1/1
/mnt/ostree/repo/refs/heads/ostree/1/1/0
/mnt/ostree/repo/refs/heads/ostree/container
/mnt/ostree/repo/refs/heads/ostree/container/blob
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_065abb000ccdd0aa83e91005902070dc6f9736ec8c8ba450a354f7adaa1746d4
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_072ff097c099583008eff1bcc46de26e30f8dd89b93bcffd5bfc2f6a6e62d75c
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_07508ea206d7b0596ec2ada8534b9733212ab4ebc4f9656fa27e8835ec0960ec
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_0fa8134e6b4a2f376b5dbbf7a136e798a25a0b97480530302d0babbce23dd454
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_13a5166d2bc3ba1ed0fecc554e4a9030ff89c5166452ef6cf00e3482c4956be3
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_16d739e08e6c18e63be9f2955d29f9636d67ee383f27c1b54f7493381caf94eb
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_17ba8bce96d0e7b4cfd4c13491f9869a9afb44eeda776b195feb8239d69e0830
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_1b03dd3d829407fef8597bdb4386d7f19d744703bce3bf8396582648bd2c4ad3
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_1b84992b5c479d5cae082dba1c86b9908e4eafec398de806e595d08dcccd1649
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_1ba3c4f93897798199b9ea264650aac8e7e33aa1691b75110f62e90e3fccc6b3
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_1d4a5326b916cfd5872537aa85ac6454f3a13759b5bb16ba2203e706ddff5d1c
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_289e3ffc114dd98a9d44321090a6ac8255a88d587e44c4e7457eaa0af3953fb9
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_28f67cb15b8220cfa744b4c084d9935b7e7d82865fdada6748b1cbdf31769210
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_396b679c7f3e46db32d65d8bf851eeb1c6eb854ab8add3086be915d3e18cad98
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_3adbf1666fd744cc754f03dc7e32d2144d25ddb3686e113945b467c4e7765e95
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_419ed6c96a4a57955051946bd40514763f045a0531c2ee2b12c40815cd48fbc2
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_434d156e235c10cdfad2e936e0d38b3d9108b1b394fda9b2a0b8113a3d2a924f
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_43bbff2f194201cc02299f27a3eaffe72358fb3aec7d3fb640fc093759075af4
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_49378a090f78527d80c69daa3a880aff8ad67c3739fcda6712a02aa51b015b7e
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_4dba63e9932aa706d03d1d3255d6898770be2a207384c537ba8a0cde02792889
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_4efbd150e260224a511873778fcf260107cc124b1b73b0fe0891e3df939b7570
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_58527239320776b40a3159fe23384f62cba98c1fee5bb7b64710d2afc698e7fc
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_59ba734f01b8e5dcb78d720ce99f46eb34b50400332bf50da3fd12a43f4dcd78
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_5aa6e6c90d4394bd022b4eb75e419e27e98e54e3b9d54777229a0735c2c61005
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_5f9d3dcf5281c5f6512471366be68bee46c2485eddf4fd1887da6b240712be5f
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_6a7d4cfe666cc22739fcee759442b411ee3b7e7a21478278f8ee0304f9df96ab
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_743ba51e9721e121a4303c4f19ed1e8fed7d42345278359fab63c3b333775468
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_75dbde1744f160d6664b1fd0cf6b9e72f1e691bb14b91ac5fceeb362e529b0b8
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_7d529836dd7ccf79aa1968d5823c4649d2e36a2644d43e654f4884500ddd5ec3
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_846d0973a8726bfcf505c0cfc568f31805f76cf9628dff45e40357a989674bf7
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_84e960f2b7cde81c03d5c4b2c294d75237034b688b389553d3f8da48d8f845ff
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_8a85f85946b32c11ec982b427ba83169f8ce1f34fc09e3d2d1bbb2956ad9a993
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_8c5be97dea11a0652f07c8d0d86134120d26ffb04d6bc2901a01a627cb14513a
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_942c4e62004c28d73375133ac4b2e89a7094457a1ff93d4d651fb1f295159512
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_94c3c4de11179cade3a96efbc315a5c169c3804459ec7b3f49b7fe19087cbb81
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_9577c43de795f7f22e0c633e234a7a34657dd370d4a682a6a22d7c47ffb9fd37
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_9efd448a28346dc35f32d4a62da171679081728f05c77052835e834bb7f80faf
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_a94a418838ece60c010e69d1158f0b1cb2e1f5bec715a3fa0d42308c06f2f287
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_aa1b9f3a6c6c65b018ec83f7567083485d575d98df420ce672b7912d9f0f25ba
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_aa507848048188d4c00f5c045292a4c4e73e688e1f226f9927d4d67bfb6bce83
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_aa6048ac939ed0e0c8e938760da5c2f0aa2251bb295be28c2dc14762fb9318f8
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_b0690f8744b7a02f69e499a427d92dff3b36da0964fe842336713f1a54d2afb2
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_b6ac248b5ca8f770f20f59ebf2db1842ca75480d1666fe88ce831af56490173f
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_c2f0b7ba1e7e8c87301c5883334aa9a4dbb158a68c639aa2736f2a584f23bcf7
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_c37e72a6b6d151fc0086bccd96bfb69b67e274241bf14c645713e483ef258837
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_cae28fcdd8477cbdc747de6735046f8561bf8975c452d3044ca6b2428b8fb9ab
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_cc0dcfbbe37670a96c0a776d58bd87edf34376e3d3813e91c3b9f13a090c7da1
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_d6f411a4aa522d14a0ec6bb7d5b06f790fe4ffbbdcfb2bf93c62b9935cc18f5a
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_e41fb37f5155d4a8f823a02c3c4bc4287a00a47d44b79c30aa0c717483adf9c8
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_f0beed5dd68387195a64429142c445c22ae31076261432943141fe53bbb030a1
/mnt/ostree/repo/refs/heads/ostree/container/blob/sha256_3A_fd738284d581053774208bf6f19ae04aac6995f203fd15f30367c9af8502a025
/mnt/ostree/repo/refs/heads/ostree/container/image
/mnt/ostree/repo/refs/heads/ostree/container/image/docker_3A__2F__2F_quay_2E_io
/mnt/ostree/repo/refs/heads/ostree/container/image/docker_3A__2F__2F_quay_2E_io/fedora
/mnt/ostree/repo/refs/heads/ostree/container/image/docker_3A__2F__2F_quay_2E_io/fedora/fedora-coreos_3A_rawhide
/mnt/ostree/repo/refs/mirrors
/mnt/ostree/repo/refs/remotes
/mnt/ostree/repo/state
/mnt/ostree/repo/tmp
/mnt/ostree/repo/tmp/cache
/mnt/ostree/deploy
/mnt/ostree/deploy/fedora-coreos
/mnt/ostree/deploy/fedora-coreos/backing
/mnt/ostree/deploy/fedora-coreos/backing/aa2f3fc39ebf4ba64ef384dbc83ae74f87e69b0da173371a47c3eab202dc0d33.0
/mnt/ostree/deploy/fedora-coreos/backing/aa2f3fc39ebf4ba64ef384dbc83ae74f87e69b0da173371a47c3eab202dc0d33.0/root-transient
/mnt/ostree/deploy/fedora-coreos/backing/aa2f3fc39ebf4ba64ef384dbc83ae74f87e69b0da173371a47c3eab202dc0d33.0/root-transient/upper
/mnt/ostree/deploy/fedora-coreos/backing/aa2f3fc39ebf4ba64ef384dbc83ae74f87e69b0da173371a47c3eab202dc0d33.0/root-transient/work
/mnt/ostree/deploy/fedora-coreos/deploy
/mnt/ostree/boot.1.1
/mnt/ostree/boot.1.1/fedora-coreos
/mnt/ostree/boot.1.1/fedora-coreos/d8db71772a2d385c6c7222637856968a45a371bb8a9622eb4cc19074bd1778c0
/mnt/ostree/boot.1.1/fedora-coreos/d8db71772a2d385c6c7222637856968a45a371bb8a9622eb4cc19074bd1778c0/0

Then all the dirtree and dirmeta files can be trageted with : grep /ostree/repo/objects | grep .dirmeta and grep /ostree/repo/objects | grep .dirtree

I am going to update the PR with a proposed script and do some testing.

cgwalters commented 2 days ago

Just a reminder that bootc install does all this correctly nowadays. One avenue is to investigate using it.

dustymabe commented 2 days ago

Just a reminder that bootc install does all this correctly nowadays. One avenue is to investigate using it.

You've made us aware. This discussion is about how to fix existing systems.

coreos / fedora-coreos-tracker