Tracking upstream projects that do not support hermetic-usr for configuration

bluca commented 1 year ago

This issue will be used to track Linux projects that do not currently support hermetic-usr configuration style (ie: /usr/lib/foo as default, /run/foo as ephemeral local override, /etc/foo as persistent local override). The purpose is to have a cross-distribution list of items to slowly work through, to be able to have a bootable and working minimal Linux image-based system with only /usr.

This list is not definitive and will get updated as we go.

[ ] glibc:
- [ ] /etc/nsswitch.conf (https://bugzilla.suse.com/show_bug.cgi?id=1215487)
- [ ] /etc/host.conf
- [ ] /etc/rpc
- [ ] nscd (/etc/nscd.conf)
- [ ] ldconfig (/etc/ld.so.conf, /etc/ld.so.conf.d, but no /usr/...)
- [ ] /etc/shells (ref: https://github.com/linux-pam/linux-pam/issues/498#issuecomment-1288457808)
- [ ] /etc/services (used by getservbyname/getservbyport, which make little sense and are not widely used, but there still are some applications which do). It should be moved under /usr.
[ ] update-alternatives (from dpkg) uses /etc/alternatives. There is https://github.com/openSUSE/libalternatives but this only works for binaries, not e.g. manual pages
[ ] SELinux
- [ ] policycoreutils/sestatus (/etc/sestatus.conf)
- [ ] selinux-policy installs the policy in /etc/selinux/ and /var/lib/selinux/. Especially the last is a problem, as this mixes user changes with policy defaults. /var/lib/selinux needs to be split: the package defaults belongs somewhere to /usr, which should be merged with the admin changes (should be stored in /etc/, not /var) at runtime.
[ ] apparmor
[ ] sudo has /etc/sudo.conf, /etc/sudoers and /etc/sudo_logsrvd.conf
[ ] audit (/etc/audit/auditd.conf, /etc/libaudit.conf)
[ ] rsync (/etc/rsyncd.conf)
[ ] chrony (/etc/chrony.conf)
[ ] podman and releated tools/libraries (/etc/containers)
[ ] apt
[ ] dpkg
[x] dnf5 which requires /etc/dnf/dnf.conf
[ ] systemd "empty" main config files in /etc/systemd
[ ] udev "empty" main config files in /etc/udev
[ ] Xorg: /etc/X11/xinit/xinitrc.d/50-systemd-user.sh
[ ] mailx (/etc/mail.rc)
[ ] util-linux (/etc/blkid.conf)
[ ] fcoe-utils (/etc/fcoe/)
[ ] open-iscsi (/etc/iscsi/iscsid.conf)
[ ] krb5 (/etc/krb5.conf)
[ ] libnl (/etc/libnl/)
[ ] libssh (/etc/libssh/)
[ ] e2fsprogs (/etc/mke2fs.conf)
[ ] nftables (/etc/nftables/)
[ ] smartmontools (/etc/smartd.conf, /etc/smartd_warning.sh)
[ ] wpa_supplicant (/etc/wpa_supplicant/wpa_supplicant.conf)
[ ] attr (/etc/xattr.conf)
[ ] zypper (/etc/zypp/zypp.conf, /etc/zypp/zypper.conf)

thkukuk commented 1 year ago

A first bunch, starting with a minimal OS running containers.

update-alternatives (from dpkg) uses /etc/alternatives. There is https://github.com/openSUSE/libalternatives but this only works for binaries, not e.g. manual pages
selinux-policy installs the policy in /etc/selinux/ and /var/lib/selinux/. Especially the last is a problem, as this mixes user changes with policy defaults. /var/lib/selinux needs to be splitted: the package defaults belongs somewhere to /usr, which should be merged with the admin changes (should be stored in /etc/, not /var) at runtime.
sudo has /etc/sudo.conf, /etc/sudoers and /etc/sudo_logsrvd.conf
audit (/etc/audit/auditd.conf, /etc/libaudit.conf)
rsync (/etc/rsyncd.conf)
chrony (/etc/chrony.conf)
podman and releated tools/libraries (/etc/containers)

julian-klode commented 1 year ago

APT and dpkg both don't support it. I haven't figured out the behavior for APT yet, tbh, it's weird because essentially everything is drop-ins for apt.conf these days.

thkukuk commented 1 year ago

glibc for /etc/services (used by getservbyname/getservbyport, which make little sense and are not widely used, but there still are some applications which do). It should be moved under /usr.

getservbyname/getservbyport are using NSS as backend. There are NSS modules for nearly every location, not only /etc/services, NIS or LDAP, but also for /usr. There is e.g. https://github.com/openSUSE/libnss_usrfiles, which, with the /etc/nsswitch.conf option services: files usrfiles looks at first for entries in /etc/services and afterwards in /usr/etc/services. More such modules exists, so in my opinion, this task is solved, it's just in the responsibility of the Linux distributior to package the best fitting variant and configure it. openSUSE Tumbleweed/MicroOS are doing this since many years. So this is solved for aliases, ethers, protocols, rpc and services. /etc/shells is not handled by NSS, so solving that will be more complicated.

fbuihuu commented 1 year ago

I dare to add to the list :

[ ] systemd ships all its main config files in /etc/systemd or /etc/udev.

There's currently https://github.com/systemd/systemd/pull/28919 to address this issue by giving the possibility to downstream to ship the config files in /usr/lib but for some reasons the idea has not been well received.

fbuihuu commented 1 year ago

[ ] udev (/etc/udev/udev.conf)

poettering commented 1 year ago

@fbuihuu systemd's config files in /etc/ are just decoration. Entirely redundant, they are pretty much just helpful hints to people who want to sue traditional populated /etc/. If you delete them for modern systems that come up without /etc/ then behaviour is not changed whatsoever.

All options listed in them just give users hints on the available settings and their defaults, and those options are fully commented, hence these files are NOPs.

thkukuk commented 11 months ago

Next package list:

glibc:
- /etc/nsswitch.conf (https://bugzilla.suse.com/show_bug.cgi?id=1215487)
- /etc/host.conf
- /etc/rpc
- ldconfig (/etc/ld.so.conf, /etc/ld.so.conf.d, but no /usr/...)
mailx (/etc/mail.rc)
util-linux (/etc/blkid.conf)
fcoe-utils (/etc/fcoe/)
SELinux
- policycoreutils/sestatus (/etc/sestatus.conf)
open-iscsi (/etc/iscsi/iscsid.conf)
krb5 (/etc/krb5.conf)
libnl (/etc/libnl/)
libssh (/etc/libssh/)
e2fsprogs (/etc/mke2fs.conf)
nftables (/etc/nftables/)
smartmontools (/etc/smartd.conf, /etc/smartd_warning.sh)
wpa_supplicant (/etc/wpa_supplicant/wpa_supplicant.conf)
attr (/etc/xattr.conf)
zypper (/etc/zypp/zypp.conf, /etc/zypp/zypper.conf)

thkukuk commented 11 months ago

For glibc I missed yesterday:

nscd (/etc/nscd.conf)

This are now all packages from our minimal installation of MicroOS as ContainerHost beside openssh. I spoke already with the openssh developers about this some time ago, they rejected all ideas to modify configuration file handling. So we (openSUSE) reconfigured the configuration file setup to mimic hermetic-usr as far as possible, open issue is AcceptEnv, which you cannot overwrite.

Question is how far do we want to go with that list? I could go as next through a typical server installation and desktop system, but I don't know if this would be really helpful, as the list is already long and it would mix up packages with different priority.

bluca commented 11 months ago

This are now all packages from our minimal installation of MicroOS as ContainerHost beside openssh. I spoke already with the openssh developers about this some time ago, they rejected all ideas to modify configuration file handling. So we (openSUSE) reconfigured the configuration file setup to mimic hermetic-usr as far as possible, open issue is AcceptEnv, which you cannot overwrite.

Yeah I had the same experience some years past, openssh as a project is just not interested in anything that doesn't directly benefit BSD. Should we set up a Linux-focused fork, where we can co-maintain patches?

Question is how far do we want to go with that list? I could go as next through a typical server installation and desktop system, but I don't know if this would be really helpful, as the list is already long and it would mix up packages with different priority.

I think this is a good starting point, being core packages they are the highest value to fix. Once we have made a dent in the current list, then we could start looking at a server installation.

keszybz commented 11 months ago

nscd (/etc/nscd.conf)

nscd is on its way out: https://fedoraproject.org/wiki/Changes/RemoveNSCD. IIRC, the intent was to also propose the same from glibc upstream if the experiment in Fedora is successful.