Improve absolute symlink handling

[zjackwang]

Getting this error as using the Bootloader=systemd-boot option attempts to install the UKI into /boot/EFI/Linux. Not sure what's going on since the python version is 3.10, and according to https://docs.python.org/3/library/pathlib.html#pathlib.Path.mkdir, the mkdir command in the code should behave like mkdir -p.

‣  (Fixing ownership of package manager cache directory)
Traceback (most recent call last):
  File "/usr/lib/python3.10/pathlib.py", line 1175, in mkdir
    self._accessor.mkdir(self, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/zjackwang_google_com/.cache/mkosi/mkosi-workspacewsv19izg/root/boot/EFI/Linux'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/pathlib.py", line 1175, in mkdir
    self._accessor.mkdir(self, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/home/zjackwang_google_com/.cache/mkosi/mkosi-workspacewsv19izg/root/boot/EFI'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/mkosi/mkosi/run.py", line 55, in uncaught_exception_handler
    yield
  File "/mkosi/mkosi/run.py", line 96, in fork_and_wait
    target(*args, **kwargs)
  File "/mkosi/mkosi/__init__.py", line 4142, in run_build
    build_image(Context(args, config, workspace=workspace, resources=resources))
  File "/mkosi/mkosi/__init__.py", line 3489, in build_image
    install_kernel(context, partitions)
  File "/mkosi/mkosi/__init__.py", line 2270, in install_kernel
    install_uki(context, kver, kimg, token, partitions)
  File "/mkosi/mkosi/__init__.py", line 2186, in install_uki
    boot_binary.parent.mkdir(parents=True, exist_ok=True)
  File "/usr/lib/python3.10/pathlib.py", line 1179, in mkdir
    self.parent.mkdir(parents=True, exist_ok=True)
  File "/usr/lib/python3.10/pathlib.py", line 1179, in mkdir
    self.parent.mkdir(parents=True, exist_ok=True)
  File "/usr/lib/python3.10/pathlib.py", line 1175, in mkdir
    self._accessor.mkdir(self, mode)
FileExistsError: [Errno 17] File exists: '/home/zjackwang_google_com/.cache/mkosi/mkosi-workspacewsv19izg/root/boot'

Here is my full config:

[Distribution]
Distribution=ubuntu
Release=jammy
Repositories=main,universe

[Output]
Format=disk
Output=test-image
OutputDirectory=output

[Content]
Packages=
         # To make bootable
     linux-image-generic
     systemd
     systemd-sysv
     udev
     dbus

     # To install Mellanox
     linux-headers-generic   

     python3
         nfs-kernel-server
         openssh-server
         apt-transport-https
         ca-certificates
         gnupg
         unzip
         sbsigntool
         cryptsetup
         sudo
         net-tools
         netplan.io
         parted
         cron
         curl
         efitools
         fio
         vim
         efibootmgr
         udev
         lshw
         gcc
         make
         dkms
         iputils-ping
         sshpass
         iperf
         debsecan
         avahi-daemon
         avahi-dnsconfd
         libnss-mdns
         software-properties-common
         iptables
         tree
         less
         lsb-release
         lsof
         poppler-utils
         imagemagick
         gdisk
         nmap
         telnet
         netcat
         sysstat
         traceroute
         mtr
         iotop
         iputils-ping
         tcpdump
         ethtool
         hdparm
         sg3-utils
         nvme-cli
         libc6-dev
         libpq5
         rsync
         ipmitool
         samba
         logrotate
         smbclient
         rsyslog
         rhash
         rclone
         dnsutils
         fuse
         gocryptfs
         cifs-utils
         socat
     wget

WithDocs=true
WithNetwork=true
Bootable=true
Bootloader=systemd-boot

[Validation]
SecureBoot=true
SecureBootKey=/keys/binSig.key
SecureBootCertificate=/certs/binSig.crt

[Host]
ToolsTree=default

The build succeeds if I use Bootloader=uki as that installs the UKI to /efi/EFI/BOOT, which was successfully created.

[DaanDeMeyer]

Can you add a gist or so with the full logs when running with --debug?

[DaanDeMeyer]

Do you have any mkosi scripts that are used with your configuration? I cannot reproduce this error locally with the information you've provided. I'm assuming that /boot is not a directory in which case mkdir() is documented to fail with FileExistsError.

You can verify this by adding run(["bash"], stdin=sys.stdin, cwd=context.root) just before boot_binary.parent.mkdir(parents=True, exist_ok=True) is called and checking the boot/ path

[zjackwang]

So I added run(["bash"], stdin=sys.stdin, cwd=context.root) and checked out /boot.

It is a symlink: /boot -> /srv/boot. I verified in our postinstall script that we create this symlink since /srv will be mutable while / is immutable. During build time, this symlink is valid.

However, at install_uki, this symlink is broken. The symlink is now pointing at the host filesystem root srv path, which doesn't exist. Is this because the env is not chroot'd? I encounter the same problem when mounting a built image onto a random path. When I chroot <build_root>, the symlinks become whole again. Is there a reason to not do this in install_uki?

[DaanDeMeyer]

@zjackwang Please realize that I lost a lot of time investigating this because you omitted this information. Next time you should add crucial information like this in your original report.

To fix this, don't use absolute symlinks, use relative symlinks.

[DaanDeMeyer]

So we should probably add a chase() helper function that we use to resolve paths whenever we operate on the image root directory. This helper would open an fd to "/", then chroot and chdir into the image root directory, resolve the symlink and finally chroot and chdir back to "/". That way absolute symlinks would be resolved correctly.

[zjackwang]

@zjackwang Please realize that I lost a lot of time investigating this because you omitted this information. Next time you should add crucial information like this in your original report.

To fix this, don't use absolute symlinks, use relative symlinks.

Apologies for the omitted information. Confirmed relative symlinks allowed the build to succeed.