systemd / mkosi

💽 Build Bespoke OS Images
https://mkosi.systemd.io/
GNU Lesser General Public License v2.1
1.15k stars 309 forks source link

Can't fit requested partitions into available free space #3014

Open septatrix opened 6 days ago

septatrix commented 6 days ago

mkosi commit the issue has been seen with

e02d40e9ac339d1a62f53393a852c0e1ab549212

Used host distribution

Fedora 40

Used target distribution

Fedora 40

Linux kernel version used

6.10.7-200.fc40.x86_64

CPU architectures issue was seen on

None

Unexpected behaviour you saw

mkosi failed to generate an image due to improper partition sizing

Used mkosi config

### IMAGE: default
# mkosi.conf
[Config]
InitrdInclude=initrd
Include=%C/../mkosi.conf.d/common.conf

[Output]
Format=disk
ImageId=DSPlayerInstaller

[Content]
# TODO clean up, i.e. drop unused command line args, timezone, password, etc.
KernelCommandLine=rd.systemd.unit=installer.target systemd.unit=installer.target pci=noaer
Bootable=yes
Bootloader=uki
Timezone=Europe/Berlin
RootPassword=asdf
BuildSources=%C/../mkosi.output:mkosi.output
InitrdPackages=
    # Tools required by the installer
    util-linux
    bash
    systemd-udev # provides systemd-repart
    dosfstools # required to create EFI partition
Packages=
    # Required to create UKI
    kernel
    systemd-boot

# /home/septatrix/Documents/Simply5/programming/digital-signage-linux/mkosi.conf.d/common.conf
[Distribution]
Distribution=fedora
Release=40
Architecture=x86-64
CacheOnly=yes

[Output]
OutputDirectory=%C/../mkosi.output
BuildDirectory=%C/../mkosi.builddir
CacheDirectory=%C/../mkosi.cache
ManifestFormat=json

[Validation]
Checksum=no

[Host]
RuntimeSize=4G
Incremental=yes
QemuGui=yes
Ephemeral=yes
ToolsTree=default
ToolsTreePackages=
    python3-pip
    pnpm
    # Required to prevent segfault, see https://github.com/nodejs/node/issues/51752
    nodejs-full-i18n
    qemu-ui-*
    qemu-device-display-*

# File: installer/mkosi.repart/00-esp.conf
[Partition]
Type=esp
Format=vfat
CopyFiles=/boot:/
CopyFiles=/efi:/
Minimize=guess

# File: installer/mkosi.repart/10-root.conf
[Partition]
Type=root
Format=squashfs
CopyFiles=/usr/lib/repart.d
CopyFiles=/partitions
Minimize=best

# File: installer/mkosi.build
#!/usr/bin/sh

set -eux

install -d "$DESTDIR"/partitions/
cp -- mkosi.output/DSPlayerOS_"$IMAGE_VERSION"._*.raw "$DESTDIR"/partitions/

mkosi output

‣ Copying repository metadata
‣ Building DSPlayerInstaller image
‣  Copying cached trees
Create snapshot of '/work/home/septatrix/Documents/Simply5/programming/digital-signage-linux/mkosi.cache/fedora~40~x86-64.cache' in '/work/home/septatrix/.cache/mkosi/mkosi-workspace-tu9i2l0e/root'
‣  Running build script /home/septatrix/Documents/Simply5/programming/digital-signage-linux/installer/mkosi.build…
+ install -d /work/dest/partitions/
+ cp -- mkosi.output/DSPlayerOS_2.7.0._835ee95deb40ce2dbf4399a104853eed.usr-x86-64-verity.raw mkosi.output/DSPlayerOS_2.7.0._8f47b35c6579efa9f97dacd98d3250aa.usr-x86-64.raw mkosi.output/DSPlayerOS_2.7.0._b7ec98777a4345e093504f5e8c2a3fac.esp.raw /work/dest/partitions/
‣  Copying in build tree…
‣  Copying in extra file trees…
‣  Generating system users
‣  Generating volatile files
‣  Applying presets…
Removed "/buildroot/etc/systemd/system/systemd-homed.service.wants/systemd-homed-activate.service".
Created symlink /buildroot/etc/systemd/system/systemd-homed.service.wants/systemd-homed-activate.service → /usr/lib/systemd/system/systemd-homed-activate.service.
‣  Applying first boot settings
/buildroot: /etc/localtime written
/buildroot: /etc/passwd written.
/buildroot: /etc/shadow written.
‣  Generating hardware database
‣  Recording packages in manifest…
‣  Generating disk image
Pre-populating vfat filesystem of partition 00-esp.conf twice to calculate minimal partition size
Populating vfat filesystem.
Successfully populated vfat filesystem.
mkfs.fat 4.2 (2021-01-31)
/var/tmp/.#repartc4c6a87c8fcd6000 successfully formatted as vfat (label "ESP", uuid 38f83cf5)
Minimal partition size of vfat filesystem of partition 00-esp.conf is 49.3M
mkfs.fat 4.2 (2021-01-31)
WARNING: Number of clusters for 32 bit FAT is less then suggested minimum.
/var/tmp/.#repartc4c6a87c8fcd6000 successfully formatted as vfat (label "ESP", uuid 94184ffc)
Pre-populating squashfs filesystem of partition 10-root.conf twice to calculate minimal partition size
Populating squashfs filesystem.
Successfully populated squashfs filesystem.
Parallel mksquashfs: Using 16 processors
Creating 4.0 filesystem on /var/tmp/.#repart7ed97ed879e49cee, block size 131072.
[=========================================================================================================================================================================================================/] 17121/17121 100%

Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
        compressed data, compressed metadata, compressed fragments,
        compressed xattrs, compressed ids
        duplicates are removed
Filesystem size 919714.56 Kbytes (898.16 Mbytes)
        41.98% of uncompressed filesystem size (2190661.48 Kbytes)
Inode table size 31081 bytes (30.35 Kbytes)
        44.93% of uncompressed inode table size (69182 bytes)
Directory table size 310 bytes (0.30 Kbytes)
        64.85% of uncompressed directory table size (478 bytes)
Xattr table size 80 bytes (0.08 Kbytes)
        73.39% of uncompressed xattr table size (109 bytes)
Number of duplicate files found 0
Number of inodes 14
Number of files 9
Number of fragments 1
Number of symbolic links 0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 5
Number of hard-links 0
Number of ids (unique uids + gids) 1
Number of uids 1
        root (0)
Number of gids 1
        root (0)
/var/tmp/.#repart7ed97ed879e49cee successfully formatted as squashfs (no label or uuid specified)
Minimal partition size of squashfs filesystem of partition 10-root.conf is 898.1M
Automatically determined minimal disk image size as 948.5M.
Sized '/home/septatrix/.cache/mkosi/mkosi-workspace-tu9i2l0e/staging/DSPlayerInstaller_2.7.0.raw' to 948.5M.
Applying changes to /home/septatrix/.cache/mkosi/mkosi-workspace-tu9i2l0e/staging/DSPlayerInstaller_2.7.0.raw.
Copying in '/var/tmp/.#repart7ed97ed879e49cee' (898.1M) on block level into future partition 1.
Copying in of '/var/tmp/.#repart7ed97ed879e49cee' on block level completed.
Block level copying and synchronization of partition 1 complete in 646.781ms (1.3G/s).
Adding new partition 1 to partition table.
Writing new partition table.
All done.
‣ /usr/lib/firmware/{amd-ucode,intel-ucode} not found, not adding microcode
‣  Building default initrd
‣   Copying cached trees
Create snapshot of '/work/home/septatrix/Documents/Simply5/programming/digital-signage-linux/mkosi.cache/fedora~40~x86-64~default-initrd.cache' in '/work/home/septatrix/.cache/mkosi/mkosi-workspace-k6hy3jve/root'
‣   Copying in extra file trees…
‣   Generating system users
‣   Generating volatile files
‣   Applying presets…
Removed "/buildroot/etc/systemd/system/systemd-homed.service.wants/systemd-homed-activate.service".
Created symlink /buildroot/etc/systemd/system/systemd-homed.service.wants/systemd-homed-activate.service → /usr/lib/systemd/system/systemd-homed-activate.service.
‣   Applying first boot settings
/buildroot: /etc/locale.conf written.
/buildroot: /etc/localtime written
/buildroot: /etc/passwd written.
/buildroot: /etc/shadow written.
‣   Generating hardware database
‣   Removing 1 packages…
Package                                                                     Arch             Version                                                                     Repository                                       Size
Removing:                                                                                                                                                                                                                     
 shadow-utils                                                               x86_64           2:4.15.1-3.fc40                                                             updates                                       4.1 MiB
Removing unused dependencies:                                                                                                                                                                                                 
 libsemanage                                                                x86_64           3.6-3.fc40                                                                  fedora                                      293.5 KiB

Transaction Summary:
 Removing:          2 packages

After this operation 4 MiB will be freed (install 0 B, remove 4 MiB).

Running transaction
[1/3] Prepare transaction                                                                                                                                                             100% | 250.0   B/s |   2.0   B |  00m00s
[2/3] Erasing shadow-utils-2:4.15.1-3.fc40.x86_64                                                                                                                                     100% |  53.6 KiB/s | 439.0   B |  00m00s
[3/3] Erasing libsemanage-0:3.6-3.fc40.x86_64                                                                                                                                         100% | 750.0   B/s |  12.0   B |  00m00s
‣   Removing files…
‣   Creating cpio archive /home/septatrix/.cache/mkosi/mkosi-workspace-k6hy3jve/staging/initrd.cpio…
‣   Compressing /home/septatrix/.cache/mkosi/mkosi-workspace-k6hy3jve/staging/initrd.cpio with zstd
‣   /home/septatrix/.cache/mkosi/mkosi-workspace-tu9i2l0e/initrd.cpio.zst size is 36.6M, consumes 36.6M.
‣  Running modinfo to fetch kernel module dependencies
‣  Calculating required kernel modules and firmware
‣ aead2 is a dependency of cifs but is not installed, ignoring 
‣ sha256 is a dependency of cifs but is not installed, ignoring 
‣ nls is a dependency of cifs but is not installed, ignoring 
‣ aes is a dependency of cifs but is not installed, ignoring 
‣ sha512 is a dependency of cifs but is not installed, ignoring 
‣  Creating cpio archive /home/septatrix/.cache/mkosi/mkosi-workspace-tu9i2l0e/kernel-modules-6.10.8-200.fc40.x86_64.initrd…
‣  Generating unified kernel image for kernel version 6.10.8-200.fc40.x86_64
Wrote unsigned /work/home/septatrix/.cache/mkosi/mkosi-workspace-tu9i2l0e/root/efi/EFI/BOOT/BOOTX64.EFI
‣  /home/septatrix/.cache/mkosi/mkosi-workspace-tu9i2l0e/root/efi/EFI/BOOT/BOOTX64.EFI size is 156.3M, consumes 156.3M.
‣  Formatting ESP/XBOOTLDR partitions
Pre-populating vfat filesystem of partition 00-esp.conf twice to calculate minimal partition size
Populating vfat filesystem.
Successfully populated vfat filesystem.
mkfs.fat 4.2 (2021-01-31)
/var/tmp/.#repart5849f0a323aa6f01 successfully formatted as vfat (label "ESP", uuid ee0844e3)
Minimal partition size of vfat filesystem of partition 00-esp.conf is 284.5M
mkfs.fat 4.2 (2021-01-31)
/var/tmp/.#repart5849f0a323aa6f01 successfully formatted as vfat (label "ESP", uuid 94184ffc)
Automatically determined minimal disk image size as 1.1G, current image size is 948.5M.
File '/home/septatrix/.cache/mkosi/mkosi-workspace-tu9i2l0e/staging/DSPlayerInstaller_2.7.0.raw' grown from 948.5M to 1.1G by truncation.
Resized partition table.
Can't fit requested partitions into available free space (235.1M), refusing.
Automatically determined minimal disk image size as 1.1G, current image size is 1.1G.
‣ "systemd-repart --empty=allow --size=auto --dry-run=no --json=pretty --no-pager --offline=yes --seed 1be2b99d-ddd2-45ef-8933-d2a856fdf6b8 /home/septatrix/.cache/mkosi/mkosi-workspace-tu9i2l0e/staging/DSPlayerInstaller_2.7.0.raw --root=/buildroot --definitions /home/septatrix/Documents/Simply5/programming/digital-signage-linux/installer/mkosi.repart" returned non-zero exit code 1.
DaanDeMeyer commented 5 days ago

@septatrix Minimize= just doesn't work for the ESP and XBOOTLDR partitions because we run systemd-repart twice to populate the partitions. The first time to populate all partitions except the ESP and XBOOTLDR partitions. Afterwards we figure out the roothash if needed, build the UKI and then populate the ESP and XBOOTLDR. However, because the ESP is usually put before the rootfs partition, we need to allocate space for it up front when we run systemd-repart the first time. At this point the ESP does not contain your UKI yet and so any calculations done by using Minimize= will be too small.

I'm sorry but for the ESP and XBOOTLDR partitions you should set the size manually.