Partitioning fail with UEFI image

Jellyfrog commented 4 years ago

Problem description

Trying to add support for UEFI to a currently working image, by adding firmware="uefi" and <package name="grub2-x86_64-efi" arch="x86_64"/>.
After writing the image to disk, the following happen:

Error: You requested a partition from 0cyl to 14924cyl (sectors 2048..239754060).
The closest location we can manage is 0cyl to 0cyl (sectors 2048..6143).

Full output; https://paste.sh/jsaMmwsX#hK8hAMBIR-5dkBhAe5iqzxDE

Relevant kiwi config:

  <profiles>
    <profile name="default" description="x86 default for oem/vmx" import="true" />
  </profiles>

  <preferences>
    <type
      image="oem"
      initrd_system="dracut"
      kernelcmdline="splash quiet net.ifnames=1"
      firmware="uefi"

      filesystem="btrfs"
      btrfs_root_is_snapshot="true"
      fsmountoptions="acl"

      boot="oemboot/suse-SLES12"
      bootloader="grub2"
      boottimeout="5"
      bootprofile="default"
      bootpartition="false"

      installpxe="true"
      installiso="false"
      >
      <oemconfig>
        <oem-silent-boot>true</oem-silent-boot>
        <oem-boot-title>redacted 5.0.0</oem-boot-title>
        <oem-unattended>true</oem-unattended>
        <oem-swap>true</oem-swap>
        <oem-swapsize>4096</oem-swapsize>
      </oemconfig>
      <systemdisk>
        <volume name="home"/>
        <volume name="tmp"/>
        <volume name="opt"/>
        <volume name="srv"/>
        <volume name="boot/grub2/i386-pc"/>
        <volume name="boot/grub2/x86_64-efi" mountpoint="boot/grub2/x86_64-efi"/>
        <volume name="var/cache"/>
        <volume name="var/crash"/>
        <volume name="var/lib/machines"/>
        <volume name="var/lib/mailman"/>
        <volume name="var/lib/named"/>
        <volume name="var/lib/mysql" copy_on_write="false"/>
        <volume name="var/lib/mariadb" copy_on_write="false"/>
        <volume name="var/lib/pgsql" copy_on_write="false"/>
        <volume name="var/lib/libvirt/images" copy_on_write="false"/>
        <volume name="usr/local"/>
        <volume name="var/log"/>
        <volume name="var/opt"/>
        <volume name="var/spool"/>
        <volume name="var/tmp"/>
      </systemdisk>
    </type>
    <version>5.0.0</version>
    <packagemanager>zypper</packagemanager>
    <rpm-excludedocs>true</rpm-excludedocs>
    <bootloader-theme>SLE</bootloader-theme>
    <locale>en_US</locale>
    <keytable>sv-latin1.map.gz</keytable>
    <hwclock>utc</hwclock>
    <timezone>UTC</timezone>
  </preferences>

Expected behaviour

Steps to reproduce the behaviour

OS and Software information

KIWI version: 9.11.17
Operating system: SLES12-SP4
OBS version: 2.9

schaefi commented 4 years ago

Please boot the machine with:

kiwidebug=1

This will drop you into a shell and we have to find out why your disk has no free space. From the attached log information the move of the GPT to the end was successful. So I have no idea why there is no free space available.

Also note you are on a legacy path: boot="oemboot/..." is old kiwi custom boot code kept for compatibility with SLE12 <= SP3. We will fix bugs in that area only for customers with L3 support. I recommend switching to dracut.

Not that this will fix the problem because when there are no free cylinders there are no free cylinders :)

Jellyfrog commented 4 years ago

Switching to dracut is not possible atm sadly, After writing the image to disk, it looks like this:

# fdisk -l /dev/sda
GPT PMBR size mismatch (2574335 != 250069679) will be corrected by w(rite).
Disk /dev/sda: 119.2 GiB, 128035676160 bytes, 250069680 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 7D0FB046-59BB-4843-8682-FBFF0CFA62C8

Device      Start     End Sectors  Size Type
/dev/sda1    2048    6143    4096    2M BIOS boot
/dev/sda2    6144   47103   40960   20M EFI System
/dev/sda3   47104  661503  614400  300M Linux filesystem
/dev/sda4  661504 2574302 1912799  934M Linux filesystem

After the partition step fails, it has removed sda1:

# fdisk -l /dev/sda
Disk /dev/sda: 119.2 GiB, 128035676160 bytes, 250069680 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 7D0FB046-59BB-4843-8682-FBFF0CFA62C8

Device      Start     End Sectors  Size Type
/dev/sda2    6144   47103   40960   20M EFI System
/dev/sda3   47104  661503  614400  300M Linux filesystem
/dev/sda4  661504 2574302 1912799  934M Linux filesystem

# parted /dev/sda
GNU Parted 3.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) unit cyl                                                         
(parted) print free                                                       
Model: ATA SanDisk SD8SB8U- (scsi)
Disk /dev/sda: 15566cyl
Sector size (logical/physical): 512B/512B
BIOS cylinder,head,sector geometry: 15566,255,63.  Each cylinder is 8225kB.
Partition Table: gpt
Disk Flags: 

Number  Start   End       Size      File system  Name      Flags
        0cyl    0cyl      0cyl      Free Space
 2      0cyl    2cyl      2cyl      fat16        p.UEFI    boot
 3      2cyl    41cyl     38cyl     btrfs        p.lxboot
 4      41cyl   160cyl    119cyl    btrfs        p.lxroot
        160cyl  15566cyl  15405cyl  Free Space

(parted)

schaefi commented 4 years ago

Looks horribly broken :( Could you attach your image description (as complete as you can) such that we can reproduce the build and get a debug env why the partition table ended up that broken. The only thing it should have done is resizing the geometry for p.lxroot

Thanks

schaefi commented 4 years ago

Switching to dracut is not possible atm sadly

why ? iirc you are building the image with kiwi-ng (python) not the old perl kiwi right ? If you are based on SLE12 SP4 or newer the official recommendation for customers is to use kiwi-ng. Or are you using custom boot image hook scripts ? In this case I should know it because that would add extra initrd code which can have an influence

So your image description would be very helpful

Thanks

Jellyfrog commented 4 years ago

Switching to dracut is not possible atm sadly

why ? iirc you are building the image with kiwi-ng (python) not the old perl kiwi right ? If you are based on SLE12 SP4 or newer the official recommendation for customers is to use kiwi-ng. Or are you using custom boot image hook scripts ? In this case I should know it because that would add extra initrd code which can have an influence

So your image description would be very helpful

Thanks

We're using python kiwi, 9.11.17, it's the last version before the dracut code "took over".
We're not using kiwi to do the actual "writing image to disk" step, that we have a custom hook for.

We hook into postProbe; write the image to /dev/sda, rm /config.vmxsystem to prevent kiwi from running its own imaging; https://github.com/OSInside/kiwi/blob/e0348ae1d77b964456a5c5d77243dc8d38646933/kiwi/boot/arch/x86_64/oemboot/dump#L123-L125 But ofc the bug could be from not running that function, I didn't even thought of it since we used this way to deploy thousands of machines.

the image description is nothing more than;

<?xml version="1.0" encoding="UTF-8"?>
<image schemaversion="6.7" name="JEOS_SLE12.4">
  <description type="system">
    <author>...</author>
    <contact>...</contact>
    <specification>SLES12</specification>
  </description>
  <profiles>
    <profile name="default" description="x86 default for oem/vmx" import="true" />
  </profiles>

  <preferences>
    <type
      image="oem"
      initrd_system="dracut"
      kernelcmdline="splash quiet net.ifnames=1"
      firmware="uefi"

      filesystem="btrfs"
      btrfs_root_is_snapshot="true"
      fsmountoptions="acl"

      boot="oemboot/suse-SLES12"
      bootloader="grub2"
      boottimeout="5"
      bootprofile="default"
      bootpartition="false"

      installpxe="true"
      installiso="false"
      >
      <oemconfig>
        <oem-silent-boot>true</oem-silent-boot>
        <oem-boot-title>5.0.0</oem-boot-title>
        <oem-unattended>true</oem-unattended>
        <oem-swap>true</oem-swap>
        <oem-swapsize>4096</oem-swapsize>
      </oemconfig>
      <systemdisk>
        <volume name="home"/>
        <volume name="tmp"/>
        <volume name="opt"/>
        <volume name="srv"/>
        <volume name="boot/grub2/i386-pc"/>
        <volume name="boot/grub2/x86_64-efi" mountpoint="boot/grub2/x86_64-efi"/>
        <volume name="var/cache"/>
        <volume name="var/crash"/>
        <volume name="var/lib/machines"/>
        <volume name="var/lib/mailman"/>
        <volume name="var/lib/named"/>
        <volume name="var/lib/mysql" copy_on_write="false"/>
        <volume name="var/lib/mariadb" copy_on_write="false"/>
        <volume name="var/lib/pgsql" copy_on_write="false"/>
        <volume name="var/lib/libvirt/images" copy_on_write="false"/>
        <volume name="usr/local"/>
        <volume name="var/log"/>
        <volume name="var/opt"/>
        <volume name="var/spool"/>
        <volume name="var/tmp"/>
      </systemdisk>
    </type>
    <version>5.0.0</version>
    <packagemanager>zypper</packagemanager>
    <rpm-excludedocs>true</rpm-excludedocs>
    <bootloader-theme>SLE</bootloader-theme>
    <locale>en_US</locale>
    <keytable>sv-latin1.map.gz</keytable>
    <hwclock>utc</hwclock>
    <timezone>UTC</timezone>
  </preferences>

  <users>
    <user password="..." home="/root" name="root" groups="root" />
  </users>

  <repository type="rpm-md">
    <source path="obsrepositories:/" />
  </repository>

  <packages type="image">
    <package name="patterns-sles-Minimal" />
    <package name="aaa_base-extras" />
    <package name="kernel-default" />
    <package name="dracut" />
    <package name="fontconfig" />
    <package name="fonts-config" />
    <package name="grub2" />
    <package name="grub2-x86_64-efi" arch="x86_64"/>
    <package name="iproute2" />
    <package name="iputils" />
    <package name="less" />
    <package name="logrotate" />
    <package name="net-tools" />
    <package name="nscd" />
    <package name="ntp" />
    <package name="openssh" />
    <package name="pam-modules" />
    <package name="parted" />
    <package name="pciutils" />
    <package name="plymouth" />
    <package name="plymouth-dracut" />
    <package name="plymouth-plugin-label" />
    <package name="psmisc" />
    <package name="rsync" />
    <package name="rsyslog" />
    <package name="shim" />
    <package name="sudo" />
    <package name="suse-build-key" />
    <package name="sysconfig" />
    <package name="syslinux" />
    <package name="systemd" />
    <package name="systemd-sysvinit" />
    <package name="tar" />
    <package name="timezone" />
    <package name="unzip" />
    <package name="vim" />
    <package name="vim-data" />
    <package name="which" />
    <package name="wicked" />
    <package name="zypper" />
    <package name="ca-certificates-mozilla" />
    <package name="bind-utils" />
    <package name="ucode-intel" />

    <package name="cron"/>
    <package name="btrfsmaintenance"/>
    <package name="snapper-zypp-plugin"/>
    <package name="grub2-snapper-plugin"/>
    <package name="snapper"/>
  </packages>

  <packages type="delete">
    <package name="dropbear"/>
  </packages>

  <packages type="bootstrap">
    <package name="udev" />
    <package name="filesystem" />
    <package name="glibc-locale" />
    <package name="ca-certificates" />
    <package name="sles-release" />
  </packages>
</image>

davidcassany commented 4 years ago

@Jellyfrog I could not reproduce this issue... note I did small changes on the provided config.xml. I changed:

initrd_system="dracut" -> initrd_system="kiwi" Otherwise it won't use the custom defined KIWI initrds.
<oem-unattended>true</oem-unattended> -> <oem-unattended>false</oem-unattended>. This way I was easily able to choose the device and see what was kiwi boot code detecting.

With this little changes I was able to build and successfully deploy the image. The PXE server based in Leap 15.1, the build host an up to date SLE12-SP4 (kiwi-ng 9.17.18) and the image OS SLE12-SP4 including all updates.

I am afraid the issue might be caused by some of your hooks.

davidcassany commented 4 years ago

Closing the issue as it does not look like a KIWI bug.

OSInside / kiwi