Writing a xorriso-modified ubuntu image fails with 'ISO image extraction failure' on v4.5, but succeeds with <v4.5

[dseeley]

Checklist

• [x] I looked at https://github.com/pbatard/rufus/wiki/FAQ to see if my question has already been answered.
• [x] I performed a search in the issue tracker for similar issues using keywords relevant to my problem, such as the error message I got from the log.
• [x] I clicked the 'Log' button (🗒️) or pressed Ctrl-L in Rufus, or used DebugView, and copy/pasted the log into the section that says <FULL LOG> below.
• [x] The log I am copying is the FULL log, starting with the line Rufus version: x.y.z - I have NOT removed any part of it.

Additionally (if applicable):

• [x] I ran a bad blocks check, by clicking Show advanced format options then Check device for bad blocks, and confirmed that my USB is not defective.
• [ ] I also tried one or more of the following:
  • [ ] Using a different USB drive.
  • [ ] Plugging the USB into a different port.
  • [ ] Running Rufus on a different computer.
• [x] If using an image, I clicked on the (✓) button to compute the MD5, SHA1 and SHA256 checksums, which are therefore present in the log I copied. I confirmed, by performing an internet search, that these values match the ones from the official image.

Issue description

I am modifying a base Ubuntu 24.04 server image (adding cloud-init files and autoinstall grub menu) using xorriso (command below). The image is flashed successfully using Rufus <=v4.4, but with v4.5, I get Error: ISO image extraction failure and the logs show Error reading ISO9660 file /[boot]/1-boot-noemul.img at LSN 1341966. When running on v4.4, it doesn't even mention trying to read this file.

xorriso cmd: "xorriso -indev ubuntu-24.04-live-server-amd64.iso -outdev dougavisor_ubuntu2404_autoinstall.iso -volid dougavisor_ubuntu2404 -blank all --pathspecs on -add nocloud/meta-data=/tmp/nocloud/meta-data nocloud/user-data=/tmp/nocloud/user-data md5sum.txt=/tmp/md5sum.txt boot/grub/grub.cfg=/tmp/boot/grub/grub.cfg -- -boot_image 'any' 'patch'"

Log

Rufus x64 v4.5.2180
Windows version: Windows 11 Pro x64 (Build 22631.3672)
Syslinux versions: 4.07/2013-07-25, 6.04/pre1
Grub versions: 0.4.6a, 2.12
System locale ID: 0x0809 (en-GB)
Will use default UI locale 0x0809
SetLGP: Successfully set NoDriveTypeAutorun policy to 0x0000009E
Localization set to 'en-US'
Found 517 officially revoked UEFI bootloaders from embedded list
Found 2351 additional revoked UEFI bootloaders from this system's SKUSiPolicy.p7b
Found USB 3.0 device ' USB  SanDisk 3.2Gen1 USB Device' (0781:55A9)
Using 'autorun.inf' label for drive D: 'Ubuntu-Server 24.04 LTS amd64'
1 device found
Disk type: Removable, Disk size: 32 GB, Sector size: 512 bytes
Cylinders: 3740, Tracks per cylinder: 255, Sectors per track: 63
Partition type: GPT, NB Partitions: 1
Disk GUID: {6240439A-1526-4AF7-8596-D96423CBFE75}
Max parts: 128, Start Offset: 17408, Usable = 30765185536 bytes
Partition 1:
  Type: Microsoft Basic Data Partition
  Name: 'Main Data Partition'
  Detected File System: FAT32
  ID: {F7CBA6BA-5384-450D-B4BB-4B93DFCF6890}
  Size: 28.7 GB (30764122112 bytes)
  Start Sector: 2048, Attributes: 0x0000000000000000
Scanning image...
ISO analysis:
  Image is an ISO9660 image
  Reported Grub version: 2.12
Disk image analysis:
  Image has an unknown Master Boot Record
  Image is a bootable disk image
ISO label: 'dougavisor_ubuntu2404'
  Size: 2.6 GB (Projected)
  Note: File on disk is larger than reported ISO size by 342 KB...
  Has a >64 chars filename
  Uses: GRUB2 (2.12)
  Uses: EFI
  Note: This ISO uses symbolic links, which may not be replicated due to file system
  limitations. Because of this, some features from this image may not work...
Using image: dougavisor_ubuntu2404_autoinstall.iso (2.6 GB)

Computing hash for '\\filer.chezdj.com\dougal\Documents\Projects\dougalseeley\python\dougavisor\dougavisor_ubuntu2404_autoinstall.iso'...
  MD5:    9a299a135312d4d62013f1dd7de2af95
  SHA1:   09f7050f4c2ce3acd054179c3baee298ad242cae
  SHA256: 48b8bae1bc1c382d81e16fcae889590ef34e21f1926f593508674af98f290a6f

Format operation started
Requesting disk access...
Will use 'D:' as volume mountpoint
Opened \\.\PhysicalDrive1 for exclusive write access
Analyzing existing boot records...
Drive has a Zeroed Master Boot Record
Clearing MBR/PBR/GPT structures...
Erasing 128 sectors
Initializing disk...
Bad Blocks: Checking from block 0 to 58679 (1 block = 512 KB)
Bad Blocks: Using offset 238915 for fake device check
Bad Blocks: Writing test pattern 0x55
Bad Blocks: Reading and comparing
Bad Blocks: Check completed, 0 bad blocks found. (0/0/0 errors)
Clearing MBR/PBR/GPT structures...
Erasing 128 sectors
Partitioning (MBR)...
● Creating Main Data Partition (offset: 1048576, size: 28.7 GB)
Waiting for logical drive to reappear...
Formatting to FAT32 (using IFS)
Using cluster size: 16384 bytes
Quick format was selected
Creating file system...
Format completed.
Opened \\.\PhysicalDrive1 for exclusive write access
Writing Master Boot Record...
Partition is already FAT32 LBA...
Set bootable USB partition as 0x80
Using Grub 2.0 MBR
Writing Grub 2.0 SBR (from embedded)
Found volume \\?\Volume{d180f0f0-e69b-450a-a01e-a4c961a97b8c}\
Notice: Volume Device Path is \Device\HarddiskVolume34
Waiting for access on \\?\Volume{d180f0f0-e69b-450a-a01e-a4c961a97b8c}...
Opened \\?\Volume{d180f0f0-e69b-450a-a01e-a4c961a97b8c} for exclusive write access
Writing Partition Boot Record...
Using Standard FAT32 partition boot record
Confirmed new volume has a primary FAT32 boot sector
Setting primary FAT32 boot sector for boot...
Confirmed new volume has a secondary FAT32 boot sector
Setting secondary FAT32 boot sector for boot...
Successfully remounted \\?\Volume{d180f0f0-e69b-450a-a01e-a4c961a97b8c}\ as D:
Extracting files...
Image is an ISO9660 image
This image will be extracted using Rock Ridge extensions (if present)
Extracting: D:\[boot]\0-boot-noemul.img (2 KB)
Extracting: D:\[boot]\1-boot-noemul.img (5.0 MB)
  Error reading ISO9660 file /[boot]/1-boot-noemul.img at LSN 1341966

Found USB 3.0 device ' USB  SanDisk 3.2Gen1 USB Device' (0781:55A9)
1 device found
Disk type: Removable, Disk size: 32 GB, Sector size: 512 bytes
Cylinders: 3740, Tracks per cylinder: 255, Sectors per track: 63
Partition type: MBR, NB Partitions: 1
Disk ID: 0x1E177D81
Drive has a Grub 2.0 Master Boot Record
Partition 1:
  Type: FAT32 LBA (0x0c)
  Detected File System: FAT32
  Size: 28.7 GB (30764154880 bytes)
  Start Sector: 2048, Boot: Yes

[pbatard]

I'm afraid that I'm going to need access to an ISO that can replicate the issue to investigate this (I am not going to attempt to recreate an ISO, as there are too many environmental elements that could make it fail to replicate the issue and I'm afraid I don't have the luxury to invest time trying to replicate an image).

So can you please upload a problematic ISO somewhere on a file sharing site?

If you don't want to make it public, you can email me the link privately to: pete@akeo.ie.

Also, the fact that you use -indev ubuntu-24.04-live-server-amd64.iso to modify an existing ISO rather than create one from scratch, could indicate that this may be an issue with xorisso not properly updating the El-Torito bootloaders rather than an issue with Rufus (as Rufus 4.5 is now a bit more stringent about ISO-9660 compliance).

[dseeley]

Thank you for looking - I'll send a link.

[pbatard]

Thanks for sharing your ISO.

From looking at it, it would seem that the root of the issue is that it wasn't mastered properly.

If we look at the El Torito Section Entry for the problematic boot image (at address 0x096860 in your ISO), and broken into the relevant fields, we get:

88 00 0000 00 00 9C27 6E791400 00 00000000000000000000000000000000000000

The first bunch of fields are not directly relevant, but then we get to the 9C27 6E791400 part.

6E791400 (Little-Endian) translates to 0x0014796E which is 1341806 in decimal, a.k.a. the address (or LSN in ISO-9660 parlance) of the first ISO-9660 2408-byte block where the bootable image data should be read. So that tells us that the 1-boot-noemul.img bootable image starts at byte 1341806 x 2048 = 2,748,018,688 of your 2,748,383,232-byte ISO. So far so good.

Then 9C27 (Little-Endian) translates to 0x279C which is 10140 virtual image sectors of 512 bytes, which now tells us that the bootable image should be exactly 5,191,680 bytes in size.

However, when you add this all up, it so happens that your El-Torito boot records declares a bootable image starting at 2,748,018,688 and ending at byte 2,748,018,688 + 5,191,680 - 1 = 2,753,210,367 of your 2,748,383,232-byte ISO.

And therein is your problem. 2,748,018,688 + 5,191,680 is greater than 2,748,383,232. In its present state, your ISO declares a bootable image that goes past the actual size of the ISO!

Now, there is some software out there (e.g. 7-zip) that will just ignore such an overflow, and truncate the bootable image if it reaches the end of the ISO file. But Rufus (and other software that do attempt to respect the El Torito specs) don't. So you will have to fix your ISO, because, from where I stand, the root of the issue is that your ISO is not ISO-9660/El-Torito compliant in the first place, and if you want to interoperate with other software, you want to address that.

[pbatard]

Oh and for those who might wonder why older versions of Rufus worked, it's simply because we didn't process and try to extract El-Torito boot records then. And I'll grant that this content does not technically need to be extracted to create bootable media when using Rufus' ISO mode, but one of the goal of Rufus is to provide as much as the original data as we can, which is why we decided to start extracting these records (and some ISOs <cough>Mint<cough> do require us to actually pick some content from the El-Torito images, so we have to process these records, which we'll expect to be valid anyway).

[dseeley]

OK - Thank you for investigating. I'll see what options are available.

[pbatard]

OK, I'll close this issue then. But you will still be able to comment on it if needed.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue if you think you have a related problem or query.