UEFI:NTFS hard crash on Windows Developer Kit 2023

[jglathe]

Not sure if this is the right place, I'll give it a try. I tried to build and boot a Windows 11 for ARM64 USB stick with rufus, this worked - sort of. What didn't work was booting from this stick on wdk2023, crashed with a Synchronous Exception.

What worked was to format the same stick with exFAT, make it bootable with UEFI:NTFS, copy over all files from the image to the exFAT partition of the stick, boot.

Which is fascinating and great, but apparrently there is an issue with ntfs in this constellation. What can I do to provide some more debugging info?

[pbatard]

Thanks for the report. This is the right place to report UEFI:NTFS issues indeed, even if it looks like an NTFS driver problem.

I am actually in the process of acquiring a Windows Developer Kit myself, so I should be able to replicate and investigate this issue once I have it. I'll keep this issue updated with my findings then.

[jglathe]

Those are nice little boxes, good amount of RAM vs processing power, and nice RAM bandwidth. Fun to do things on, especially if its the machine the monitor(s) are on.

[pbatard]

Just a note that I got my Windows Dev Kit 2023 and can replicate the issue, be it when using UEFI:NTFS or when simply loading the NTFS driver from the UEFI Shell.

My current guess is that the Qualcomm CPU used on these machines is probably a lot less forgiving than other ARM CPUs on data misalignment (e.g. trying to read a 16-bit word that is stored at an odd address rather than an even one), and there's probably a structure somewhere in the driver that has an element that is not aligned properly.

Now, the issue is that the ntfs-3g driver is quite complex, so trying to locate precisely what is the attribute from the code being accessed that is responsible for the exception may take a while. This is further compounded by the fact that, currently, I can only test this issue by recompiling the driver through EDK2 (which is cumbersome) rather than gnu-efi (which can be accomplished from within Visual Studio), on account that not only does gnu-efi appear to have a similar data alignment issue on simple structure (a mere "Hello World" compiled with gnu-efi will generate an exception on the Windows Dev Kit), but, worse, OpenSUSE, Red Hat and co. recently applies patches to gnu-efi that completely broke ARM64 executables, even on non Windows Dev Kit systems (i.e. they managed to break the sample "Hello World" application from gnu-efi, that used to work fine on non Windows Dev Kit systems prior to these patches), so I can't even look into trying to fix gnu-efi compilation...

[jglathe]

Ouch. Sounds like an issue. Probably a good idea to wait it out. Maybe as a workaround rufus could enable/allow burning the Windows ISO af exFAT? No idea if that's easy. I made it with two steps.

[pbatard]

Issues had been found and resolved. Basically it appears that the Dev Kit's firmware didn't wait to exit the function that installed the driver to try to instantiate (mount) the file systems, which means that some of the initialisation that was needed for the instantiation to work happened too late and resulted in a NULL pointer dereference.

Once we moved that initialisation to happen earlier, the problem was fixed and Rufus 4.1, which I plan to release shortly, will create UEFI:NTFS drives that are compatible with the Dev Kit.

Maybe as a workaround rufus could enable/allow burning the Windows ISO af exFAT?

That's not an option, because we CANNOT sign the exFAT drivers for Secure Boot, on account that they are licensed under GPLv3, and Microsoft has arbitrarily and abusively decided that they would never ever sign anything that's GPLv3.

As a matter of fact, even though we had a perfectly fine NTFS driver, we had to craft a whole new one just so it wasn't licensed under GPLv3 so that Microsoft would sign it, and this took us months of hard work, so we're not going to go through this exercise again for exFAT (which, by the way, Microsoft could easily have licensed for use for UEFI, just like they did for FAT, and just like they did later for exFAT with Linux, but chose not to do, even as they were well aware that the 4GB limit of FAT, which is why we have to have this whole UEFI:NTFS thing).

Oh, and for the record, Rufus can already create a blank UEFI:NTFS drive with exFAT (if you have advanced options enabled you can create blank UEFI:NTFS disks, and you can choose between NTFS and exFAT). We're just not allowing exFAT to be selected when creating a Windows drive on account that exFAT boot is not supported by older versions of Windows, and, more importantly, because then we'd get LOADS OF COMPLAINTS FROM USERS, just like we did when we used a GPLv3 NTFS driver, that they have to temporarily disable Secure Boot...

Finally, the whole irony of this is that, on the Dev Kit and on any recently released platform that has an UEFI firmware that Microsoft controls (such as Hyper-V), Secure Boot signed UEFI:NTFS won't even work by default, because Microsoft (who, it should be reminded, are the sole entity in control of Secure Boot signing, therefore it's hard to find a non nefarious reason why they had to do the following) have a special version of Secure Boot that is "more Secure Boot than Secure Boot" and that only lets EFI binaries that have been signed with special credentials from Microsoft through, and that, unlike formal Secure Boot, Microsoft will never ever allow third parties to be signed with. So, by default, you can't even boot a Secure Boot signed bootloader on the Windows Dev Kit... and you can't therefore install Windows from a Windows installation media that you created yourself and that is fully Secure Boot signed...

[pbatard]

By the way, if you want to test, you can find a build of Rufus with the updated UEFI:NTFS in the VS2022 Artifact at https://github.com/pbatard/rufus/actions/runs/5090300154.

[jglathe]

Tried the build of rufus with the same iso, rufus burning on arm64. Boots now, thank you! Regarding Microsoft and its shenanigans: They appear to have come full circle and are back to their ways of the 90s again. @thurrott is talking about this, too. I love the kind of tools rufus provides, it is like leveling the playing field a little. And regarding Secure Boot... well, if its just a tool to keep competition out, what is it worth?

[Masamune3210]

With Secure Boot, its a bit more complicated. Its a genuinely good idea and should be a thing, it just shouldn't be Microsoft of all companies holding the reins for what can be signed, who can get signed, and how the system works. The issue isn't the signing and validation of boot files, the issue is who is mandating things.

[pbatard]

With Rufus 4.1 being released today, I will consider this issue as resolved.