puppylinux-woof-CE / woof-CE

woof - the Puppy builder
GNU General Public License v2.0
398 stars 287 forks source link

uefi-usb-installer is broken..? #1277

Closed wdlkmpx closed 6 years ago

wdlkmpx commented 6 years ago

I followed all the steps and i get a non-booting usb.

It installs grub2 and syslinux, it currently is broken if you build with syslinux 6.. but i fixed this issue locally and i get a usb that doesn't boot in uefi nor bios mode.

Here is an example of how to programatically create a uefi bootable usb https://ubuntuforums.org/showthread.php?t=2276498

It requires sgdisk which is part of gptfdisk https://github.com/puppylinux-woof-CE/woof-CE/commit/318f30bce355331179e6be2223de3d0d3d5539b5

However i see grub-install ... i wonder if that step can be omitted..

jamesbond3142 commented 6 years ago

I don't know usb-uefi-installer. I assume it's an installer run from within Puppy Linux to install itself to a flash drive, so that the flash drive can be used to boot UEFI machines (with secure boot disabled).

You only need sgdisk if your flash drive has GPT disklabel (that is, partition). If your flash drive is MBR (most commonly the case), then all you need:

  1. Primary FAT32 partition
  2. On that partition, you need /EFI/boot/bootx64.efi
  3. Now that bootx64.efi can be anything; if you use grub2 then that bootx64.efi is actually grub2.efi; if you use syslinux then that file is syslinux.efi (and you need to include various syslinux modules, etc), if you use refind then that would be refind.efi renamed to bootx64.efi, etc.
  4. The config file for your bootloader in step (3), placed in the correct place (for grub2.efi, at the root of the partition, for syslinux, inside /EFI/boot). For refind, it's inside /EFI/boot too.
  5. Of course the Puppy LInux files.
  6. If you want to make it boot in BIOS machine too, then proceed to install BIOS bootloader of your choice (grub4dos, syslinux, etc) as usual. You need to have another boot config file if the UEFI bootloader and the BIOS bootloader isn't the same.
wdlkmpx commented 6 years ago

uefi-usb-installer follows the same concept as the puppy uefi live cd, but it doesn't seem to work, well, at least not with syslinux 6. or maybe i did something wrong.

In my eyes the uefi live cd is good enough, with fix-usb.sh it should create another partition inside the usb drive (haven't tested fix-usb.sh yet), so basically it's mostly an automatic procedure with an extra step.

bootflash should offer an option to create a bootable usb flash drive just by selecting any iso file (hybrid iso),

I just took a look at gyro's frugal pup, i didn't read the whole thread, but i remember reading sometime ago that it was probably a heavily modified version of this script, which i think micko posted it on the forum or something, unless he wrote another one, and decided to post it on the forum and disappear. This looks like a completely different script, or set of scripts to be precise

I'm yet to test it.

jamesbond3142 commented 6 years ago

fix-usb.sh has an update thanks to SFR to support new output format from newer sfdisk. Please don't merge blindly - do a diff and update as necessary (I'm not sure what changes you have made for Puppy). fix-usb.sh.zip

gyrog commented 6 years ago

i've been on holidays for the last few weeks.

"frugalpup" is a different set of scripts following on from my haking of 01micko's installer. It embodies a few concepts that I have recently learned. 1). A bootable uefi usb stick does not require an image, just normal copying of the appropriate files to the fat32 partition. So no "fix-usb.sh" is required. 2). uefi boot does not require, that "vmlinuz" and "initrd.gz" reside on the fat32 boot partition. So it is possible to have a bootable uefi fat32 partition that contains only efi and grub2 configuration files, all puppy files residing in a normal frugal install directory, on a different partition, which could be on a different device. So it has a "grub4dosconfig" like facility that works for uefi and mbr boots.

It also provides: A). Full support for the boot partition, the Puppy install partition, and the Puppy default save partition being different. B) Specifies the appropriate partitions by their Label or UUID, via boot parameters.

Of course "frugalpup" is meant to be a "complete" frugal install manager, not just a uefi-usb-installer.

gyrog commented 6 years ago

Actually a better example of creating a bootable usb stick by simply copying files is found in this forum topic Another way of distributing Puppy?. So the uefi install could be simply; extract the contents of the ".zip" file to the root of the usb stick, and set it's Volume label to "USBPBOOT".

wdlkmpx commented 6 years ago

uefi-usb-installer calls gparted. i followed the steps and after that it apparently does things right: just copies stuff. it probably messes up something with syslinux 6. the result is unbootable on my machine. i haven't tested it in my new 6th gen core i3 laptop.

I find Windows unusable without 7zip or a winrar with an illegal rarreg.key and both create file associations for a number of known archive formats... and these apps open and can extract iso files to anywhere. Just like a zip file. A system without 7zip, winrar, engrampa or file-roller as the default archive manager is missing something very important...

The problem is the directory layout in the puppy iso, there are files that shouldn't be in the root directory, i actually have a patch to move everything that is not part of the running system to the /boot directory. That produces "a clean" ISO and of course breaks external scripts that deal with puppy isos. I guess that's why i've been trying to do everything inside rootfs-skeleton, otherwise doing something like that becomes impossible.

However apparently some windows apps are not able to properly process puppy isos because there are no joliet extensions, but i don't use any of those apps. Maybe some of those megaiso, ultraiso, superiso apps, i don't know.

I wonder if this is still a problem in multisessions cds/dvds: https://github.com/puppylinux-woof-CE/woof-CE/blob/testing/woof-code/boot/README.HTM

Testing is needed and the clock is ticking

The point i'm trying to make is that iso(9660)/udf is a universally understood format and concept.. cd/flash/hard drives boot with an iso9660 filesystem. By dd'ing a puppy uefi iso to a flash drive you get something that boots in bios and uefi modes. With just one command.

Machine emulators boot iso files in all popular OSes, for those who don't want to reboot or just want to use an OS in a virtual machine for any reason. ISO files can be opened like zip files, the problem is this knowledge is not that widespread,,, but i've known about this for some years now...

Having said that i'm not against your reasoning, i know you want to help people with new and old puppy versions, while i'm only focused on the latest and future versions of woofce.. which are a mystery.

wdlkmpx commented 6 years ago

I think rufus (most known windows app) can detect isohybrid isos and perform a dd-like operation. Yes, it can be a single step from Windows.

Rufus is a portable tiny app that is used to create Windows bootable usb drives to install Windows from usb. It also works with a number of different isos.. it should be a linux app.

It knows how to handle certain isos, and for those unknown isos (puppy isos), it can detect the isohybrid feature.

So, those who don't visit the puppy forum, will likely attempt to use Rufus to make a bootable puppy usb flash drive. I know i would.

But puppy doesn't properly support a usb flash drive + iso9660 filesystem, it does boot but shutdownconfig will get a bit confused and the init script will probably attempt pupmode 77 if everything else works.

wdlkmpx commented 6 years ago

So, recapilulating, uefi-usb-installer is broken, puppyinstaller is a clear and present danger and some good work must happen to fix both or maybe puppyinstaller should only perform full installs and frugalpup should replace uefi-usb-installer.

Meanwhile i've been testing isos.. as this principle not only applies to puppy, but a whole universe of distros..

fix-usb.sh was indeed broken. the attached file fixed the issue. It saved me from spending a couple of hours trying to fix it myself. thanks,. https://github.com/puppylinux-woof-CE/woof-CE/commit/427a3e0de8635bd8b455d6c8ca332b81058cd8c0

Using fix-usb.sh i realized that it may be a bit intimidating for a newbie. So i added 2 cli opts to perform the task without user intervention so it can be called from another script .. https://github.com/puppylinux-woof-CE/woof-CE/commit/9653602506f80613bc5087a5aa0063d58cb0c0c7

Commit https://github.com/puppylinux-woof-CE/woof-CE/commit/73838fe82e094bf76cb9801a3322aebfac4bf305 adds support for non-uefi isohybrid isos (without efiboot.img). Works in slacko 14.0 (2012) and dpup stretch (2017) https://raw.githubusercontent.com/puppylinux-woof-CE/woof-CE/73838fe82e094bf76cb9801a3322aebfac4bf305/woof-code/rootfs-skeleton/usr/sbin/fix-usb.sh

Now bootflash offers an option to write (burn) a iso file to a usb flash drive, the filesystem for the extra partition to regain the extra space can be ext3-4/ntfs/fat

This makes bootflash the only working builtin script able to create a usable uefi(+bios) usb drive.. and it doesn't need gparted.

I booted from sdc (usb flash drive, iso9660)... was able to boot in uefi and bios modes. Created a savefolder in sdc3 (ext4) and persistence is not happening. with pmedia=cd it should look for the pupsave, but i'm yet to find out what's wrong... in the future..

gyrog commented 6 years ago

I just extracted "uefi-usb-installer" from "bionicpup64-7.9.6-uefi.iso", and produced a bootable usb stick with it. It successfully booted as "uefi" and "legacy" on my desktop machine. Where can I get the broken one?

I intend to produce an equivalent installer, except that it will format the drive, rather than running "gparted", and use grub4dos instead of syslinux. It will have only 4 choices:

  1. The usb stick to use.
  2. The source type.
  3. The source file.
  4. Include "legacy" boot or not.
gyrog commented 6 years ago

If it needs a " fix-usb.sh" then it's too complicaed, and not using the usb device in it's normal way. A uefi bootable usb stick does not need an image, just ordinary copying of the appropriate files into a fat32 partition.

wdlkmpx commented 6 years ago

What a relief. I used it 3 times i think. I created a (msdos) partition table and added a fat32 partition.

Is it version 0.2 or 0.5? As far as i know only peabee produces updated isos regularly, i don't know about the rest. When i want to test something i use the latest release by peabee, assuming i'm in the mood to test something outside my little bubble

It didn't work, i followed all the steps, the script finished and the usb drv was not bootable.

The menu and directory layout are exactly the same as the live cd, except that a sed command changes pmedia=cd to pmedia=usbflash

And let's say i apply change_cd_structure.patch, what could happen

wdlkmpx commented 6 years ago

To support the new .efi boot images (which are in this pet pkg http://distro.ibiblio.org/puppylinux/pet_packages-noarch/grub2_efi-2.02.2018.08-fd64.pet , this alt version can also be used http://distro.ibiblio.org/puppylinux/pet_packages-noarch/grub2_efi_fed-2.02.2018.08-fd64.pet )

This line is to properly detect the presence of those files: https://github.com/puppylinux-woof-CE/woof-CE/blob/testing/woof-code/rootfs-packages/grub2_efi64/usr/bin/uefi-usb-installer#L92

And this is to copy the new files, but also supporting the old grubx64.efi.tar.xz from the rootfs-package grub2_efi64 https://github.com/puppylinux-woof-CE/woof-CE/blob/testing/woof-code/rootfs-packages/grub2_efi64/usr/bin/uefi-usb-installer#L402

which i plan to delete, but there's something going on

wdlkmpx commented 6 years ago

I guess now i know why it doesn't boot (at least in uefi mode)

This cp ${GRUBDIR}/grubia32.efi ${HOMEDIR}/EFI/boot/ cp ${GRUBDIR}/grubx64.efi ${HOMEDIR}/EFI/boot/

should be cp ${GRUBDIR}/grubia32.efi ${HOMEDIR}/EFI/boot/bootia32.efi cp ${GRUBDIR}/grubia32.efi ${HOMEDIR}/EFI/boot/bootx64.efi

Ah people can you hear me? And now I know and now I know And now I know and now I know That you can hear me And now I know and now I know And now I know now I know Now I know now I know Now I know now I know Now I know The earth will shake in two will break Death all around around around around Around around around around Now I know now I know Now I know now I know Now I know now I know Now I know now I know Now I know now I know Now I know

wdlkmpx commented 6 years ago

I agree that, at least from a running Puppy, it doesn't make sense to dd a puppy ISO to a usb flash drive.

Since "automatic" formatting and installation is available. For wee + grub4dos, the drive can be formatted as ext4, ext3, vfat, ntfs and it will boot.

For a uefi+bios install, basically wee + grub4dos in a vfat partition + EFI/boot/grub*.efi, and that's it (to be added). It takes adding 2 lines to support a efi boot.

I noticed that grub4dos fails to load large files in an ext4 partition, i cannot test isos with grub4dos, and this bug probably affects other bootloaders. It cannot load a huge initrd, it doesn't affect puppy, just saying that i see why people use ext3

However offering to write any other ISO (that is not a isohybrid puppy iso) to a flash drive doesn't sound like a bad idea, a service to test any other distro that uses isohybrid (i.e. debian live)

wdlkmpx commented 6 years ago

I wonder if micko will get angry if i delete uefi-usb-installer. Bootflash as it is, and adding a few more lines revising the logic to select source, seems to be more than enough.

I think a similar logic can be applied to puppyinstaller.... about 1000 lines can be deleted and then bootflash can be integrated into puppyinstaller

gyrog commented 6 years ago

If you are modifying installers, it would be nice if you ensured that they fully support "init": a) write "pupsfs=" (and "psubdir=") parameters to the boot entry, preferrably using a Label or UUID to specify the partition. b) offer an option to specify a different location for "saves", and if so, write either a "psavemark=" or "psave=" boot paramter to specify this separate location.

Note: My "FrugalPup" is an example of an installer that does this.