nixos: make iso network bootable

[cillianderoiste]

With syslinux, we should be able to create a hybrid image, which can be copied to a usb-stick using dd or burnt to a CD with: isohybrid filename.iso

http://www.syslinux.org/wiki/index.php/Doc/isolinux#HYBRID_CD-ROM.2FHARD_DISK_MODE

[offlinehacker]

Yep, isolinux is one option. On Apr 2, 2014 4:08 PM, "cillianderoiste" notifications@github.com wrote:

With syslinux, we should be able to create a hybrid image, which can be copied to a usb-stick using dd or burnt to a CD with: isohybrid filename.iso

http://www.syslinux.org/wiki/index.php/Doc/isolinux#HYBRID_CD-ROM.2FHARD_DISK_MODE

Reply to this email directly or view it on GitHubhttps://github.com/NixOS/nixpkgs/issues/2100#issuecomment-39334021 .

[FrozenCow]

Same issue here. The wiki noted that making the image hybrid should be possible, but it is not since the current bootloader on the ISO is grub. I've removed this section from the wiki: https://nixos.org/w/index.php?title=Installing_NixOS_from_a_USB_stick&diff=22883&oldid=22783

Ideally you'd want an image that boots on both CD and USB as well as legacy bioses, UEFI and Mac OSX. From the different distributions that I've tried, ArchLinux does this best. It uses a recent isolinux with some additional work to get different UEFI systems to work using Gummiboot. From what I could tell, NixOS already uses Gummiboot for UEFI. I don't have much experience with PXE, but I know that same image can be used for PXE as well (not sure whether UEFI/legacy is an issue here).

It might be a good idea to take a look at how ArchLinux does this. The script for creating their ISOs is located in the archiso project (https://projects.archlinux.org/archiso.git/). More specifically https://projects.archlinux.org/archiso.git/tree/archiso/mkarchiso and https://projects.archlinux.org/archiso.git/tree/configs/releng/build.sh

I've once needed an ISO that supported booting on all of the different setups. I made a simple/minimal script based on the code of archiso. You can find this project here: https://github.com/FrozenCow/drivedroid-image The only thing that's missing from this project is support for PXE booting.

I'm new to NixOS and the packaging system, so I haven't been able to figure out how to get all this working with Nix. If I could get some pointers where to look at/for, I can try to implement the different booting mechanisms.

[vikstrous]

This! Please! I don't have CDs and I don't want to buy CDs. Even if I had one, I don't have anything to burn a CD with any more. All of my options for installing NixOS on hardware are extremely inconvenient right now.

[wkennington]

@vikstrous You could just use unetbootin for writing the current iso to a usb drive until a hybrid iso is made.

[vikstrous]

Unetbootin is not working for me on OS X Mavericks. The drives dropdown doesn't have anything in it. I tried two different USB sticks and it doesn't see either one.

[Fuuzetsu]

You need to mount the USB somewhere before unetbootin can see it. Make sure you're running it as root. I can only say that unetbootin does work with NixOS ISOs.

[lucabrunox]

Btw I can say that unetbootin does not always work. Debian since a couple of years has ISOs bootable from usb, and that's very handy. So :+1: for usb-bootable iso images, please.

[FrozenCow]

It would be very convenient if syslinux could be used and a simple dd can write the image to an USB disk. Tools like unetbootin feel a bit hacky since it replaces the existing bootloader and boot configuration with one of unetbootin.

It would also allow me to add NixOS to DriveDroid, which makes it possible to run/install NixOS on a PC using an Android phone. It will be as simple as choosing NixOS from a list, wait for the download to finish, choosing the NixOS image to host over USB and booting the PC.

I'm willing to dig into this a bit more, but I need some pointers on where to start with the integration of syslinux.

[lucabrunox]

@FrozenCow I can help with that if you jump on irc. I don't know anything about uefi.

[aristidb]

:+1: This would be cool.

[bobvanderlinden]

For anyone interested, I've submitted #4678 that uses syslinux instead of grub for the ISO. That way the ISO can be burned to an USB stick using dd and does not need tools like UnetBootin anymore. This should make the use of the ISO on USB less error-phone. Thanks @lethalman for the help getting things going.

Additionally this enables usage in DriveDroid, so that you can boot NixOS through an Android phone.

As for network booting, this isn't in the PR. I don't think an ISO is ideal for PXE booting. I'd rather see some directory being used to install NixOS to with PXE-booting support. NFS can in turn be used to host the directory. Using the ISO for PXE is convoluted (or so I've seen in ArchLinux), where you need to mount the ISO on the PXE server and do some fiddling afterwards. It would be nice to have some nix-build -A pxe_minimal.x86_64-linux that creates a PXE/NFS compatible root directory.

[lucabrunox]

The usb bootable part has landed in master, I'd remove this issue from the 14.11 milestone.

[ehmry]

I've successfully PXE booted something much like the installer, But its quite a hack, we're going to need network support in the initrd to make it work: https://github.com/NixOS/nixpkgs/issues/5265

Other than getting the network going, its was trivial to replace the nix store on the PXE client with CIFS rather than squashfs, there isn't much more to do than that.

[wmertens]

Incidentally, it would be awesome if nixos.org could host an iPXE target so e.g. vultr.com VMs could be booted straight from there...

[ehmry]

I have a module to do a little of the PXE hosting configuration. I tried to get generate a squashfs and then insert that into the initrd, but generating the store closure before the initrd causes an infinite recursion. I'm not making a PR for the PXE stuff but its here if someone wants to pick it up.

https://github.com/ehmry/nixpkgs/commit/571f5ec4b12b6bec5704002f895628dec0d95412

[cleverca22]

i have also been playing with booting normal nixos from the network, and i think ipxe would allow booting a syslinux/grub based iso over the network, but you would need some initrd support to find the network and mount it properly

[ehmry]

Yes, I think the thing to do with the least amount of obscure hackery and the most amount of reuse would be to get the initrd to do networking.

[cleverca22]

first thing to decide on is which network protocols to support, nfs would be easy in the initrd, but need a bit more config to make it boot

iscsi should be easy, but you would need to install a target daemon like tgtd on the system serving the iso

i don't think ftp/http(s) can be loopback mounted, but the initrd could just curl it to a tmpfs and boot that

there is also the question of how much network configuration will be required, dhcp, http, iscsi, nfs, tftp

ipxe cant boot from nfs, but it can fetch the kernel+initrd over tftp or http, then initrd can mount nfs for the iso

[ehmry]

I think NFS is garbage, so I would say CIFS or diod and 9P, but I've been told that samba seemed to outperform diod.

[woffs]

I would be happy with kernel+initrd over tftp and iso/root over nfs. Or sqashfs over tftp.

[cleverca22]

this is what i have so far: https://gist.github.com/cleverca22/e266881fb6645c126710

bzImage/initrd/nix-store.squashfs are just copied right out of the iso, the init= must match what the squashfs contains, so you have to read the isolinux config file

ipxe will load the config+kernel+initrd over http, but tftp could also be used, the initrd should mount the dir over nfs(9P or anything else the kernel supports also work) for the squashfs file, but it currently lacks nfs support

[pille]

not directly netboot, but there's a convention for loopback booting via grub: http://www.supergrubdisk.org/wiki/Loopback.cfg

perhaps you could generate that along.

[bobvanderlinden]

Just saw http://netboot.xyz/#netbootxyz, might be a good use if nixos was hosting kernel, initrd and squashfs.

ArchLinux seems to just download squashfs (https://github.com/rcrowley/archiso/blob/master/archiso/initcpio/hooks/archiso_pxe_http#L30-L61) and the url for squashfs is passed as a parameter for the kernel (which could be done from the ipxe file).

[fpletz]

@bobvanderlinden Thanks for mentioning netboot.xyz. Ever since boot.kernel.org was shut down I was searching for an alternative. I'll try to add NixOS next week.

[domenkozar]

@fpletz did you manage to? I'd love to use this.

[globin]

I've opened an issue in their issue tracker.

[bobvanderlinden]

@globin NixOS doesn't have a kernel + initrd + squashfs (or other files that iPXE to boot NixOS) hosted over HTTP as far as I know. It would be nice to have something like that, instead of the ISO (which probably requires fiddling). @cleverca22's work seems like the closest there is, but it needs to be in nixpkgs so that hydra (and others) can reproduce it as well. I think it would be nice if there was one build(-attribute) that would result in these 3 files, but there might be better ways(?).

[domenkozar]

It should be possible to factor that outside the ISO generation module.

[domenkozar]

Also for reference see https://github.com/sleinen/nixos-pxe-installer

[antonym]

To get the ISO booting from memdisk via iPXE, have you all looked into the memdiskfind, phram and mtdblock method to identify where the ISO has been mapped in memory and then mount that once the kernels are loaded? I've been poking around the internet a bit collecting info about that method here.

[nshalman]

I have a similar use case, but for a small stripped down system, and I was thinking that it might be interesting to just embed the squashfs inside the initrd. Once the kernel and initrd have been downloaded over the network the system would have everything it needs to boot. If anyone has the time to help me implement that I'd appreciate it.

[nshalman]

Followers of this ticket may be interested in trying out https://github.com/NixOS/nixpkgs/compare/master...nshalman:netboot-v2 as I think it's nearly ready to be converted into a PR...

[bobvanderlinden]

The PR is submitted: https://github.com/NixOS/nixpkgs/pull/14740

Suggestions are welcome!

[domenkozar]

@offlinehacker could you open a new issue describing exactly what in your case means "network boot"? I'd really like to close this catch-all issue that serves no use. If we want ISO to be network friendly, it's really easy as adding another profile that allows root ssh and sets password to "".