Cannot boot live cd

[lucabrunox]

So I hear this from people from time to time, and it happened to me as well with usb stick. Then I rebooted twice and it happened to work. It's kind of random, I think some timing issue in the initrd or such.

The error I got was something like:

mount: mounting /mnt-root/iso/nix-store.squashfs on /mnt-root/nix/.ro-store failed: No such file or directory

I think musicmatze on irc got a similar message, and still cannot boot the live cd. Together with this also a warning about lvm leaking some file descriptors like:

File descriptor 3 (/nix/store/something-initrd-fsinfo) leaked on lvm invocation. Parent PID 1:...

So maybe the two are related.

cc @edolstra may know more.

This is a serious issue.

[matthiasbeyer]

Hi, yes I have this issue. Booting from USB.

[lucabrunox]

Same problem from @kylescottmcgill even with screenshot (thanks!):

https://cloud.githubusercontent.com/assets/245931/5970091/51603af2-a892-11e4-8d17-55fcea30a481.JPG

[lucabrunox]

Could it be that the wrong /dev/root gets mounted to /iso? Also note that "Invalid argument" in fsck.vfat.

[lucabrunox]

As I read the code of the stage-1: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/boot/stage-1-init.sh

It seems like that file either doesn't exist under /mnt-root/iso or the wrong /dev/root gets mounted.

[wkennington]

I noticed a bug in new kernels regarding xhci bus initialization so the USB device never comes up. As far as I can tell this only affects haswell on 3.18+ kernels. On Feb 9, 2015 9:10 AM, "lethalman" notifications@github.com wrote:

As I read the code of the stage-1: https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/boot/stage-1-init.sh

It seems like that file either doesn't exist under /mnt-root/iso or the wrong /dev/root gets mounted.

— Reply to this email directly or view it on GitHub https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-73547518.

[wmertens]

I think https://github.com/NixOS/nixpkgs/blob/80a85541d557df23a01f578aaf31e9777a894add/nixos/modules/system/boot/stage-1-init.sh#L348-L361 is somewhat wrong... it should either try forever or fail obviously after 20 tests.

Right now it just continues after 20 tests, as can be seen by the 20 lvm error messages.

So the problem is that USB doesn't come online (where it's expected/at all) and the stage 1 doesn't make that problem obvious.

On Mon Feb 09 2015 at 6:18:21 PM William A. Kennington III < notifications@github.com> wrote:

I noticed a bug in new kernels regarding xhci bus initialization so the USB device never comes up. As far as I can tell this only affects haswell on 3.18+ kernels. On Feb 9, 2015 9:10 AM, "lethalman" notifications@github.com wrote:

As I read the code of the stage-1:

https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/boot/stage-1-init.sh

It seems like that file either doesn't exist under /mnt-root/iso or the wrong /dev/root gets mounted.

— Reply to this email directly or view it on GitHub https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-73547518.

— Reply to this email directly or view it on GitHub https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-73549053.

[wmertens]

So I wonder if that udev settling is needed when the file exists, maybe it should be behind a "if try is 20" test.

Also note that the lvm error message is probably due to the fd3 magic used for reading the fsinfo: https://github.com/NixOS/nixpkgs/blob/80a85541d557df23a01f578aaf31e9777a894add/nixos/modules/system/boot/stage-1-init.sh#L313-L318

On Mon Feb 09 2015 at 6:21:48 PM Wout Mertens wout.mertens@gmail.com wrote:

I think https://github.com/NixOS/nixpkgs/blob/80a85541d557df23a01f578aaf31e9777a894add/nixos/modules/system/boot/stage-1-init.sh#L348-L361 is somewhat wrong... it should either try forever or fail obviously after 20 tests.

Right now it just continues after 20 tests, as can be seen by the 20 lvm error messages.

So the problem is that USB doesn't come online (where it's expected/at all) and the stage 1 doesn't make that problem obvious.

On Mon Feb 09 2015 at 6:18:21 PM William A. Kennington III < notifications@github.com> wrote:

I noticed a bug in new kernels regarding xhci bus initialization so the USB device never comes up. As far as I can tell this only affects haswell on 3.18+ kernels. On Feb 9, 2015 9:10 AM, "lethalman" notifications@github.com wrote:

As I read the code of the stage-1: https://github.com/NixOS/nixpkgs/blob/master/nixos/ modules/system/boot/stage-1-init.sh

It seems like that file either doesn't exist under /mnt-root/iso or the wrong /dev/root gets mounted.

— Reply to this email directly or view it on GitHub https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-73547518.

— Reply to this email directly or view it on GitHub https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-73549053.

[wkennington]

Well I mean, the stage 1 can't know that you are using a USB storage device for the NixOS iso before the USB device with the label NIXOS_ISO comes up. It was intended to be a cd. The worst part for me is that I don't even have keyboard access (since xhci is broken) on the machines which fail so I can't really troubleshoot. On Feb 9, 2015 9:22 AM, "Wout Mertens" notifications@github.com wrote:

I think

https://github.com/NixOS/nixpkgs/blob/80a85541d557df23a01f578aaf31e9777a894add/nixos/modules/system/boot/stage-1-init.sh#L348-L361 is somewhat wrong... it should either try forever or fail obviously after 20 tests.

Right now it just continues after 20 tests, as can be seen by the 20 lvm error messages.

So the problem is that USB doesn't come online (where it's expected/at all) and the stage 1 doesn't make that problem obvious.

On Mon Feb 09 2015 at 6:18:21 PM William A. Kennington III < notifications@github.com> wrote:

I noticed a bug in new kernels regarding xhci bus initialization so the USB device never comes up. As far as I can tell this only affects haswell on 3.18+ kernels. On Feb 9, 2015 9:10 AM, "lethalman" notifications@github.com wrote:

As I read the code of the stage-1:

https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/boot/stage-1-init.sh

It seems like that file either doesn't exist under /mnt-root/iso or the wrong /dev/root gets mounted.

— Reply to this email directly or view it on GitHub https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-73547518.

— Reply to this email directly or view it on GitHub https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-73549053.

— Reply to this email directly or view it on GitHub https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-73549801.

[lucabrunox]

@wmertens the usb drive is mounted, otherwise the mount of /iso would fail. The problem is that the squashfs file inside is not found. So either burning went wrong, or the wrong /dev/root is being mounted.

[wmertens]

I proposed #6266 which might help here - if it can't find the device it does an ls of /dev/ for people without keyboards :grin:. Should be harmless to merge.

[wkennington]

USB Issues should be fixed in b556983eb74c07c3168600eaa8ee5fb9f28f6bb6

[kylescottmcgill]

As for the xhci bus initialization, i can disable that on my PC at home, and that will provide some more details (maybe?). Also that PC (which has the same issue) is a i7-3770K, which from memory isnt a Haswell Processor. But the laptop (the one in the screenshot) is certainly a Haswell Processor.

If i can provide anything, please let me know, happy to test certain builds etc.

[wkennington]

@kylescottmcgill I think I've worked that one out. There was a refactoring in the 3.18 kernel which requires the xhci_pci module to also be added to the initrd.

[matthiasbeyer]

Btw, I got it working by doing what's in the manual (copying the iso contents to the usb drive by hand, setting the usb drive label appropriately and so on).

[lucabrunox]

@wkennington but nixos live cd doesn't use 3.18 by default, does it? Default linux is 3.14 currently.

[wkennington]

Default yes, but I use the new kernel iso On Feb 10, 2015 1:31 AM, "lethalman" notifications@github.com wrote:

@wkennington https://github.com/wkennington but nixos live cd doesn't use 3.18 by default, does it? Default linux is 3.14 currently.

— Reply to this email directly or view it on GitHub https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-73669247.

[bobvanderlinden]

This might be of interest here as well. It seems an image based on 96d6344 (comment) doesn't boot with UEFI, but an image based on 2d19af84de (comment) does boot. Do note that these are not the standard grub-based images, but syslinux-based images. The same problems have popped up for syslinux-based images as described in this thread.

[domenkozar]

Indeed, this is fixed in master. Re-open if the problem re-appears.

[bobvanderlinden]

@iElectric could you reference the code that fixes this problem? I'm a bit confused on what the problem was and what fixed it.

[domenkozar]

I don't know. It does work now with latest iso.

[ryantm]

I downloaded nixos-minimal-15.09.768.1a86bf8-x86_64-linux.iso and installed it on a flash drive with dd bs=4M if=nixos.iso of=/dev/sdX and then tried to boot it using refind. It displays typical linux syslog info and gets stuck with:

timed out waiting for device /dev/root, trying to mount anyway.
mounting /dev/root on /iso...
mount: mounting /dev/root on /mnt-root/iso failed: No such file or directory

An error occurred in stage 1 of the boot process, which must mount the root filesystem on `/mnt-root' and then start stage 2. Press one of the following keys:
  r) to reboot immediately
  *) to ignore the error and continue

If I try to ignore it it says it is waiting for device /mnt-root/iso/nix-store.squashfs to appear but it gives up with:

Timed out waiting for device /mnt-root/iso/nix-store.squashfs, trying to mount anyway.
mounting /mnt-root/iso/nix-store.squashfs  on /nix/.ro-store
[   417.288237] squashfs: version 4.0 (2009/01/31) Phillip Lougher
mount: mounting /mnt-root/iso/nix-store.squashfs on /mnt-root/nix/.ro-store failed: No such file or directory

An error occurred in stage 1 of the boot process...

Ignoring again causes a kernel panic because it cannot execute init.

[ryantm]

I also tried nixos-graphical-15.09.768.1a86bf8-x86_64-linux.iso with the same problem.

[ryantm]

I do not have this problem if I boot the live USB from the BIOS directly instead of from rEFInd running on my SSD.

[lucabrunox]

@ryantm thanks for the tests.

I think that in general we must print some more debugging info when /dev/root is not found. I hear about this problem too often.

The iso boot works by detecting a disk with LABEL=NIXOS_ISO, which is then symlinked to /dev/root. So when it fails we have to let people easily debug this stuff, like at least print the blkid output.

[otezun]

@lethalman

Thank you for the tip with LABEL=NIXOS_ISO. I used SDCardFormatter to format my USB and give it the label NIXOS_ISO, then installed nixos-graphical(64 bit) onto it with UNetbootin. It now boots the USB key. As far as I could see, it would not boot if the label was None or not NIXOS_ISO.

This is a minor annoyance. But it works, so whatever.

[bobvanderlinden]

Does that mean the label is not set correctly to the image? Or is it not correctly set when it is used for a USB drive?

This is where the label is written to the ISO: https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/make-iso9660-image.sh#L105

[otezun]

Well I can only tell you what I did, which was using SDFormatter, formatting USB Drive, then writing the iso onto the USB drive with unetbootin. It did not change the USB Drives label when I did that. So it is most certainly not correctly set when it is used for a USB Drive.

I don't know how it is with the CD though. Never tried that, since this laptop does not have an optical drive.

[davidak]

I have this problem with the latest image. Used UNetbootin to copy to usb-stick like described in manual.

nixos-graphical-15.09.903.788f56a-x86_64-linux.iso and also nixos-graphical-16.03pre75088.b7ff030-x86_64-linux.iso

hp dc7900 with elementary OS (Ubuntu 14.04 based) installed on HDD, BIOS boot

Dell Laptop with i7, EFI boot

Macbook Pro (Retina), Mac EFI boot

That problem is solved by renaming Volume to NIXOS_ISO on EFI boot.

[matthiasbeyer]

Try the fallback image as described in https://github.com/NixOS/nixpkgs/issues/8633 - Maybe this works...

[davidak]

I tried dd but then there is no EFI option. and BIOS boot also don't work

hp dc7900, BIOS

Macbook EFI

[jgillich]

I don't think UNetbootin works with the image (or most images, really). What do you mean by there is no EFI option when you use dd?

[davidak]

On the Macbook, i see an emulated BIOS entry and an EFI entry for the image copied with UNetbootin. With dd only BIOS.

[davidak]

i tried older images i have laying around but they have all this error.

nixos-graphical-15.09.546.e64b9e7-x86_64-linux.iso (3.10.15) nixos-graphical-14.12.877.0341382-x86_64-linux.iso (31.8.15)

so i am not able to install NixOS on a PC with BIOS. this issue should have priority.

[jgillich]

Are you sure there isn't something wrong with your usb stick? I did boot 15.09 not too long ago using UNetbootin as well dd. Although I've had UNetbootin break a lot with other distros, so I generally advise against it.

[davidak]

I have the same problem with another usb-stick. maybe someone can reproduce it.

[davidak]

I still can't boot NixOS on a PC with BIOS!

tested latest images:

nixos-graphical-15.09.1113.23395c4-x86_64-linux.iso nixos-graphical-16.03.30.2068621-x86_64-linux.iso

[davidak]

I tried again with dd and it works!

@domenkozar so only the manual must get changed.

On OS X you have to use bs=4m instead of bs=4M. You don't have dd installed on Windows. Other Distros recommend http://rufus.akeo.ie/ someone with Windows could try it.

[jgillich]

The wiki is currently being converted into documentation, after which it will be shut down. Help is always welcome, #13310 would be the issue for this.

Win 32 Disk Imager is a good alternative to dd on Windows: https://sourceforge.net/projects/win32diskimager/

[akisystems]

Renaming my USB disk to NIXOS_ISO fixed the stage 1 boot issue for me.

[lminaudier]

Hi,

I am trying to install NixOS on a Dell XPS 13 (9350, late 2016). I have created a bootable usb stick via dd command, disabled Secure Boot and moves from RaidOn to AHCI in BIOS settings.

But I still get an error on stage 1

mount: mounting /dev/root on /mnt-root/iso failed: No such file or directory

when I try to boot on the stick.

I think I have labelled my partition correctly with NIXOS_ISO. Here is the output of e2label.

$ sudo e2label /dev/sda1
e2label: Bad magic number in super-block while trying to open /dev/sda1
/dev/sda1 contains a iso9660 file system labelled 'NIXOS_ISO'

/dev/sda is my usb stick and I boot with UEFI.

I tried with 16.09 and 17.03-pre version, same issue.

There is probably something that I am missing or doing wrongly but I can't spot it.

By any chance, do you have any clue ?

EDIT

I tried to boot with legacy mode and reached the same error.

[lminaudier]

Ok the error was due to my usb stick. Using another one with dd and it booted correctly.

[abcdw]

Changing devices label to NIXOS_ISO fixed issue for me.

[bmenaman]

had this problem today with the latest minimal iso on a 2015 Macbook Pro with refiind installed. fixed it by re-running dd with ´bs=4M´. Though I suppose itś possible that the first dd failed. If bs=4M is required the manual doesn´t mention it

[tomfitzhenry]

I experienced timed out waiting for device /dev/root, trying to mount anyway. on nixos_minimal_18.09.2375.680f9d7ea90_x86_64_linux.iso.

tl;dr Using a different USB stick worked.

I apologise in advance for not having as much log output as ideal.

On the failed boot attempt, I dropped into an interactive shell and observed that the USB stick (from which I was installing NixOS) wasn't in /dev (including /dev/disk/). Upon ejecting and reinserting the USB stick, the USB stick did appear in /dev, with its two partitions. The root partition of the USB stick failed to mount, however. fdisk looked seriously wrong, as if the partition table and the actual partitions were inconsistent.

/mnt-root # fdisk -l /dev/sdc
Disk /dev/sdc: 3902 MB, 4091543552 bytes, 7991296 sectors
497 cylinders, 255 heads, 63 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Device  Boot StartCHS    EndCHS        StartLBA     EndLBA    Sectors  Size Id Type
/dev/sdc1 *  0,0,1       527,63,32            0    1081343    1081344  528M  0 Empty
Partition 1 has different physical/logical end:
     phys=(527,63,32) logical=(67,79,12)
/dev/sdc2    1023,254,63 1023,254,63      11804      52763      40960 20.0M ef EFI (FAT-12/16/32)
Partition 2 has different physical/logical start (non-Linux?):
     phys=(1023,254,63) logical=(0,187,24)
Partition 2 has different physical/logical end:
     phys=(1023,254,63) logical=(3,72,33)

Yet when I performed fdisk -l on Laptop (Debian Stable), it looked fine:

$ sudo fdisk -l /dev/xvdi
Disk /dev/xvdi: 3.8 GiB, 4091543552 bytes, 7991296 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc8e0e6ee

Device     Boot Start     End Sectors  Size Id Type
/dev/xvdi1 *        0 1081343 1081344  528M  0 Empty
/dev/xvdi2      11804   52763   40960   20M ef EFI (FAT-12/16/32)

I tried the same procedure with another USB stick and I was able to boot the live CD. :man_shrugging:

[bobvanderlinden]

@tomfitzhenry there are quite a few variables that influence the booting process:

1. Was this on a Macbook?
2. If not Macbook, was it using UEFI or legacy bios? (Usually an option in the firmware options)
3. Did you burn the image using UNetBootin, dd or something else?

[tomfitzhenry]

1. Was this on a Macbook?

The to-be-installed NixOS machine is a https://www.pcengines.ch/apu1d.htm

2. If not Macbook, was it using UEFI or legacy bios? (Usually an option in the firmware options)

Legacy BIOS (coreboot).

3. Did you burn the image using UNetBootin, dd or something else?

I tried two times:

• cp foo.iso /dev/sda
• dd if=foo.iso of=/dev/sda bs=4M

[jemilsson]

I an having the same issue with a apu4 also using coreboot. @tomfitzhenry Did you ever find a workaround for this issue? I have verified the checksum of the image.

<<< NixOS Stage 1 >>>

loading module loop...
loading module vfat...
loading module nls_cp437...
loading module nls_iso8859-1...
loading module fuse...
loading module dm_mod...
running udev...
kbd_mode: KDSKBMODE: Inappropriate ioctl for device
starting device mapper and LVM...
mounting tmpfs on /...
waiting for device /dev/root to appear...[    5.708469] sd 2:0:0:0: [sdb] No Caching mode
[    5.713928] sd 2:0:0:0: [sdb] Assuming drive cache: write through

mounting /dev/root on /iso...
mounting /mnt-root/iso/nix-store.squashfs on /nix/.ro-store...
[    6.638677] squashfs: SQUASHFS error: unable to read id index table
mount: mounting /dev/loop0 on /mnt-root/nix/.ro-store failed: Invalid argument

An error occurred in stage 1 of the boot process, which must mount the
root filesystem on `/mnt-root' and then start stage 2.  Press one
of the following keys:

  i) to launch an interactive shell
  f) to start an interactive shell having pid 1 (needed if you want to
     start stage 2's init manually)
  r) to reboot immediately
  *) to ignore the error and continue

> 

[tomfitzhenry]

@jemilsson, per https://github.com/NixOS/nixpkgs/issues/6265#issuecomment-475928159 I tried a different USB stick and the problem went away. I didn't look into it more than that.

[jemilsson]

@tomfitzhenry Thank you for the reply. I also was able to get it to work when trying different USB-drives etc. I am not really sure what action resolved this for me.

[johnalotoski]

Just ran into this issue last night. I can reproduce this problem when trying to boot the NixOS ISO (19.09 or 20.03 beta) from a USB thumbdrive on a USB 3 port in a System76 Gazelle laptop. However, if I use the same USB thumbdrive (version 2 or 3) in a USB 2 port on the same machine, then there is no issue.