Raspberry Pi 3 Support

[fdietze]

So far, I was unable to install nix on rpi3. What I tried:

• Flash Armv7 image to sdcard and boot Pi. Kernel does not boot, missing dtb files. So I copied bcm2710-rpi-3-b.dtb and bcm2837-rpi-3-b.dtb from the official raspberry image on the sdcard.
• It boots but has a black screen.
• I copied my public key into /root/.ssh/authorized_keys, bootet pi and blindly typed systemctl start sshd. It worked! Now I had ssh access to my pi.

• I changed my configuration.nix to:

  { config, pkgs, lib, ... }:
  {
  # NixOS wants to enable GRUB by default
  boot.loader.grub.enable = false;
  # Enables the generation of /boot/extlinux/extlinux.conf
  boot.loader.generic-extlinux-compatible.enable = true;
  
  boot.kernelPackages = pkgs.linuxPackages_rpi;
  
  # Manual doesn't currently evaluate on ARM
  services.nixosManual.enable = false;
  
  nix.binaryCaches = lib.mkForce [ "http://nixos-arm.dezgeg.me/channel" ];
  nix.binaryCachePublicKeys = [ "nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%" ];
  
  # File systems configuration for using the installer's partition layout
  fileSystems = {
   "/boot" = {
     device = "/dev/disk/by-label/NIXOS_BOOT";
     fsType = "vfat";
   };
   "/" = {
     device = "/dev/disk/by-label/NIXOS_SD";
     fsType = "ext4";
   };
  };
  
  environment.systemPackages = [ pkgs.vim ];
  services.openssh.enable = true;
  users.extraUsers.root.openssh.authorizedKeys.keys =
     [ "ssh-rsa  xxxxxxxxxxxxx" ];
  }

  Then:

  nixos-rebuild switch --fast --option binary-caches http://nixos-arm.dezgeg.me/channel --option binary-cache-public-keys nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%

After success, the newly installed rpi-kernel does not boot. Any Ideas?

[Mic92]

Is this related to https://github.com/NixOS/nixpkgs/pull/21988 ?

[wizeman]

@Mic92: indeed.

@fdietze: at least for me, my Raspberry Pi 3 did not boot until I made the modifications in #21988. Note that those changes haven't been merged into the master branch yet (much less made it onto a stable release).

Apart from those changes, you will also need to disable the generic-extlinux-compatible boot loader, and enable the raspberryPi boot loader instead.

The linuxPackages_rpi kernel is also needed, but you already have that in your configuration.

Although I'm not completely sure, another thing which is probably needed for the Raspberry Pi 2 or 3, but is not usually mentioned anywhere, is that you need to have the following text in your /boot/config.txt:

kernel=kernel7.img
initramfs initrd

Although if someone could confirm whether the above point is really needed or not, that would be great.

[dezgeg]

I am waiting for this kernel patch: https://lkml.org/lkml/2017/1/19/542 to go in to make sd-image-armv7l-linux.img boot out-of-the-box.

[wizeman]

@dezgeg As far as I can see, that only contains the Raspberry Pi 3's DTB for the arm64 architecture, not the arm32 that we are using.

IIRC, there was a similar patch a few months ago which also added the RPi3 DTB for arm32, but there was some pushback, both because of duplicate code, but also because the kernel developers seemed to prefer to support only 64-bit kernels (if the hardware supports it).

Last time I checked, NixOS still doesn't support the arm64/aarch64 architecture, so if I'm not mistaken, the patch you mentioned wouldn't be very useful currently... :disappointed:

[wizeman]

Ah wait, maybe I've misread the patch. It seems that it adds the 32-bit DTB, so indeed it would be very useful. Thanks for keeping track of this, @dezgeg!

[dezgeg]

No, that kernel patch is for 32-bit ARM. The DT is just identical between 32-bit and 64-bit (and it already exists under arch/arm64). It's true that the arm-soc maintainers might still not want that, let's hope they do.

Aarch64 support is implemented here: #22117

[FPtje]

I can confirm the black screen on boot issue on the Raspberry pi 3. U-Boot shows fine, but the screen goes blank when the kernel starts loading. The exact same SD card would boot fine (with HDMI working) on a Raspberry pi 2.

It's a minor problem to me personally, since we don't use the HDMI port anyway.

Also, I've got a Linux kernel override that includes the DTB patch. For those who want to build their own RPI3 image, override the kernel like this:

{
    linux_4_8 = pkgs.linux_4_8.override {
      kernelPatches = [
        { patch = ./rpi3-dtb.patch; name = "rpi3-dtb"; }
      ];
    };
}

rpi3-dtb.patch containing of course this patch.

In other news, the Linux kernel for the raspberry pi is very limited. Things like firewalls, TUN, iptables stuff and I2C don't work out of the box because of missing kernel configs. Is this intentional? I've enabled a bunch of these things in my linux_4_8 override, using the extraConfig field, but I could imagine that it could be desirable to have NixOS support most of the same stuff Raspbian does.

[dezgeg]

FWIW, when I tried RPi3 on Aarch64 the HDMI worked for me. Maybe it got fixed/enabled in kernel 4.9.

For the kernel config question, it's because sometimes (not always) enabling the kernelAutoModules option for the config generator (which tries to build anything that's not already enabled as a module) makes the kernel crash on boot. Really weird problem. But do tell what's missing and I can explicitly enable it.

[FPtje]

My entire override can be found here. There's I2C stuff, which might be useful for tinkering. The INET, IP[6]_..., NETFILTER_, NF_ stuff is for the firewall (and for our VPN).

Many things (like the PPP stuff) are probably not needed. The default Raspbian stuff can be found here, scattered in Kconfig files.

[Mic92]

@fdietze #21988 was merged now. Does nixos now work for you on rpi3?

[dezgeg]

That one is not related to OP's config, as it uses boot.loader.generic-extlinux-compatible.enable = true;

However, this might help:

nixpkgs.config = {
    platform = pkgs.platforms.raspberrypi2;
  };

[fdietze]

Sorry for the silence, didn't have time yet to try again. I'll report back when I do.

[fdietze]

So I tried it again:

• downloaded latest armv7 image, still had to copy missing dtb files to boot partition
• It still boots to a black screen.

• copied my public key into /root/.ssh/authorized_keys, bootet pi, blindly typed systemctl start sshd I tried to include the suggestions from this thread into my configuration.nix:

  { config, pkgs, lib, ... }:
  {
  
  # NixOS wants to enable GRUB by default
  boot.loader.grub.enable = false;
  # Enables the generation of /boot/extlinux/extlinux.conf
  boot.loader.generic-extlinux-compatible.enable = false;
  boot.loader.raspberryPi.enable = true;
  boot.loader.raspberryPi.version = 3;
  boot.extraKernelParams = [ "nomodeset" ]; # doesn't have any effect
  
  boot.kernelPackages = pkgs.linuxPackages_rpi;
  
  nixpkgs.config = {
  #platform = pkgs.platforms.raspberrypi2; # would produce infinite recursion?!
  allowUnfree = true;
  };
  
  # Manual doesn't currently evaluate on ARM
  services.nixosManual.enable = false;
  
  nix.binaryCaches = lib.mkForce [ "http://nixos-arm.dezgeg.me/channel" ];
  nix.binaryCachePublicKeys = [ "nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%" ];
  
  # File systems configuration for using the installer's partition layout
  fileSystems = {
   "/boot" = {
     device = "/dev/disk/by-label/NIXOS_BOOT";
     fsType = "vfat";
   };
   "/" = {
     device = "/dev/disk/by-label/NIXOS_SD";
     fsType = "ext4";
   };
  };
  
  environment.systemPackages = [ pkgs.vim ];
  services.sshd.enable = true;
  users.extraUsers.root.openssh.authorizedKeys.keys =
     [ "ssh-rsa ..." ];
  }

then run:

nixos-rebuild switch --fast --option binary-caches http://nixos-arm.dezgeg.me/channel --option binary-cache-public-keys nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%

and rebooted.

Still black screen after reboot and ssh service does not start automatically (I can still blindly start it via keyboard). How can I know if it worked? Any suggestions?

[fdietze]

It seems to work with a newer kernel:

boot.kernelPackages = pkgs.linuxPackages_4_10;

At least the ssh service is started automatically now.

[fdietze]

Does anyone know why my Pi gets a new MAC-address on every boot?

[dezgeg]

I noticed it as well. I tracked down into an bug in either U-Boot or the mainline DTBs. (Basically U-Boot currently doesn't recognize aliases { ethernet = <&whatever>; }, it wants the alias name to be ethernet0 or similar.)

[dezgeg]

BTW, now that we have aarch64 builds on the official Hydra, I recommend the mainline kernel on aarch64 even more on the RPi3. The images aren't currently being built by Hydra but here's a fairly recent one: https://www.cs.helsinki.fi/u/tmtynkky/sd-image-aarch64-linux.img

Still, for the 32-bit port, there has been some activity from the bcm2835 maintainer in February: https://www.spinics.net/lists/arm-kernel/msg561622.html, so I expect that to land in the 4.12 kernel...

[fdietze]

The aarch64 image works way better. It boots correctly without any hacks and the default binary cache works well. My problem now is that nixos-rebuild switch fails, because there is no space left on /boot. I tried resizing it using gparted, but that didn't work. Any advice on resizing the boot partition?

[FPtje]

My RPI boot partition always fills up when I deploy new versions of the kernel often enough. New boot files are put into /boot when a new kernel is installed. The solution is to perform a sudo nix-collect-garbage -d. WARNING: this will remove all previous generations of NixOS, so you won't be able to boot to an earlier version of the system. A nixos-rebuild switch after collecting garbage should clear the redundant /boot files, clearing up space.

In my personal case I can't switch even after collecting garbage. The message about not enough space being left is thrown before it actually cleans /boot AFAIK. I've had to delete files from /boot before. Be very careful with that, though, since you can render your system unbootable.

[vcunat]

For full /boot see https://github.com/NixOS/nixpkgs/issues/23926

[dezgeg]

In retrospect, actually mounting the FAT partition at /boot was a bad idea... U-Boot can load the kernel+initrd+etc. just fine from the ext4 partition. To do that:

• umount /boot
• comment out the fileSystems."/boot" entry from configuration.nix
• use fdisk to remove the bootable flag from the FAT32 partition, and set it for the ext4 partition
• nixos-rebuild switch

[ghost]

Tuomas Tynkkynen notifications@github.com writes:

BTW, now that we have aarch64 builds on the official Hydra, I recommend the mainline kernel on aarch64 even more on the RPi3.

great, thank you!

The images aren't currently being built by Hydra but here's a fairly recent one: https://www.cs.helsinki.fi/u/tmtynkky/sd-image-aarch64-linux.img

I copied the img to sd card and booted. It hangs after the message about loading the kernel. Am I supposed to do something special for it to start nixos?

[dezgeg]

It certainly works for me, though it certainly stays there for several seconds. Maybe you have a slow(er) card.

[rickynils]

@tohl I got the behavior you're describing before switching out the power adapter for a more powerful one.

[ghost]

that did the trick, it works now, thank you

[ghost]

Is there an example configuration.nix somewhere?

The one in the sd-image just imports the sd-image expression, but it is not clear to me, how would I go from there to my own configuration? E.g. nixos-hardware repo does show only sample config for rpi2 but not for rpi3. Simply trying to enable ssh with that default import does not work, for example.

[dezgeg]

Use one from section "NixOS installation & configuration" from https://nixos.org/wiki/NixOS_on_ARM. Don't pick linuxPackages_rpi but rather linuxPackages_latest.

[ghost]

Use one from section "NixOS installation & configuration" from https://nixos.org/wiki/NixOS_on_ARM. Don't pick linuxPackages_rpi but rather linuxPackages_latest.

yes, that worked, thank you

also, the custom binary cache configuration can be ignored for aarch64

[ghost]

it seems to work very well, except firewall doesnt work:

firewall = {
  enable = true;
  allowPing = true;
  allowedTCPPorts = [
    22 # sshd
  ];
};

Apr 22 16:59:02 nixos systemd[1]: Starting Firewall... Apr 22 16:59:02 nixos firewall-start[12601]: iptables: No chain/target/match by that name. Apr 22 16:59:02 nixos systemd[1]: firewall.service: Main process exited, code=exited, status=1/FAILURE Apr 22 16:59:02 nixos systemd[1]: Failed to start Firewall. Apr 22 16:59:02 nixos systemd[1]: firewall.service: Unit entered failed state. Apr 22 16:59:02 nixos systemd[1]: firewall.service: Failed with result 'exit-code'.

is that expected?

[dezgeg]

Did you try upgrading channel? I think it's fixed in the latest kernel builds.

[ghost]

Did you try upgrading channel? I think it's fixed in the latest kernel builds.

How can I upgrade the channel?

There isn't seem to be any by default and when I add one, it cannot unpack the channels.

Tomas

===

[root@nixos:~]# nix-channel --list

[root@nixos:~]# nix-channel --add https://nixos.org/channels/nixos-unstable nixos

[root@nixos:~]# nix-channel --list nixos https://nixos.org/channels/nixos-unstable

[root@nixos:~]# nix-channel --update downloading Nix expressions from ‘https://d3g5gsiof5omrk.cloudfront.net/nixos/unstable/nixos-17.09pre105622.f0fac3b578/nixexprs.tar.xz’... downloading ‘https://d3g5gsiof5omrk.cloudfront.net/nixos/unstable/nixos-17.09pre105622.f0fac3b578/nixexprs.tar.xz’... [8009/8486 KiB, 1136.6 KiB/s] unpacking channels... builder for ‘/nix/store/s8kvq42vx1dxl8d3n5kcsymipw2ahig0-user-environment.drv’ failed to produce output path ‘/nix/store/9vgcjy5xgn2yjclp66k4kpad99npm11c-user-environment’ error: build of ‘/nix/store/s8kvq42vx1dxl8d3n5kcsymipw2ahig0-user-environment.drv’ failed cannot unpack the channels at /run/current-system/sw/bin/nix-channel line 169.

[root@nixos:~]# nixos-rebuild switch --upgrade downloading Nix expressions from ‘https://d3g5gsiof5omrk.cloudfront.net/nixos/unstable/nixos-17.09pre105622.f0fac3b578/nixexprs.tar.xz’... downloading ‘https://d3g5gsiof5omrk.cloudfront.net/nixos/unstable/nixos-17.09pre105622.f0fac3b578/nixexprs.tar.xz’... [8006/8486 KiB, 1136.5 KiB/s] unpacking channels... builder for ‘/nix/store/s8kvq42vx1dxl8d3n5kcsymipw2ahig0-user-environment.drv’ failed to produce output path ‘/nix/store/9vgcjy5xgn2yjclp66k4kpad99npm11c-user-environment’ error: build of ‘/nix/store/s8kvq42vx1dxl8d3n5kcsymipw2ahig0-user-environment.drv’ failed cannot unpack the channels at /run/current-system/sw/bin/nix-channel line 169.

[dezgeg]

I can't replicate that problem. Maybe you had something getting corrupted on the partition when the boot crashed due to the power problem.

Of course, building from a git checkout (-I nixpkgs=/path/to/nixpkgs) should also work.

[purefn]

The wiki has been shutdown. Will someone put up an ARM page on the https://github.com/nixos-users wiki, with example configuration.nix?

[Mic92]

you can.

[makefu]

@purefn Current status is that the aarch64 image works pretty much out of the box (except for wifi). I've documented this in the unofficial nixos wiki at https://nixos.wiki/wiki/NixOS_on_ARM#Raspberry_Pi_3 .

I am running nixos-unstable (2017-10-08).

[fdietze]

Same for me! Things work out of the box (using the configuration.nix from the wiki article). I think this can be closed.

[mogria]

I'm running the AArch64 image as described on the unoffical wiki. Does anyone have the Audio working via the jack port?

I tried putting dtparam=audio=on in config.txt and copying the .dtb files from the raspberrypifw package into /boot. But that doesn't seem to have any effect.

No sound cards are detected. I've also tried manually loading the snd_bcm2385 module, but with no effect at all. I tried to debug using the tools in raspberrypifw but they don't seem to work as of now (See https://github.com/NixOS/nixpkgs/issues/30026).

[florianjacob]

@mogria same problem here ☹️ also tried audio_pwm_mode=2 and kernel 4.14. This gentoo image claims they have it working, though.

[ghost]

I have had no luck with audio as well.

The main problem I am having is hard crashing on u-boot initializing the OS. I have had this with multiple re-images of a few revisions of dezgeg's aarch64 image, I am not sure how to troubleshoot this really.

Seems like a memory issue possibly & occasionally the tty seems to crash to the U-boot init text, but the Alt-Fn shortcuts allow switching back to a functioning tty.

[iblech]

@hcmensch: Are you sure that there are no problems with the power supply? The phenomenon you describe is precisely what is stated at the bottom of https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi about problems with the power supply.

[ghost]

I started off with bad power supplies, then upgraded to a correct spec supply from one of the official RPI re-sellers after reading about the potential problems. I think I will try to test the supply at my university's ee lab and get a different one in case I received a fake or faulty unit.

[garbas]

could we add the rpi3 hardware configuration here -> https://github.com/NixOS/nixos-hardware/tree/master/raspberry-pi?

[bachp]

I was also playing with rpi3 and an aarch64 image. Almost everything seems to work out of the box.

I now to updated the raspberrypifw packet to see if I can get the camera working, but the change in firmware doesn't seem to be picked up.

After some analysis I figured found the options:

boot.loader.raspberryPi.enable = true;
boot.loader.raspberryPi.version = 3;

and from looking at the source of the module I think it should copy the new firmware to the /boot partition. But it doesn't.

Does anybody know how the rpi3 firmware loading is supposed to work?

[bachp]

It turns out I had boot.loader.generic-extlinux-compatible.enable = true; set which prevents boot.loader.raspberryPi.enable = true; from taking effect.

After disabling boot.loader.generic-extlinux-compatible the new firmware files were copied to /boot but the system didn't boot anymore as the bootloader entry didn't update.

I did some additional digging and found out the the initial firmware comes from sd-image-aarch64.nix. It seems to be copied there when the image is initially created but I couldn't find any mechanism that would allow to update the firmware on a running NixOS.

Is this is just not implemented yet or am I missing something?

[dezgeg]

@bachp Yes, the answer currently is essentially 'not implemented yet'. To upgrade the firmware, replace these files in /boot with new versions: bootcode.bin fixup_cd.dat fixup.dat fixup_db.dat fixup_x.dat start_cd.elf start_db.elf start.elf start_x.elf

[bachp]

@dezgeg Ok thanks for the info. Do you already have something in mind how this could be done? I might find some time to work on this.

[florianjacob]

With the firmware upgrade instructions, I tried again to get analog audio to work with 18.03 pre, 4.15.12 kernel, manually downloaded firmware 20180328, dtparam=audio=on, modprobe snd_bcm2835 but no luck. :frowning_face:

(To people wondering why someone would want to use the analog audio of a RaspberryPi 3: It's not as bad as it used to be, and especially with audio_pwm_mode=2 the noise nterference is gone. :wink: )

[bachp]

@florianjacob I was trying to get the camera working but no luck so far too. I also tried with the latest firmware 20180328.

Did anybody manage to get the camera working under NixOS?

[bkchr]

So, I got the camera working.

The following patch is required: https://patchwork.kernel.org/patch/9626797/

And you need to add the following to /boot/config.txt:

gpu_mem=128
start_x=1

[florianjacob]

No sound cards are detected. I've also tried manually loading the snd_bcm2385 module, but with no effect at all. I tried to debug using the tools in raspberrypifw but they don't seem to work as of now (See #30026).

@mogria now that #30026 seems to be mostly resolved, could you name which tool(s) you intended to use for debugging the sound issues? I'm having trouble to find something useful in that area.