Raspberry Pi 3 Support

[fdietze]

So far, I was unable to install nix on rpi3. What I tried:

• Flash Armv7 image to sdcard and boot Pi. Kernel does not boot, missing dtb files. So I copied bcm2710-rpi-3-b.dtb and bcm2837-rpi-3-b.dtb from the official raspberry image on the sdcard.
• It boots but has a black screen.
• I copied my public key into /root/.ssh/authorized_keys, bootet pi and blindly typed systemctl start sshd. It worked! Now I had ssh access to my pi.

• I changed my configuration.nix to:

  { config, pkgs, lib, ... }:
  {
  # NixOS wants to enable GRUB by default
  boot.loader.grub.enable = false;
  # Enables the generation of /boot/extlinux/extlinux.conf
  boot.loader.generic-extlinux-compatible.enable = true;
  
  boot.kernelPackages = pkgs.linuxPackages_rpi;
  
  # Manual doesn't currently evaluate on ARM
  services.nixosManual.enable = false;
  
  nix.binaryCaches = lib.mkForce [ "http://nixos-arm.dezgeg.me/channel" ];
  nix.binaryCachePublicKeys = [ "nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%" ];
  
  # File systems configuration for using the installer's partition layout
  fileSystems = {
   "/boot" = {
     device = "/dev/disk/by-label/NIXOS_BOOT";
     fsType = "vfat";
   };
   "/" = {
     device = "/dev/disk/by-label/NIXOS_SD";
     fsType = "ext4";
   };
  };
  
  environment.systemPackages = [ pkgs.vim ];
  services.openssh.enable = true;
  users.extraUsers.root.openssh.authorizedKeys.keys =
     [ "ssh-rsa  xxxxxxxxxxxxx" ];
  }

  Then:

  nixos-rebuild switch --fast --option binary-caches http://nixos-arm.dezgeg.me/channel --option binary-cache-public-keys nixos-arm.dezgeg.me-1:xBaUKS3n17BZPKeyxL4JfbTqECsT+ysbDJz29kLFRW0=%

After success, the newly installed rpi-kernel does not boot. Any Ideas?

[bachp]

@bkchr What firmware version have you been using?

[bkchr]

@bachp the one you created.

[bachp]

@bkchr I should read more carfully. I'm probably missing the patch you mentioned. Anyway as you say it is working with the new FW I submit is as PR.

[timclassic]

I tried booting the sd-image-aarch64-linux.img from https://www.cs.helsinki.fi/u/tmtynkky/nixos-arm/installer/ on a Raspberry Pi 3B+ and it failed to boot at all. I then updated the firmware files in the /boot filesystem to the latest from https://github.com/raspberrypi/firmware/tree/master/boot and this allowed U-boot to load. However, U-Boot tried (and failed) to load "bcm283x-rpi-other.dtb" which appears to be a placeholder name to indicate an unknown model. I suspect like I'm out of luck for the moment.

However, I found a few related patches to U-Boot:

• https://lists.denx.de/pipermail/u-boot/2018-April/324722.html
• https://lists.denx.de/pipermail/u-boot/2018-April/324724.html

Is there already work underway to support the Pi 3B+ in NixOS, and I just haven't found it yet? I'm happy to open a separate issue if appropriate.

[bkchr]

@timclassic did you tried booting with the raspberry pi kernel? Upstream probably does not have the required devicetree yet.

[dezgeg]

Actually it's U-Boot which needs to be upgraded to know about RPi 3B+; I saw some patches related to that on the mailing list so hopefully it's in U-Boot 2018.05.

[bkchr]

Using https://github.com/NixOS/nixpkgs/pull/38490 and https://github.com/NixOS/nixpkgs/pull/38342, the camera module should be work out of the box. You will need the following options in your configuration:

boot.loader.raspberryPi.enable = true;
boot.loader.raspberryPi.version = 3;
boot.loader.raspberryPi.uboot.enable = true;
boot.loader.raspberryPi.uboot.startx = true;
boot.loader.raspberryPi.uboot.gpu_mem = 256;

[makefu]

@bkchr so many great hints and pointers, would you mind putting this info into either the wiki or nixos-hardware?

[bkchr]

I will try to put it into the wiki. :)

[bkchr]

@makefu I added it to the wiki :) https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi#Camera

[makefu]

@bkchr yes i already saw that via the wiki notification bot in freenode#nixos-wiki , thanks!

[bkchr]

@makefu @dezgeg someone of you could merge the two pull requests?

[makefu]

@bkchr unfortunately i am nothing more than an ordinary nixos user. @Mic92, @dezgeg pretty please? :+1:

[dezgeg]

I will look at them once I have some time.

[bachp]

For those interested in order to get the camera to show up as /dev/video0 you need to load the bcm2835-v4l2 module.

I also updated this in the wiki: https://nixos.wiki/wiki/NixOS_on_ARM/Raspberry_Pi#Camera

[bachp]

Did somebody get an xserver running on rpi3 with aarch64?

[dezgeg]

Yes it works fine (including OpenGL) with the modesetting video driver.

[bachp]

@dezgeg Are there any special settings you need to provide like cma size on the commandline or gpu_mem size in config.txt.

It might be worth to add this to the wiki too.

[dezgeg]

Right, the cma setting might be needed (in order to even get a virtual console at all). But if that's needed we should rather bump the default CMA size up in the kernel config so things work out-of-the-box. I don't think gpu_mem needs to be changed.

[bachp]

I had to set cma=128M to get the xserver running. Everything lower wouldn't work.

[timclassic]

@bkchr No, I have not tried the Raspberry Pi kernel. Do you think that it's necessary, even with the U-Boot updates I likely need?

@dezgeg I'd like to try building the latest U-Boot myself. I know that I can set up cross-compliation for aarch64 in general, and I've also found the instructions for building just U-Boot at https://nixos.wiki/wiki/NixOS_on_ARM#Building_u-boot_from_your_NixOS_PC. I'm not sure if either of these alone is enough, though. Can you point me at the process for generating the installation images you host at http://nixos-arm.dezgeg.me/installer, and I'll just start from there?

[bkchr]

No, I think then first try uboot and see what happens :)

[timclassic]

@dezgeg Feel free to ignore my question. I've been able to generate my own SD card image via an older rpi3. Hopefully I can work from this to bootstrap rpi3+ support.

[hlolli]

Useing the lastest sd-image-aarch64-linux.img from @dezgeg for rpi3+. Keep getting stuck on the rainbow splash screen (don't have the serial cable, so I rely on hdmi + keyboard). Just reading very related threads, I have good suspicion that the release v2018.05 of u-boot from earlier today https://github.com/u-boot/u-boot/releases/tag/v2018.05 solves the rpi3+ booting problems (given that I'm encountering one). I hope the images get updated in next days.

[dezgeg]

Waiting for kernel 4.18 to be released is also required, I think.

[hlolli]

@dezgeg I base my suspicion primarily on this reddit comment, and the comment under it https://www.reddit.com/r/raspberry_pi/comments/88k5bl/rpi_3b_and_uboot_problems/dwlb5u6

but no stress, my rpi3+ is completly hobby related :Þ

[bkchr]

Does someone is able to boot the raspberry pi 3 with latest master branch?

I looked into the last generation that worked for me and I think that it is either glibc 2.27 or binutils 2.30 that prevents my raspberry pi from booting.

Here is my log of the failing boot:

<<< NixOS Stage 1 >>>

loading module dm_mod...
running udev...
kbd_mode: KDSKBMODE: Inappropriate ioctl for device
Gstarting device mapper and LVM...
[    1.969164] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    1.969164] 
[    1.978476] CPU: 0 PID: 1 Comm: init Not tainted 4.16.8 #1-NixOS
[    1.984580] Hardware name: Raspberry Pi 3 Model B (DT)
[    1.989801] Call trace:
[    1.992301]  dump_backtrace+0x0/0x1c8
[    1.996025]  show_stack+0x24/0x30
[    1.999396]  dump_stack+0x9c/0xc0
[    2.002766]  panic+0x124/0x294
[    2.005872]  complete_and_exit+0x0/0x30
[    2.009771]  do_group_exit+0x40/0xa8
[    2.013406]  get_signal+0x280/0x5b0
[    2.016954]  do_signal+0x88/0x240
[    2.020325]  do_notify_resume+0xd8/0x130
[    2.024311]  work_pending+0x8/0x10
[    2.027774] SMP: stopping secondary CPUs
[    2.031763] Kernel Offset: disabled
[    2.035308] CPU features: 0x0802004
[    2.038850] Memory Limit: none
[    2.041963] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    2.041963] 
[    2.865264] random: crng init done

I also searched for the failure and some suggested that the SD card could be the problem. I have run fsck multiple times and it does not reported any big failures.

[bkchr]

I opened a pull request that fixes glibc 2.27 for raspberry pi: https://github.com/NixOS/nixpkgs/pull/40572

[qolii]

Urrrggghhhh. Is anyone willing to share a ready-to-go U-Boot for the RPi 3B+? I actually only got this board to bootstrap compiling U-Boot for my EspressoBin (since I can't for the life of me get aarch64 cross-compiling on NixOS to work for it from the guides I've seen); I saw that Raspberry Pi 3B was supported "out of the box" and thought it couldn't go wrong. Turns out the "+" is important, and now I'm stuck with two unbootable boards! I naively assumed the 3B and 3B+ were totally compatible.

I followed pretty much exactly the path of @timclassic (did you get your own bootstrapping process to work?), although I additionally tried copying the bcm2710-rpi-3-b-plus.dtb into the expected bcm283x-rpi-other.dtb, just in case. It then hangs trying to start the kernel (which may not be a surprise to anyone who knows what they are doing)

Anyhow, I'll be eternally grateful if anyone has: 1) A u-boot-rpi3.bin I can just write onto my SD card to get me going. 2) Instructions for how to cross-compile a U-Boot for AArch64 (from either AMD64 or ARMv7) 3) Some other magic.

Here's hoping!

[mestaritonttu]

@qolii did you try this for cross-compiling: https://github.com/ElvishJerricco/cross-nixos-aarch64

[timclassic]

@qolii I took the cowards way out and decided to wait until the upstream kernel supported more 3B+ features. I happen to have a couple of 3Bs that I'm using in the meantime. From reading other threads, I realized that I'd likely have network trouble next even if I got one my 3B+s to boot with the upstream kernel.

I'm sorry I don't have better news :(

[dezgeg]

Ok, for the early bird testers, here's a image based on kernel v4.18-rc1 which should have RPi 3B+ support (untested though): http://cs.helsinki.fi/u/tmtynkky/sd-image-aarch64-with-kernel-4.18-rc1.img

Note that this means using linuxPackages_testing is required with RPi 3B+ until Linux 4.18 is released and hits the channel.

Edit: As kernel v4.18 is released, there is no need for this workaround for RPi 3B+ anymore.

[qolii]

@mestaritonttu, that looks really interesting, thanks. I have not tried it, nope. But I'll look at that.

@timclassic, hehe, that's exactly what I would have done :)

@dezgeg, you are a total hero! Works perfectly. Network is great, not seen any problems yet.

[sdevmaster]

I just am not able to get audio working, does it work for anyone?! Even with new image from @dezgeg

[florianjacob]

@sdevmaster no, as far as I know still no one has managed to get analog audio from a raspberry pi on NixOS.

[bkchr]

@sdevmaster: Are you using the latest upstream kernel? If yes, I get the analog interfaces to show up in Alsa with the following commit: https://github.com/bkchr/nixpkgs/commit/3c1b35e694d3a54c8de483780e58011c8e389999 If you use the raspberry pi kernel, you probably need to add dtparam=audio=on to your /boot/config.txt.

[bkchr]

I can also confirm, that the analog output is working for me.

[jfrankenau]

Using https://github.com/ElvishJerricco/cross-nixos-aarch64 I've sucessfully compiled an SD image for my Raspberry Pi 3 which works fine so far.

I'd like to use the lirc-rpi module however adding dtoverlay=lirc-rpi to the config.txt has no effect. I've also copied the overlays folder from the raspberrypifw package to /boot but this doesn't change anything.

[makefu]

@jfrankenau you may need to build your kernel with CONFIG_LIRC=y (see https://discourse.nixos.org/t/the-correct-way-to-override-the-latest-kernel-config/533 )

[jfrankenau]

@makefu, thank you for the quick answer. This may indeed be it. At least /proc/config.gz tells me that CONFIG_LIRC is not set. I will recompile my kernel and report back.

[jfrankenau]

@makefu, I've found out that with kernel 4.16 lirc-rpi is no longer supported. Instead there is a gpio-ir overlay which also doesn't need LIRC but instead directly registers as an input device which can be configured using ir-keytable which is much simpler and quicker in my opinion than setting up a new IR remote with LIRC.

For us however there are still a few hoops to jump through. Loading overlays using dtoverlay= in /boot/config.txt has to supported by the kernel (and/or U-boot?) which isn't the case for our mainline kernel. I believe with the Raspberry Pi kernel this should work, haven't tried it however.

What I did instead was to take the base device tree bcm2837-rpi-3-b.dtb and manually apply the gpio-ir.dtbo overlay using the dtmerge tool. I then put the resulting dtb file at /boot/nixos/*-linux-4.17.8-dtbs/broadcom/ replacing the original dtb. To be able to use dtmerge though, the dtb/dtbo files need to be compiled with symbol names which in turn means that I had to use the original Raspberry Pi kernel sources and run make DTC_FLAGS=-@ ARCH=arm dtbs in order to get dtb/dtbo files which are ready to be used by dtmerge.

So, I don't know now what the progress is on having native overlay support in NixOS but if using U-Boot or configfs doesn't work out this may be an alternative.

[florianjacob]

I found out that the audio issues boil down to the same essential problems as @jfrankenau explained: dtparam=audio=on is depending on device tree overlays as well, and is therefore totally ignored by the current vanilla kernel.

The kernel patch from @bkchr essentially enables dtparam=audio=on as a default, I can confirm it works. Do you intend to upstream it as NixOS kernel patch?

[bkchr]

I'm not sure. I could upstream it to Nixos, but maybe we want to wait for upstream?

[florianjacob]

Does someone know whether dynamic device tree configuration is something that is in development for the upstream linux kernel? Could not find much information on that.

[bachp]

@florianjacob There is always discussion about it a Linux Conferences but there are some concerns about security and stability. I don't have any resources describing the status of upstream work in that direction.

[tuuzdu]

Why doesn't work bootdelay=0 in u-boot? Parameters in kernelParams = [""] don't impact on boot process too. How is possible disable interrupt on autoboot by serial input?

[bkchr]

Hey, does someone has problems with the latest kernel and wifi? Somehow my wifi is not working anymore..

[milnet2]

I have the same issue as @bkchr. After downgrading to boot.kernelPackages = pkgs.pkgs.linuxPackages_4_18; seems to work again.

[bkchr]

Yeah, I just switched one minor version back. There needs to be a regression upstream.

[florianjacob]

As the link to the audio patch is broken: The branch https://github.com/bkchr/nixpkgs/commits/raspberrypi contains the commit "Raspberry Pi audio", which is currently at https://github.com/bkchr/nixpkgs/commit/53939657a8609dfafea9e7e89aae4babedee1e03 .