search

MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.8k stars 494 forks source link

Image | NanoPi R5S #5598

Closed MichaIng closed 1 year ago

MichaIng commented 2 years ago

Creating an image request

Formal device information

Is the SBC officially supported by the Debian installer?

If not, is a reliable 3rd party Debian image available for this SBC?

If not, are there install instructions for Debian available?

Joulinar commented 2 years ago

Can't wait to see the image available 😜

MichaIng commented 2 years ago

FriendlyELEC's Debian Buster image does not contain any firmware/kernel/bootloader packages, but all related files are copied/flashed manually. Basically the procedure is done with this repository: https://github.com/friendlyarm/sd-fuse_rk3568

It uses a GPT partitioning. I'll put together the files into a DEB package and host it on our server, so it can be installed onto a single ext4 partition and updates can be easily done (by users). It will look similar like the Quartz64 firmware packages: https://dietpi.com/downloads/binaries/firmware-quartz64a.deb

MichaIng commented 2 years ago

Works so far, but room for enhancements. The image we currently offer is converted from FriendlyELEC's Debian image, freed by the overlayfs. I still don't fully understand how this setup works, all intransparently done in the unmountable set of GPT partitions, where I wasn't able to find the sources for. I used the image by @StephanStS, converted to DietPi already and removed the 9th "userdata" partition (along with further cleanup), which appeared to be empty, then did another DietPi-Installer run. I was surprised to see the R5S booting into a desktop, first thinking it booted from internal eMMC. The internal eMMC however holds FriendlyWRT, not a desktop, and further investigation showed that it was indeed the two times via DietPi-Installer converted SD card image which was booted from, however showing the original FriendlyELEC Debian desktop. I'm not 100% sure but think due to the overlayfs, all changes done on that system may have been stored on the userdata partition (not being visible when mounting it on the original image), while the actual root partition stayed untouched. So after removing the userdata partition and rebooting, the actual rootfs was mounted the first time, which was also visible in df, now showing /dev/mmcblk0p8 instead of overlay as rootfs mount source.

Previously I generated an image from scratch with a simple MBR partition table and a single ext4 filesystem, but it doesn't boot. I'm not sure whether the device tree partition provided by their GitHub repo is an actual single device tree file or not, probably I need to recompile the kernel to get it, but this is the best guess my end why it may fail to boot. Sadly there are no UART pins, so early boot debugging is difficult until I find time to hopefully find the connectors and soldering the pins manually.

One issue with a mainline U-Boot image (as I aim to create) is that the custom FriendlyELEC bootloader will be booted from with priority, i.e. if the internal eMMC isn't changed and an SD card with mainline U-Boot is attached, the board will boot from eMMC unless the mask button is pressed. One needs to flash the mainline U-Boot image onto eMMC or format it (at least the boot sector) to have the desired boot order.

@StephanStS @Joulinar Would be great if you could test it when you find time, check whether all hardware components (you are able to test) work, whether there are still unwanted files left on the image, we need to cleanup etc, doing a dietpi-benchmark 🙂.

One issue I already found, while I'm not sure how to solve best:

Joulinar commented 2 years ago

I'm not able to best in next 2 weeks as I did not take the SBC to the Baltic sea. What a pity 🤣

MichaIng commented 2 years ago

Could have been a handy remote hotspot, ad blocker and VPN relay, actually 😛. Enjoy your well deserved holidays 🙂!

Joulinar commented 2 years ago

I took my RPi3B with me running PiHole 😉

pyavitz commented 2 years ago

Just putting this out there, I don't own the board so I can't do any testing myself.

Using patches found at an OpenWRT hub that enables one to use U-Boot v2022.07 I'm able to get the board to boot, but it seems to only get as far as Starting kernel ...

If someone is interested in experimenting or playing with the u-boot and kernel patches I'm currently using they can be found in the links above.

Note: The kernel defconfig and patches are the same ones I use on the Odroid M1 and there appears to be no issue there.

mstaack commented 2 years ago

Works so far, but room for enhancements. The image we currently offer is converted from FriendlyELEC's Debian image, freed by the overlayfs. I still don't fully understand how this setup works, all intransparently done in the unmountable set of GPT partitions, where I wasn't able to find the sources for. I used the image by @StephanStS, converted to DietPi already and removed the 9th "userdata" partition (along with further cleanup), which appeared to be empty, then did another DietPi-Installer run. I was surprised to see the R5S booting into a desktop, first thinking it booted from internal eMMC. The internal eMMC however holds FriendlyWRT, not a desktop, and further investigation showed that it was indeed the two times via DietPi-Installer converted SD card image which was booted from, however showing the original FriendlyELEC Debian desktop. I'm not 100% sure but think due to the overlayfs, all changes done on that system may have been stored on the userdata partition (not being visible when mounting it on the original image), while the actual root partition stayed untouched. So after removing the userdata partition and rebooting, the actual rootfs was mounted the first time, which was also visible in df, now showing /dev/mmcblk0p8 instead of overlay as rootfs mount source.

Previously I generated an image from scratch with a simple MBR partition table and a single ext4 filesystem, but it doesn't boot. I'm not sure whether the device tree partition provided by their GitHub repo is an actual single device tree file or not, probably I need to recompile the kernel to get it, but this is the best guess my end why it may fail to boot. Sadly there are no UART pins, so early boot debugging is difficult until I find time to hopefully find the connectors and soldering the pins manually.

One issue with a mainline U-Boot image (as I aim to create) is that the custom FriendlyELEC bootloader will be booted from with priority, i.e. if the internal eMMC isn't changed and an SD card with mainline U-Boot is attached, the board will boot from eMMC unless the mask button is pressed. One needs to flash the mainline U-Boot image onto eMMC or format it (at least the boot sector) to have the desired boot order.

@StephanStS @Joulinar Would be great if you could test it when you find time, check whether all hardware components (you are able to test) work, whether there are still unwanted files left on the image, we need to cleanup etc, doing a dietpi-benchmark 🙂.

One issue I already found, while I'm not sure how to solve best:

  • Network setup at first boot only works with the WAN port (eth0).
  • The problem is that, as long as the interface is no set up, there is no way do know whether a cable is actually connected or not (no carrier signal).
  • In this case, all three ports appear identical for the system, so the first interface eth0 is tried to be configured. This is actually the same issue with all boards with multiple Ethernet ports.
  • The only solution I can think of is to loop through all interfaces at first boot and do ip l s up dev eth0/1/2. Giving it a second or so, then checking back via ip -br l l dev eth1 whether NO-CARRIER is shown (no cable connected) or not.

the nanopi-r5s has uart pins, using them already

next to the usbc power connector: image

but i recommend to use the ones on the other side, so you dont have to remove the board: image

baudrate is a little different: tio /dev/tty.usbserial-0001 -b 1500000

MichaIng commented 2 years ago

Great, thanks for sharing. Clever to access them from the bottom 👍 .

StephanStS commented 2 years ago

I added a Samsung M.2 NVMe disk shown in lsblk: image

Formatted via dietpi-drive_manager: image

The device works fine.
With this an easy net NVMe storage would be an option if one has a spare NVMe disk lying around.

Benchmark via dietpi-config:

MichaIng commented 2 years ago

Awesome, many thanks for testing.

pyavitz commented 2 years ago

With a lot of trial and error and @mstaack helping me out on the boot front, we've now got the board up and running using Mainline U-Boot v2022.07 and Linux 5.19. It's of course not perfect and still needs a lot of work, but I thought @MichaIng and whom ever else, might wanna take a swing at trying to squash whatever bugs and what not there are. And there are plenty :)

The credentials npi:npi and images

mstaack commented 2 years ago

did a geekbench aarch64 preview run: https://browser.geekbench.com/v5/cpu/16451425

MichaIng commented 2 years ago

For all features to work, I'd currently prefer to stay with FriendlyELEC's kernel, but probably the device tree from your build works with it. I'm not able to extract it from FriendlyELEC's image, and so far was too lazy to build the kernel from their sources.

Ah, but I learned this one, which I'll try first:

dtc -I fs -O dts /sys/firmware/devicetree/base -o nanopi5.dtb

But the mainline image looks pretty good. Probably we can do some comparison about which hardware feature works on mainline already.

pyavitz commented 2 years ago

But the mainline image looks pretty good. Probably we can do some comparison about which hardware feature works on mainline already.

As far as I know, the only major downer using the mainline kernel is that the PCIe M.2 doesn't work. pcieport 0000:00:00.0: of_irq_parse_pci: failed with rc=-22 Beyond that I'm still having trouble finding a trigger for the eth port LEDs. I some what have a theory on how to get them to work right, but not having the board my self makes it tedious to test.

And no, they can't be setup exactly the same as the vendor kernel.

As for the PCIe, there is still several patches awaiting approval that might remedy the issue. Currently waiting for v5 of the patch set to be posted.

MichaIng commented 2 years ago

See here for getting Ethernet LED triggers t work: #5679

Sadly with the manually generated device tree, the manually generated image with single partition but FriendlyELEC kernel + bootloader still does not boot. Probably all needs to be compiled manually to work without EFI partition table and the 8 partitions but with a single MBR ext4 and extlinux instead.

Interesting, WireGuard support has been actually enabled, but the module is not contained in the latest FriendlyELEC Debian image build, probably on FriendlyWRT only 🤔: https://github.com/friendlyarm/kernel-rockchip/commit/ba2b80a

As a general resource:

pyavitz commented 2 years ago

I tried using vendor u-boot with a single partition and extlinux and could never get a successful boot, beyond u-boot. U-boot seemed functional, it would just never find the boot / root partition. Hence the reason I ended up going the route I did. If you review the defconfig wireguard isn't ticked on.

MichaIng commented 2 years ago

Matches my results so far: It denies to boot without the original GPT partitioning. Will try to check and set the U-Boot configs with fw_printenv/fw_setenv, maybe the defaults are hardcoded so limited in the vendor U-Boot build.

MichaIng commented 1 year ago

The multiple partitions are mentioned here: https://github.com/friendlyarm/uboot-rockchip/blob/nanopi5-v2017.09/include/configs/nanopi5.h

R5S support was added with this commit: https://github.com/friendlyarm/uboot-rockchip/commit/8c91d15

In the config:

# CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_USE_SECTOR is not set
CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_USE_PARTITION=y
...
# CONFIG_ISO_PARTITION is not set

Not sure whether this is what enforces partitioning?

MichaIng commented 1 year ago

@pyavitz I tried to flash the U-Boot of your recent Debian Bullseye onto my debootstrapped R5S image with single ext4 partition, vendor kerne/dtb and /boot/extlinux/extlinux.conf, via:

dd 'if=/root/idbloader.bin' "of=$1" seek=64 conv=fdatasync
dd 'if=/root/u-boot.itb' "of=$1" seek=16384 conv=fdatasync

But it does not boot (red LED remains lit instead of blinking), any idea why it might not work?

Was flashed to internal eMMC, with and without SD card attached.

pyavitz commented 1 year ago

Is this isolated to the eMMC or can you not boot from SD either?

Earlier today, someone was running tests on the imgs and was also having trouble flashing them to the eMMC.

He kept getting this:

unable to select a mode : -70
device_remove: Device 'mmc@fe310000.blk' failed to remove, but children are gone

Which I've personally never seen before and no one had yet to report the error to me. After some trial and error he reported back and said he got it to boot from eMMC. Unfortunately that error was still visible in the logs and I suspect its got something to do with the freq being used (speed in which u-boot is suggesting the eMMC should operate), but I haven't had time to really look into it.

Did you try compiling vendor u-boot with that bit in the defconfig ticked off?

pyavitz commented 1 year ago

The multiple partitions are mentioned here: https://github.com/friendlyarm/uboot-rockchip/blob/nanopi5-v2017.09/include/configs/nanopi5.h

R5S support was added with this commit: friendlyarm/uboot-rockchip@8c91d15

In the config:

# CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_USE_SECTOR is not set
CONFIG_SYS_MMCSD_RAW_MODE_U_BOOT_USE_PARTITION=y
...
# CONFIG_ISO_PARTITION is not set

Not sure whether this is what enforces partitioning?

Not sure if this is the standard for all RK356Xs "using vendor uboot", but it requires GPT along with a partition labelled uboot which is where the bin gets flashed. It looks to me, the above is related to that the uboot partition. The question then becomes, can it be disabled and the norms ticked on? Not sure. I can tell you it compiles if we do so.

MichaIng commented 1 year ago

I'll try to boot with your U-Boot build and vendor kernel. When I find time I'll also try to play with the vendor U-Boot configs and compiling them.

pradheeshrj commented 1 year ago

I'll try to boot with your U-Boot build and vendor kernel. When I find time I'll also try to play with the vendor U-Boot configs and compiling them.

Can I get the uboot configs and DTSI File?

HeyMeco commented 1 year ago

The people at Armbian also made progress regarding U-Boot and building newer kernels for the R5S: https://github.com/armbian/build/pull/4247

Averell7 commented 1 year ago

I am using the build of pyavitz mentioned here : https://forum.armbian.com/topic/21211-nanopi-r5s-armbian-image/ And it works really well. I used it from SC card. I tried to load it to eMMC, and it does not start, but I cannot be sure if the image I made was correct. It is not possible to do a new test, because I have no way to access the eMMC now. The USB load which should solve the problem fails. Unfortunately the use of u-boot has some draw backs due to the very special way this device is handling the precedence for eMMC boot or SD boot. If it was possible to use the Rockchip MiniLoader (used by FriendlyElec distri) it would be much better. See here for details : https://wiki.friendlyelec.com/wiki/index.php/NanoPi_R5S#The_Boot_order_between_eMMC_and_SD_card.

Oorweeg commented 1 year ago

Don’t understand any of this enough to know if this is helpful or not to translate over to Debian/Ubuntu, but this guy is building OpenWRT for the Nanopi R5S with kernel 5.19, 6.0+ and seemingly u-boot 22.10?

MichaIng commented 1 year ago

I'll mark this as closed. Our recent image ships with WireGuard module, so the most urgent need has been solved. Depending on how soon an Armbian image is available, I'll however come back to pyavitz's mainline kernel builds.

darkgeek commented 7 months ago

Is it possible to boot from nvme? Or boot from sdcard but rootfs on nvme? I tried to alter the /etc/fstab on the rootfs mount to nvme drive, but after reboot it still booted to sdcard, even if I added bootdev=/dev/nvme0n1p8 to dietpiEnv.txt (this file does not exist under /boot on the latest image, so I created that).

MichaIng commented 7 months ago

Currently it is not possible, since the U-Boot environment which defines the rootfs is hardcoded on one of the other partitions where it cannot (easily) be edited, if at all. But I am working on a transition to Armbian's mainline based kernel and bootloader, after which this will work: https://github.com/MichaIng/DietPi/pull/6902

Direct boot from NVMe still won't be possible, as the R5S lacks an SPI flash where a bootloader could be put on. So it requires at least the bootloader and boot scripts/configs on an SD card or eMMC, from where you can tell the kernel to take any other partition as rootfs.

HeyMeco commented 7 months ago

But I am working on a transition to Armbian's mainline based kernel and bootloader, after which this will work: #6902

Just to avoid confusion with armbian "mainline" you actually mean their rockchip legacy branch and not the actual linux mainline kernel right? As moving to linux mainline would mean the loss of MPP and such.

MichaIng commented 7 months ago

Nope, for R5S/R5C I indeed mean the "current" mainline-based branch. RK3568 has pretty good mainline support already, no need to stick with vendor kernel sources.

As moving to linux mainline would mean the loss of MPP and such.

Are you sure this is still true with recent mainline kernel? And if so, which applications are effected in particular, which cannot use other video codec interfaces?

HeyMeco commented 7 months ago

Are you sure this is still true with recent mainline kernel? And if so, which applications are effected in particular, which cannot use other video codec interfaces?

Yes I keep a very close look on this and the people which are responsible for media on the linux kernel are expecting code quality that isn‘t like the current state of the VPU acceleration driver from RK. This would for example affect Jellyfin (technically that feature will be added in 10.9 but there’s a preview and the dev @ nyanmisaka also has merged many changes already).

I assume legacy will become rk6.1 too so at least the option to choose between vendor and mainline kernel should be there in the final dietpi update/image.

MichaIng commented 7 months ago

To be true, dealing any longer with vendor kernel specifics and even offer a switch and support both cases concurrently, is above what we are able to maintain. If MPP is not even supported in current Jellyfin, then I guess it is not such a huge must-have feature, and there are other accelerated codecs/interfaces (right?), even when they are not as fast. So I will switch to mainline Linux. Of course the legacy kernel can be manually installed afterwards. It might just break Ethernet LEDs, as the sysfs nodes might have different names in both kernel variants, but that is a minor issue and of course can be fixed manually as well. If there is really a significant number of people finally missing MPP or other things from legacy kernel, we might add a little documentation block about how to switch to legacy and rename the Ethernet LED nodes in the related udev rule.

darkgeek commented 7 months ago

Currently it is not possible, since the U-Boot environment which defines the rootfs is hardcoded on one of the other partitions where it cannot (easily) be edited, if at all. But I am working on a transition to Armbian's mainline based kernel and bootloader, after which this will work: #6902

Direct boot from NVMe still won't be possible, as the R5S lacks an SPI flash where a bootloader could be put on. So it requires at least the bootloader and boot scripts/configs on an SD card or eMMC, from where you can tell the kernel to take any other partition as rootfs.

Will there be a beta image with mainline conponents recently? I'm happy to test it if available.😄

MichaIng commented 7 months ago

It is already there: https://dietpi.com/downloads/images/testing/

Tchoupinax commented 6 months ago

Hello, I came here because I had the same question "How can I boot directly from NVMe disk?". Understanding the issue, I gave a look to tell the internal EMMC to redirect the rootfs to the NVMe and I found this repository. There is an easy instruction to install a "bootloader". At the moment the DietPi installation is broken probably due to my fault but it has booted on NVMe!