search

ayufan-rock64 / linux-build

Rock64 Linux build scripts, tools and instructions
MIT License
561 stars 98 forks source link

Mainline u-boot problem: focal-minimal-rock64-0.10.12-1184-arm64 fails to boot on 1GB Rock64 v2.0 #428

Open ReimuHakurei opened 4 years ago

ReimuHakurei commented 4 years ago

This seems to be more of a bug in the mainline u-boot portion, but there doesn't seem to be an issue tracker on that repo.

I get the following on serial console when I attempt to boot:

U-Boot TPL 2020.01-ayufan-2013-gdb96b24f (Apr 13 2020 - 08:43:36)
data training error
col error
data training error
LPDDR3, 800MHz
BW=16 Col=12 Bk=8 CS0 Row=16 CS=1 Die BW=8 Size=4096MB

It hangs there forever. Note that it says 4GB of memory and 800 MHz, there, but this is a 1GB model. Related?

For reference, here's the bootloader portion of the latest stable build, which boots fine. Notice it shows 786 MHz and 1GB, not 800 MHz and 4GB. I'm guessing this is the root of the problem, but I'm not familiar enough with u-boot to troubleshoot further.

DDR version 1.13 20180428
ID:0x805 Y
In
LPDDR3
786MHz
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=32 Size=1024MB
ddrconfig:1
OUT

U-Boot SPL 2017.09-rockchip-ayufan-1065-g95f6152134 (Aug 26 2019 - 12:40:32)
board_init_sdmmc_pwr_en
setup_ddr_param  1
booted from SD
Trying to boot from MMC2
NOTICE:  BL31: v1.3(debug):9d3f591
NOTICE:  BL31: Built : 14:39:02, Jan 17 2018
NOTICE:  BL31:Rockchip release version: v1.3
INFO:    ARM GICv2 driver initialized
INFO:    Using opteed sec cpu_context!
INFO:    boot cpu mask: 1
INFO:    plat_rockchip_pmu_init: pd status 0xe
INFO:    BL31: Initializing runtime services
WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
ERROR:   Error initializing runtime service opteed_fast
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x200000
INFO:    SPSR = 0x3c9

U-Boot 2017.09-rockchip-ayufan-1065-g95f6152134 (Aug 26 2019 - 12:41:04 +0000)

Model: Pine64 Rock64
DRAM:  1022 MiB
MMC:   rksdmmc@ff520000: 0, rksdmmc@ff500000: 1
SF: Detected gd25q128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB
*** Warning - bad CRC, using default environment

In:    serial@ff130000
Out:   serial@ff130000
Err:   serial@ff130000
Model: Pine64 Rock64
misc_init_r
cpuid=55524b50303930343200000000021c29
serial=9935ded23386afc
Net:   eth0: ethernet@ff540000
Hit any key to stop autoboot:  0
Card did not respond to voltage select!
mmc_init: -95, time 10
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:7...
Found /boot/extlinux/extlinux.conf
Retrieving file: /boot/extlinux/extlinux.conf
1055 bytes read in 21 ms (48.8 KiB/s)
select kernel
1:      kernel-4.4.190-1233-rockchip-ayufan-gd3f1be0ed310
2:      kernel-4.4.190-1233-rockchip-ayufan-gd3f1be0ed310-memtest
Enter choice: 1:        kernel-4.4.190-1233-rockchip-ayufan-gd3f1be0ed310
Retrieving file: /boot/initrd.img-4.4.190-1233-rockchip-ayufan-gd3f1be0ed310
6335322 bytes read in 309 ms (19.6 MiB/s)
Retrieving file: /boot/vmlinuz-4.4.190-1233-rockchip-ayufan-gd3f1be0ed310
20545544 bytes read in 953 ms (20.6 MiB/s)
append: rw panic=10 init=/sbin/init coherent_pool=1M ethaddr=ba:53:8f:68:92:c4 eth1addr=ba:53:8f:68:92:e4 serial=9935ded23386afc cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1 root=LABEL=linux-root rootwait rootfstype=ext4
Retrieving file: /boot/dtbs/4.4.190-1233-rockchip-ayufan-gd3f1be0ed310/rockchip/rk3328-rock64.dtb
73636 bytes read in 89 ms (807.6 KiB/s)
## Flattened Device Tree blob at 01f00000
   Booting using the fdt blob at 0x1f00000
   Loading Ramdisk to 3d8f7000, end 3df01b5a ... OK
   Loading Device Tree to 000000003d8e2000, end 000000003d8f6fa3 ... OK

Starting kernel ...
ReimuHakurei commented 4 years ago

Just to confirm, I replaced the bootloader partition's contents with an older bootloader, and the system booted just fine, so this is definitely a bootloader problem.

DDR version 1.13 20180428
ID:0x805 Y
In
LPDDR3
786MHz
Bus Width=32 Col=10 Bank=8 Row=15 CS=1 Die Bus-Width=32 Size=1024MB
ddrconfig:1
OUT

U-Boot SPL 2017.09-rockchip-ayufan-1065-g95f6152134 (Apr 06 2020 - 08:10:17)
board_init_sdmmc_pwr_en
setup_ddr_param  1
booted from SD
Trying to boot from MMC2
NOTICE:  BL31: v1.3(debug):9d3f591
NOTICE:  BL31: Built : 14:39:02, Jan 17 2018
NOTICE:  BL31:Rockchip release version: v1.3
INFO:    ARM GICv2 driver initialized
INFO:    Using opteed sec cpu_context!
INFO:    boot cpu mask: 1
INFO:    plat_rockchip_pmu_init: pd status 0xe
INFO:    BL31: Initializing runtime services
WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
ERROR:   Error initializing runtime service opteed_fast
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x200000
INFO:    SPSR = 0x3c9

U-Boot 2017.09-rockchip-ayufan-1065-g95f6152134 (Apr 06 2020 - 08:10:39 +0000)

Model: Pine64 Rock64
DRAM:  1022 MiB
MMC:   rksdmmc@ff520000: 0, rksdmmc@ff500000: 1
SF: Detected gd25q128 with page size 256 Bytes, erase size 4 KiB, total 16 MiB
*** Warning - bad CRC, using default environment

In:    serial@ff130000
Out:   serial@ff130000
Err:   serial@ff130000
Model: Pine64 Rock64
misc_init_r
cpuid=55524b50303930343200000000021c29
serial=9935ded23386afc
Net:   eth0: ethernet@ff540000
Hit any key to stop autoboot:  0
Card did not respond to voltage select!
mmc_init: -95, time 9
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:3...
Found /extlinux/extlinux.conf
Retrieving file: /extlinux/extlinux.conf
937 bytes read in 17 ms (53.7 KiB/s)
select kernel
1:      kernel-5.6.0-1137-ayufan-ge57f05e7bf8f
2:      kernel-5.6.0-1137-ayufan-ge57f05e7bf8f-memtest
Enter choice: 1:        kernel-5.6.0-1137-ayufan-ge57f05e7bf8f
Retrieving file: /initrd.img-5.6.0-1137-ayufan-ge57f05e7bf8f
21683730 bytes read in 1014 ms (20.4 MiB/s)
Retrieving file: /vmlinuz-5.6.0-1137-ayufan-ge57f05e7bf8f
22915584 bytes read in 1065 ms (20.5 MiB/s)
append: rw panic=10 init=/sbin/init coherent_pool=1M ethaddr=ba:53:8f:68:92:c4 eth1addr=ba:53:8f:68:92:e4 serial=9935ded23386afc cgroup_enable=cpuset cgroup_memory=1 cgroup_enable=memory swapaccount=1 root=LABEL=linux-root rootwait rootfstype=ext4
Retrieving file: /dtbs/5.6.0-1137-ayufan-ge57f05e7bf8f/rockchip/rk3328-rock64.dtb
37388 bytes read in 62 ms (588.9 KiB/s)
## Flattened Device Tree blob at 01f00000
   Booting using the fdt blob at 0x1f00000
   Loading Ramdisk to 3ca53000, end 3df00e12 ... OK
   Loading Device Tree to 000000003ca46000, end 000000003ca5220b ... OK

Starting kernel ...
clach04 commented 4 years ago

@ReimuHakurei excellent research 👍 ! I don't think I'm going to be able to setup serial console access due to the 1.5M rate (I don't have a USB TTL converter and I don't think the raspberry pi I have can run that fast over gpio) to confirm if I'm seeing the same thing.

However I could try the bootloader swap you did:

I replaced the bootloader partition's contents with an older bootloader, and the system booted just fine, so this is definitely a bootloader problem.

How did you do this and what partition? I'm assuming you used dd on the SD card (SD not MMC)?

thanks!

ReimuHakurei commented 4 years ago

For the USB TTL adapter, I used this one and can confirm it works fine with the Rock64: https://www.ebay.com/itm/382561156127

Only SD card reader I had handy was the one in my Windows laptop, so I actually just used a hex editor (HxD) to dump in the bootloader contents starting at 0x8000 on the card. Easier with dd for sure - it's the partition that's like 4 MB in size, I think in my case it was the partition that came first on the disk.

I used this bootloader partition: https://nextcloud.unreliable.network/index.php/s/ijTWS5nGXfzSFWz - if you'd rather extract it yourself, I believe that one came from this image (if you open the .img file with 7-Zip it shows up as "bootloader.img"; alternatively it starts at offset 8000h / 32768, and is 4128768 bytes long) https://github.com/ayufan-rock64/linux-u-boot/releases/download/2017.09-rockchip-ayufan-1065-g95f6152134/u-boot-erase-spi-rock64.img.xz - any of the non-mainline-kernel releases should work I would think, that file just so happened to be sitting on my desktop at the time.

clach04 commented 4 years ago

@ReimuHakurei thanks for taking the time to share more detail. I put an order in for the USB TTL :)

I have a very simple script, extract_first_partition.sh

#!/bin/sh
# hack/workaround for Rock64 boot failure
#  *  https://github.com/ayufan-rock64/linux-build/issues/426
#  *  https://github.com/ayufan-rock64/linux-build/issues/428

IMAGE_NAME=$1

OUT_FILE=out.img

fdisk -lu ${IMAGE_NAME}

# wget/curl/etc. https://github.com/ayufan-rock64/linux-u-boot/releases/tag/2017.09-rockchip-ayufan-1065-g95f6152134
# either flash or erase
# $ unxz u-boot-flash-spi-rock64.img.xz
# $ fdisk -lu u-boot-flash-spi-rock64.img
# GPT PMBR size mismatch (32767 != 73727) will be corrected by w(rite).
# The backup GPT table is corrupt, but the primary appears OK, so that will be used.
# Disk u-boot-flash-spi-rock64.img: 36 MiB, 37748736 bytes, 73728 sectors
# Units: sectors of 1 * 512 = 512 bytes
# Sector size (logical/physical): 512 bytes / 512 bytes
# I/O size (minimum/optimal): 512 bytes / 512 bytes
# Disklabel type: gpt
# Disk identifier: D5B9B5CB-EA91-45C8-A1BB-62870D4E4F8F
# 
# Device                       Start   End Sectors Size Type
# u-boot-flash-spi-rock64.img1    64  8127    8064   4M Linux filesystem
# u-boot-flash-spi-rock64.img2  8192 32734   24543  12M Microsoft basic data

# start
SKIP_COUNT=64
BLOCK_COUNT=8064

dd if=${IMAGE_NAME} of=${OUT_FILE} bs=512 skip=${SKIP_COUNT} count=${BLOCK_COUNT}

echo Issue:
echo sudo dd bs=4M if=out.img of=/dev/mmcblk0p1
echo 'where /dev/mmcblk0 is the SD (eMMC) device'

this allowed me to boot the latest unstable :-)

The boot partitions are different sizes, luckily the destination was smaller than the source one so we lucked out there :)

I noticed https://github.com/ayufan-rock64/linux-mainline-u-boot/releases has different images for rock64 and what looks like rock64-4Gb. I ran out of time today so I've not had chance to play with those. If we're lucky the https://github.com/ayufan-rock64/linux-build/releases/tag/0.10.12 build picked up the wrong bootloader

clach04 commented 4 years ago

Tried main line bootloader, same silent failure. No Ethernet lights (no serial USB available to check boot log)

clach04 commented 4 years ago

My TTL USB arrived and as expected I'm seeing the exact same thing in serial console :-)

I scanned the code real fast and my best guess is that Die BW should be used to make ram size decisions, I was looking at https://github.com/ayufan-rock64/linux-mainline-u-boot/blob/release-2020-01/drivers/ram/rockchip/sdram_common.c#L84 I don't see how this can support 1, 2, and 4Gb.

No idea about speed detection, I know some boards like Pi's have a slow ram speed at boot (CPU as well) and then ramp up after boot. It may simply be a reporting issue rather than a speed setting looking at https://github.com/ayufan-rock64/linux-mainline-u-boot/blob/release-2020-01/drivers/ram/rockchip/sdram_rk3328.c#L90

@ayufan any ideas? Do you have a 1Gb rock64 board? If not I'm happy to run test bootloaders on my board and send serial output if that's helpful.

dkimmortal commented 4 years ago

can you give a noob guide on what to do, how do i use the script? I have a macbook and i have the bootloader.img downloaded and the fossa minimal image from April downloaded, whats the next step? I don't have experience with hex editor. Hope you can help thanks

I ran into this same issue, have an og rock64 1gb

clach04 commented 4 years ago

@dkimmortal copy into a text file using your favorite text editor and save to filename extract_first_partition.sh then issue:

sh extract_first_partition.sh IMAGE_NAME

where IMAGE_NAME is the name of the good bootloader image. Then follow instructions that script emits. NOTE those later instructions assume your SD card device name is /dev/mmcblk0

dkimmortal commented 4 years ago

@dkimmortal copy into a text file using your favorite text editor and save to filename extract_first_partition.sh then issue:

sh extract_first_partition.sh IMAGE_NAME

where IMAGE_NAME is the name of the good bootloader image. Then follow instructions that script emits. NOTE those later instructions assume your SD card device name is /dev/mmcblk0

hey, thanks for your help. What if the sd card device is an sd card and its in /dev/sdb with the first partition in the sdb being /dev/sdb1 and it has sdb1,b2,b3,b4 in that sd card. do I change the script in the echo line at the end to

echo sudo dd bs=4M if=out.img of=/dev/sdb1 ? thx

clach04 commented 4 years ago

@dkimmortal , 👍 yes - you got it :-)

First partition of the SD card.

dkimmortal commented 4 years ago

dang so I tried many times, couldn't get it to work. It just wouldn't boot. :? but a workaround did it for me. I installed the last known working version, 0.9.16 then updated that to the latest then upgraded using the commands from the release page

sudo apt-get update -y && sudo apt-get install linux-rock64-0.10-mainline -y

Upgrade all other packages

sudo apt-get update -y && sudo apt-get dist-upgrade -y

Remove invalid X11 config, and fix bootloaders

sudo rm /etc/X11/xorg.conf.d/20-armsoc.conf sudo rm /etc/apt/sources.list.d/ayufan-rock64-0.8-*.list sudo new_extlinux_boot.sh rootfs sudo rock64_upgrade_bootloader.sh

the last bootloader upgrade script failed with the error "Cannot detect board from u-boot-rock64-2020.01-ayufan-2013-gdb96b24f." this allowed me to keep the old bootloader and start. I was hoping Wireguard was available with the latest kernel and repo but i got it to work with instructions from the engineerworkshop it finally worked after some additional workarounds because i was trying to connect to a server with pihole and ufw. (had to change pihole -a -i local (interface local) from its default.

clach04 commented 4 years ago

I was reading about the new https://www.armbian.com/newsflash/armbian-20-08-caple/ release and a couple of things caught my eye:

ayufan has been quiet for the last few months, I hope he is OK.

I'm going to try and make time for an Armbian test this week.

andrewrynhard commented 3 years ago

I am seeing the same thing with rock64 4GB FWIW:

<debug_uart>
U-Boot TPL 2020.10 (Nov 28 2020 - 03:59:47)
data training error
col error
data training error
DDR3, 333MHz
BW=16 Col=12 Bk=8 CS0 Row=16 CS=1 Die BW=8 Size=4096MB
clach04 commented 3 years ago

@andrewrynhard I recommend taking a look at https://www.armbian.com/rock64/ - it documents unsupported, I installed this a few months back and its been super stable. I thought it was supported at the time so I'm a little hazy on that.

I'm using: Focal with Linux 5.7.15-rockchip64

Instructions are super clear, devs are responsive to PRs. Everything I've thrown at it just works (e.g. Pi-Hole supported out of box).

I pinged ayufan and sent a donation earlier in the year, I suspect he no longer has time for this project. I'm super grateful for his work, it's what got me up and running so quickly on my device but I really want to have something maintained and there are more developers for Armbian.

andrewrynhard commented 3 years ago

@clach04 I am actually building a linux distro and need to build u-boot for it.

andrewrynhard commented 3 years ago

Well I found my issue... there is a rock64-rk3328_defconfig in mainline. I was using evb-rk3328_defconfig.

xixixihaha commented 2 years ago

Is there any update on this issue? I tried to burn the latest Armbian_21.08.1_Rock64_buster_legacy_4.4.213_xfce_desktop.img onto 1G and 2G version boards. Both failed with the same issue. The 2GB version board is roc-rk3328-cc open source board.

U-Boot TPL 2020.10-armbian (Aug 08 2021 - 18:02:28) data training error col error data training error LPDDR3, 800MHz BW=16 Col=12 Bk=8 CS0 Row=16 CS=1 Die BW=8 Size=4096MB

xixixihaha commented 2 years ago

@clach04 as clach04 said, I did the test with the same version Armbian_20.08_Rock64_focal_current_5.7.15_desktop.img. RAM size is detected correctly and u-boot can be loaded correctly. Besides, I tried the Renegade latest version Armbian_21.08.1_Renegade_focal_current_5.10.60_xfce_desktop.img. There is no issue to boot up. There must be something wrong with the rock64 branch.

HinataKato commented 2 years ago 
U-Boot TPL 2022.04-armbian (Jul 01 2022 - 15:56:43)
data training error
col error
data training error
LPDDR3, 800MHz
BW=16 Col=12 Bk=8 CS0 Row=16 CS=1 Die BW=8 Size=4096MB

A bit unrelated, but I compiled a U-Boot tagged v2022.04 with Armbian's build system yesterday and it looks like they haven't fixed it in about two years or so? :( (I'm using a RK3328+1G DDR3 motherboard) Edit: Seems like it's because the old version used the rk ddr blob and the mainline won't try another memory version?