MichaIng / DietPi

Lightweight justice for your single-board computer!
https://dietpi.com/
GNU General Public License v2.0
4.68k stars 491 forks source link

Odroid C2 | Does not boot on reboot without power cycling #5414

Closed yandritos closed 1 month ago

yandritos commented 2 years ago

DietPi version | G_DIETPI_VERSION_CORE=8 G_DIETPI_VERSION_SUB=1 G_DIETPI_VERSION_RC=2 G_GITBRANCH='master' G_GITOWNER='MichaIng' G_LIVE_PATCH_STATUS[0]='applied' G_LIVE_PATCH_STATUS[1]='applied' G_LIVE_PATCH_STATUS[2]='not applicable' G_LIVE_PATCH_STATUS[3]='not applicable'

Distro version | bullseye

Kernel version | Linux DietPi 5.10.81-meson64 #21.08.6 SMP PREEMPT Mon Nov 22 11:21:51 UTC 2021 aarch64 GNU/Linux

SBC model | Odroid C2 (aarch64)

Power supply used | 5V 2.1A

SDcard used | SanDisk ultra 16Gb

Additional Information (if applicable)

Was the software title installed freshly or updated/migrated? -Fresh install Can this issue be replicated on a fresh installation of DietPi? Yes

Bug report ID | echo $G_HW_UUID Not applies

Steps to reproduce -Whenever you send a reboot comand after shooting down, the Odroid C2 never boots up, I need to unplug/plug manually the power suply. (No way to work remotely after a reboot command). -After plug an screen to the Odroid C2 board and send even locally the reboot, I can see how the OS shut down properly finishing all the tasks, then a black screen and never boots again unles I unplug the board (this never happend with older verions).

Expected behaviour -A regular restart of the OS.

Actual behaviour -The board never boots again after a reboot command.

MichaIng commented 2 years ago

Many thanks for your report.

Do you know whether there was a kernel upgrade done? I.e. are the modules in /lib/modules newer than v5.10.81? On a quick look at the Armbian APT repository it seems like it is still the newest "current" kernel, but probably the repos are still syncing.

If a kernel upgrade was done, can you try to flash the new U-Boot:

. /usr/lib/u-boot/platform_install.sh
write_uboot_platform "$DIR" "$(lsblk -npo PKNAME "$G_ROOTFS_DEV")"
yandritos commented 2 years ago

Hi MichaIng,

Thanks a lot for the very fast reply. I executed both commands without errors but after that the result is the same.

I checked the version from the dietpi-config advanced options. --> The current version is Linux 5.10.102-meson64

MichaIng commented 2 years ago

Okay, so probably the kernel upgrade from 5.10.81 to 5.10.102 is responsible. If you have a spare card/drive, could you try it with the current Armbian image, please: https://dl.armbian.com/odroidc2/Bullseye_current If so, it should be reported here: https://forum.armbian.com/forum/16-amlogic-s905x-s922x/

Btw, how long did you wait before power cycling? Probably there is one service/unit hanging on shutdown. A maximum timeout for any hanging ExecStop command or such should be 90 seconds.

yandritos commented 2 years ago

Hi, I waited even 10 mins without luck. When I plugged the screen on the hdmi port I saw the job with the counter for 1min 30 secs stopping other tasks, that counter finishes even earlier but when all is halted and I get the black screen the new boot never happens.

I have another SD card so today I'll try to use the iso of your link "[Bullseye_current]" (The web SSL cert expired a couple of days ago :D). I'll be back asap with the results.

I have a similar error as commented by a user of an Odroid C4 in your second link: https://forum.armbian.com/topic/19340-odroid-c4-will-not-reboot-after-any-sort-of-kernel-update-have-tried-running-nand-sata-install/

Thanks for your help!

yandritos commented 2 years ago

Well, confirmed, is a kernel topic. After installing armbian I have the same problem. I reported the isue as you suggested --> https://forum.armbian.com/topic/20241-odroid-c2-never-starting-after-a-soft-reboot-reboot-command/

The universe is telling me to go back to my dietpi from Sep 2021 :D the best option for my Odroid C2 rightnow (preserving the possibility to have the fs-boot in an external drive).

Thanks again for the help!

MichaIng commented 2 years ago

Could you try it with the edge kernel? At least the automated smoke tests, which include a restart test, go through with it:

apt install linux-image-edge-meson64 linux-dtb-edge-meson64 linux-u-boot-odroidc2-edge
yandritos commented 2 years ago

Hi MichaIng,

Sorry I don't want to spend more of your time and mine. I went back finally to the last iso I had from Sep 2021 and I got back the option to run the "root fs" from my external usb ssd. It is working like a charm so I don't really need to run the last version.

Thanks for your help and time!

yandritos commented 2 years ago

I'll continue with the last year version untill a real/relevant reason avoid me to do that :)

MichaIng commented 2 years ago

Let me reopen the issue to keep track of it, since staying on old kernel with vulnerabilities like dirty pipe is not an option for the image we provide.

MichaIng commented 2 years ago

Cross-linking a forum report: REMOVED OBSOLETE LINK

yandritos commented 2 years ago

Let me reopen the issue to keep track of it, since staying on old kernel with vulnerabilities like dirty pipe is not an option for the image we provide.

Hi Michalng sorry for the delay, I think I'll be able to do this with another sd-card during the next days. Do you need me to execute the command "apt install linux-image-edge-meson64 linux-dtb-edge-meson64 linux-u-boot-odroidc2-edge" using the last dietpi release or the current Armbian image you requested before?

MichaIng commented 2 years ago

Good question. I suggest to test it with a fresh Armbian image first, so we can report results on their forums as well. If this works, then it should do so on DietPi as well.

yandritos commented 2 years ago

Hi, after executing the command over the fresh installation of the last armbian you linked, some packages are not found.

Maybe this is easy to solve but my knowledge is very little. Please see below the traces:


Welcome to Armbian 22.02.1 with Linux 5.10.102-meson64

No end-user support: community creations

System load: 36% Up time: 0 min Memory usage: 6% of 1.89G IP: 192.168.1.111 CPU temp: 27°C Usage of /: 2% of 117G

Last login: Thu Apr 14 14:11:29 2022 from 192.168.1.95 root@odroidc2:~# apt install linux-image-edge-meson64 linux-dtb-edge-meson64 linux-u-boot-odroidc2-edge Reading package lists... Done Building dependency tree... Done Reading state information... Done E: Unable to locate package linux-image-edge-meson64 E: Unable to locate package linux-dtb-edge-meson64 E: Unable to locate package linux-u-boot-odroidc2-edge_

MichaIng commented 2 years ago

Did you run apt update already?

yandritos commented 2 years ago

Hi Michalng,

This is after apt update:


Fetched 112 MB in 48s (2,333 kB/s)
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
44 packages can be upgraded. Run 'apt list --upgradable' to see them.
N: Repository 'http://deb.debian.org/debian bullseye InRelease' changed its 'Ver                                     sion' value from '11.2' to '11.3'
root@odroidc2:~# apt install linux-image-edge-meson64 linux-dtb-edge-meson64 lin                                     ux-u-boot-odroidc2-edge
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  linux-u-boot-odroidc2-current
The following NEW packages will be installed:
  linux-dtb-edge-meson64 linux-image-edge-meson64 linux-u-boot-odroidc2-edge
0 upgraded, 3 newly installed, 1 to remove and 44 not upgraded.
Need to get 55.1 MB of archives.
After this operation, 99.7 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 https://es-mirror.bret.dk/armbian/apt bullseye/main arm64 linux-dtb-edge-m                                     eson64 arm64 22.05.0-trunk.0038 [313 kB]
Get:3 https://es-mirror.bret.dk/armbian/apt bullseye/main arm64 linux-u-boot-odr                                     oidc2-edge arm64 22.02.1 [307 kB]
Err:2 https://armbian.site-meganet.com/apt bullseye/main arm64 linux-image-edge-                                     meson64 arm64 22.05.0-trunk.0038
  404  Not Found [IP: 188.165.219.168 443]
Fetched 619 kB in 1s (647 kB/s)
E: Failed to fetch https://armbian.site-meganet.com/apt/pool/main/l/linux-5.17.1                                     -meson64/linux-image-edge-meson64_22.05.0-trunk.0038_arm64.deb  404  Not Found [                                     IP: 188.165.219.168 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-mis                                     sing?

Then, as per the error recommendation I added "--fix-mis". This time it was installed, this was the output


root@odroidc2:~# apt install linux-image-edge-meson64 linux-dtb-edge-meson64 linux-u-boot-odroidc2-edge
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  linux-u-boot-odroidc2-current
The following NEW packages will be installed:
  linux-dtb-edge-meson64 linux-image-edge-meson64 linux-u-boot-odroidc2-edge
0 upgraded, 3 newly installed, 1 to remove and 44 not upgraded.
Need to get 54.5 MB/55.1 MB of archives.
After this operation, 99.7 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 https://es-mirror.bret.dk/armbian/apt bullseye/main arm64 linux-image-edge-meson64 arm64 22.05.0-trunk.0038 [54.5 MB]
Fetched 54.5 MB in 9s (6,271 kB/s)
(Reading database ... 41072 files and directories currently installed.)
Removing linux-u-boot-odroidc2-current (22.02.1) ...
Selecting previously unselected package linux-dtb-edge-meson64.
(Reading database ... 41066 files and directories currently installed.)
Preparing to unpack .../linux-dtb-edge-meson64_22.05.0-trunk.0038_arm64.deb ...
Unpacking linux-dtb-edge-meson64 (22.05.0-trunk.0038) ...
Selecting previously unselected package linux-image-edge-meson64.
Preparing to unpack .../linux-image-edge-meson64_22.05.0-trunk.0038_arm64.deb ...
Unpacking linux-image-edge-meson64 (22.05.0-trunk.0038) ...
Selecting previously unselected package linux-u-boot-odroidc2-edge.
Preparing to unpack .../linux-u-boot-odroidc2-edge_22.02.1_arm64.deb ...
Unpacking linux-u-boot-odroidc2-edge (22.02.1) ...
Setting up linux-dtb-edge-meson64 (22.05.0-trunk.0038) ...
Setting up linux-image-edge-meson64 (22.05.0-trunk.0038) ...
dkms: running auto installation service for kernel 5.17.1-meson64:.
update-initramfs: Generating /boot/initrd.img-5.17.1-meson64
update-initramfs: Converting to u-boot format
Setting up linux-u-boot-odroidc2-edge (22.02.1) ...

After that the result is the same, I execute a reboot and no effect. When is down the led remains in red and never again boots until I unplug and re-plug the power suply.

image

yandritos commented 2 years ago

Even I tried a reínstall, but already detected as the last version. Never working after the reboot.

root@odroidc2:~# apt install linux-image-edge-meson64 linux-dtb-edge-meson64 linux-u-boot-odroidc2-edge
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
linux-dtb-edge-meson64 is already the newest version (22.05.0-trunk.0038).
linux-image-edge-meson64 is already the newest version (22.05.0-trunk.0038).
linux-u-boot-odroidc2-edge is already the newest version (22.02.1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
MichaIng commented 2 years ago

Can you again try to flash U-Boot after installing the edge packages:

. /usr/lib/u-boot/platform_install.sh
write_uboot_platform "$DIR" "$(lsblk -npo PKNAME "$G_ROOTFS_DEV")"

It is strange that neither the smoke tests, nor the one user (with Debian Buster) are able to replicate the issue. I wonder whether it may be due to insufficient voltage/power, other due to other attached peripherals/hardware. I see you have a 2.1A PSU, which from the numbers is sufficient: https://wiki.odroid.com/odroid-c2/hardware/hardware#specifications

The ODROID-C2 consumes less than 0.8A in most cases, but it can climb to 2A if many passive USB peripherals are attached directly to the main board.

But PSUs as well as USB cables have a high quality range, not all PSUs are really capable of reliably provide the 5V voltage with tight tolerance, especially when being actually a charger and not for driving power consumers.

yandritos commented 2 years ago

Hi,

I am thinking that the power suply is not the problem. In the other sdcard I have my old dietpi released on sep 2021 with an additional ssd unit connected via USB (not additional power suply for the ssd unit) working like a charm and rebooting without any problem, even that deitpi is a debian buster one if I remember well.

I executed the commands requested, this is the output (note "lsblk: : not a block device"):

Note: for these tests I don't have connected the external ssd drive via usb.


Last login: Thu Apr 14 16:16:24 2022 from 192.168.1.95
root@odroidc2:~# . /usr/lib/u-boot/platform_install.sh
root@odroidc2:~# write_uboot_platform "$DIR" "$(lsblk -npo PKNAME "$G_ROOTFS_DEV")"
lsblk: : not a block device
root@odroidc2:~# 
MichaIng commented 2 years ago

even that deitpi is a debian buster one if I remember well.

That is interesting, the user on the Armbian forum who wasn't able to replicate as well ran a Buster image. I have no idea how the userland should affect it, since AFAIK, the kernel, dtb and U-Boot builds are exactly the same. But an idea for further debugging the userland part would be to enable persistent system logs:

dietpi-software uninstall 103
mkdir /var/log/journal
reboot # give it some time before power cycling, since we want the /var/log/journal dir to be synced to disk successfully ;)

Now journalctl logs are stored to /var/log/journal. So when you now reboot again, you can then call journalctl and should see all logs from the previous shutdown sequence, at least until systemd-journald itself is shut down. This includes also kernel logs, in case they occur.

yandritos commented 2 years ago

journalctl

Hi Michalng, I don't got if you want the traces of my old dietpi which runs well (and reboots properly), or the traces of the new armbian I installed for this testing.

I am providing the traces of the armbian one which is the one not working after the reboot command. I ignored the command "dietpi-software uninstall 103" because it is not detected.

These are the output traces after "journalctl" for the last armbian release:

Note: I enabled the traces following your instructions, I executed "reboot" I waited for 5 minutes without any signal of live and then I unpluged/pluged the system getting this output for the journal


give me some time, collecting all in a file, the output was truncated

yandritos commented 2 years ago

Hi, Please find attached the full content of the journal folder. all_logs.txt

MichaIng commented 2 years ago

Ah sorry, I forgot that it's an Armbian system now. Please try to disable armbian-ramlog via armbian-config, or via:

systemctl disable armbian-ramlog
reboot

I hope we see some more logs of the shutdown sequence then.

yandritos commented 2 years ago

Hi, just as requested


Last login: Thu Apr 14 21:14:34 2022 from 192.168.1.95
root@odroidc2:~# systemctl disable armbian-ramlog
Removed /etc/systemd/system/sysinit.target.wants/armbian-ramlog.service.
root@odroidc2:~# reboot

Then I waited for 10 mins prior to unplug/plug and get the trace file below.

all_logs2.txt

ManuVice commented 2 years ago

Hi, I had the problem that the latest DietPi image (12/2021/ didnt boot. Now I have a emmc storage and latest image does boot + reboot is working just fine.

If I remove emmc and insert SD Card with fresh flashed latest image it doesnt boot.

GXBB:BL1:08dafd:0a8993;FEAT:EDFC318C;POC:3;RCY:0;EMMC:800;NAND:81;SD:0;READ:0;CHK:0;

TE: 198762 no sdio debug board detected

BL2 Built : 11:44:26, Nov 25 2015. gxb gfb13a3b-c2 - jcao@wonton

Board ID = 8 set vcck to 1100 mv set vddee to 1050 mv CPU clk: 1536MHz DDR channel setting: DDR0 Rank0+1 same DDR0: 2048MB(auto) @ 912MHz(2T)-13 DataBus test pass! AddrBus test pass! Load fip header from SD, src: 0x0000c200, des: 0x01400000, size: 0x000000b0 Load bl30 from SD, src: 0x00010200, des: 0x01000000, size: 0x00009ef0 Sending bl30........................................OK. Run bl30... Load bl301 from SD, src: 0x0001c200, des: 0x01000000, size: 0x000018c0 Wait bl30...Done Sending bl301.......OK. Run bl301... l31 from SD, src: 0x00020200, des: 0x10100000, size: 0x00011130

--- UART initialized after reboot --- [Reset cause: unknown] [Image: unknown, amlogic_v1.1.3046-00db630-dirty 2016-08-31 09:24:14 tao.zeng@droid04] bl30: check_permit, count is 1 bl30: check_permit: ok! chipidLoad bl33 from SD, src: 0x00034200, des: 0x01000000, size: 0x000a1d50 : ef be ad de d f0 ad ba ef be ad de not ES chip [0.313552 Inits done] secure task start! high task start! low task start! NOTICE: BL3-1: v1.0(debug):4d2e34d NOTICE: BL3-1: Built : 17:08:35, Oct 29 2015 INFO: BL3-1: Initializing runtime services INFO: BL3-1: Preparing for EL3 exit to normal world INFO: BL3-1: Next image address = 0x1000000 INFO: BL3-1: Next image spsr = 0x3c9

U-Boot 2021.07-armbian (Nov 22 2021 - 11:48:10 +0000) odroid-c2

Model: Hardkernel ODROID-C2 SoC: Amlogic Meson GXBB (S905) Revision 1f:c (0:1) DRAM: 2 GiB MMC: mmc@72000: 0, mmc@74000: 1 Loading Environment from nowhere... OK In: serial Out: serial Err: serial Net: eth0: ethernet@c9410000 Hit any key to stop autoboot: 0 => boot Card did not respond to voltage select! : -110 MMC Device 2 not found no mmc device at slot 2 starting USB... Bus usb@c9100000: USB DWC2 scanning bus usb@c9100000 for devices... 2 USB Device(s) found scanning usb for storage devices... 0 Storage Device(s) found

Device 0: unknown device Speed: 1000, full duplex BOOTP broadcast 1 DHCP client bound to address 10.0.0.6 (6 ms) *** Warning: no boot file name; using '0A000006.img' Using ethernet@c9410000 device TFTP from server 10.0.0.1; our IP address is 10.0.0.6 Filename '0A000006.img'. Load address: 0x1080000

Loading: T T T T T T T T T T Retry count exceeded; starting again missing environment variable: bootfile Retrieving file: pxelinux.cfg/0A0 Speed: 1000, full duplex Using ethernet@c9410000 device TFTP from server 10.0.0.1; our IP address is 10.0.0.6 Filename 'pxelinux.cfg/0A0'. Load address: 0x1080000 Loading: T T T T T T T T T T

ManuVice commented 2 years ago

Reboot doesnt work with SD Card on Armbian:

Reboot doesnt work: Armbian 5.10.102 u-boot 2019.10.txt

Reboot doesnt work: Armbian 5.17.1 u-boot 2019.10.txt

Boot doesnt work: Armbian 5.17.1 u-boot 2022.01.txt

eMMC Boot and Reboot works with Kernel 5.10.102 and u-boot 2021.07 Dietpi 5.10.103 u-boot 2021.07.txt

yandritos commented 2 years ago

Thanks ManuVice for your inputs, (off topic: just wondering if you installed hyperion ng server on a odroid c2).

Just in case it can help, the old dietpi in which the reboot works properly for me (downloaded on Sep 2021) is DietPi v8.3.1 with firmware [Linux 3.16.85+]. This old version is different on the boot as well and the root fs was moved to an external ssd unit (but it worked in the sd card as well).

ManuVice commented 2 years ago

@yandritos Hyperion ng has a mem leak with my karatelight. Cannot use it atm :-( But in past I used a RPi3b which is now a server for some other services. Hyperion uses too much cpu and that's why I will using odroidc2 now.

Btw I have updated last Dietpi image with emmc to last kernel and u-boot. Updated and rebooted without a problem

apt install linux-image-edge-meson64 linux-dtb-edge-meson64 linux-u-boot-odroidc2-edge . /usr/lib/u-boot/platform_install.sh write_uboot_platform "$DIR" "$(lsblk -npo PKNAME "$G_ROOTFS_DEV")"

yandritos commented 2 years ago

@yandritos Hyperion ng has a mem leak with my karatelight. Cannot use it atm :-( But in past I used a RPi3b which is now a server for some other services. Hyperion uses too much cpu and that's why I will using odroidc2 now.

Btw I have updated last Dietpi image with emmc to last kernel and u-boot. Updated and rebooted without a problem

apt install linux-image-edge-meson64 linux-dtb-edge-meson64 linux-u-boot-odroidc2-edge . /usr/lib/u-boot/platform_install.sh write_uboot_platform "$DIR" "$(lsblk -npo PKNAME "$G_ROOTFS_DEV")"

Thanks for hte info, maybe I'll prepare an old raspi 2 isolated only for hyperion ng (in the past y runned kodi+hyperion in the raspberry pi1, but this NG version appear to be more heavy).

As per your comments appear to be something relatd to the microSD unit if all goes well with the emmc unit. I remember to see in the original instructions that the boot sequence in the Odroid C2 is: -1. emmc -2. if no emmc present, then boot from micro sd

Maybe with the last kernel after a reboot command the boot never jumps from smmc to micro SD

MichaIng commented 2 years ago

Okay, so reboot works with eMMC but using the same image on SD card, reboot does not work? That would be a good hint for the Armbian forum for devs to replicate.

ManuVice commented 2 years ago

yep 🙂

yandritos commented 2 years ago

Hi guys, yes it was notified a week ago or so, another user has the same behavior, emmc reboot works, sd card doesn't. Let's see if the root cause is discovered. Have a nice weekend!

MichaIng commented 2 years ago

I generated a new image, from scratch this time: https://dietpi.com/downloads/images/testing/ I'm not aware of a solution for the reboot issue, but who knows 😉.

ManuVice commented 2 years ago

great 🙂 I will test it tommorow and give feedback

ManuVice commented 2 years ago

sd card doesnt boot with test image

sdcard test image.txt print env.txt

=> mmc list
mmc@72000: 0 (SD)
mmc@74000: 1
=> mmc part

Partition Map for MMC device 0  --   Partition Type: DOS

Part    Start Sector    Num Sectors     UUID            Type
  1     8192            1508200         d4c70000-01     83 Boot

UPDATE: I'am online!!!

=> bootcmd_mmc0=devnum=0 => run mmc_boot => boot working console log.txt

reboot doesnt boot reboot.txt

poweroff gives a different message than reboot poweroff.txt

Update2: If I attach SD Card and MMC at the same time it boots from sdcard without any error like above.

MichaIng commented 2 years ago

Hmm, you mean the current testing image does not boot at all from SD card, it only, as the previous one, does not boot on reboot?

ManuVice commented 2 years ago

the current testing Image you posted did not boot from sd card. I got it work with bootcmd_mmc0=devnum=0, run mmc_boot, boot over serial console to boot into os. and after that I could try the reboot command which failed too.

but with sdcard and mmc insert at the same time it booted from sdcard. Was funny fault because I forget to eject the sdcard and my login from mmc image didnt work 😂

MichaIng commented 2 years ago

but with sdcard and mmc insert at the same time it booted from sdcard

That is very confusing 😄.

It is strange, in the logs I see that it looks for "MMC Device 2" (eMMC, I guess) and "USB", but not for the SD card as boot device, even that it loaded U-Boot itself from SD card 🤔. I don't see an issue in the env, do you?

boot_targets=romusb mmc0 mmc1 mmc2 usb0 pxe dhcp
bootcmd=run distro_bootcmd
bootcmd_mmc0=devnum=0; run mmc_boot
bootcmd_mmc1=devnum=1; run mmc_boot
bootcmd_mmc2=devnum=2; run mmc_boot
distro_bootcmd=for target in ${boot_targets}; do run bootcmd_${target}; done

So it loops through all boot targets, including mmc0, and calls the related bootcmd_mmc0 command, which is exactly what you did manually call.

I'll check back the build log and implementation in the build scripts.

ManuVice commented 2 years ago

I dont see any issue in env too. I looked at u-boot src code but my code understanding ends here. Very strange 👻

summary from today: I compared u-boot 2015 from hardkernel and u-boot 2022.01 from last testing image. I cannot find any errors, but I'am not a coder :-(

I have tested some conditions with latest testing image: only sdcard ->

Card did not respond to voltage select! : -110
MMC Device 2 not found
no mmc device at slot 2

sdcard + emmc insert at the same time both with fresh testing image -> boot from mmc0 (sdcard)

only emmc -> boot from mmc1

OnixGH commented 1 year ago

This may or may not be related, but the symptoms look the same: https://patches.linaro.org/project/u-boot/patch/20201217090624.14902-1-m.szyprowski@samsung.com/

Summary:

For the proper reboot Odroid C4 board requires to switch TFLASH_VDD_EN pin to the input (high impedance?) mode, otherwise the board is stuck in the middle of loading early stages of the bootloader from SD card.

  1. if SD card is not used (TFLASH regulator is not even instantiated, so GPIO pin is in input state as its initial value left by earlier BL stages), the reboot works fine
  2. If SD card is touched, then reboot hangs somewhere between BL2->BL3 stages

Which was solved as follows:

Could you try switching the tflash_vdd to opendrain like :


--- a/arch/arm/dts/meson-sm1-odroid-c4.dts
+++ b/arch/arm/dts/meson-sm1-odroid-c4.dts
@@ -52,7 +52,7 @@
regulator-min-microvolt = <3300000>;
regulator-max-microvolt = <3300000>;
OnixGH commented 1 year ago

On the C4 reboot was broken in kernel 5.10.102-meson64 as well, just tested with kernel 5.10.144-meson64 from sept. and it appears to be fixed (tested 3x). Edit: Armbian / bullseye.

sormy commented 1 year ago

Hmm, C4 soft reboot from SD card is broken for me on Armbian 22.08.2 Jammy with Linux 5.10.144-meson64

Joulinar commented 1 year ago

Jammy is Ubuntu, you would need to use Bullseye Debian.

MichaIng commented 1 year ago

Kernel and bootloader are the same. There is a new version 22.08.4 from 3 days ago:

apt update
apt full-upgrade

On Armbian there should be an option to flash the new bootloader in armbian-config, on DietPi it can be done via:

. /usr/lib/u-boot/platform_install.sh
write_uboot_platform "$DIR" "$(lsblk -npo PKNAME "$G_ROOTFS_DEV")"

Let's hope this has it fixed. I'll rebuild our images.

MichaIng commented 1 year ago

Images for Odroid C2 and C4 both have been updated. At least both use the same kernel, so there is a tiny chance that also the bootloader has been fixed for both regarding reboot.

Joulinar commented 1 year ago

@MichaIng might this be related https://dietpi.com/forum/t/c4-does-not-reboot-after-last-update/14566

MichaIng commented 1 year ago

Interesting, with only the kernel updated 🤔.

x-Felix commented 1 year ago

It's same issue for Odroid N2+ with Armbian on eMMC. Soft reboot hangs every time with no blue LED. While I tried Debian 11 image debian-bullseye-server-odroidn2plus-20220128 from https://docs.linuxfactory.or.kr/install/odroidn2/image.html, no issue with soft reboot.

MichaIng commented 1 year ago

Seems to be an issue on all meson64 boards with recent Armbian kernel/bootloader. I read a lot of reports on their forum.

Maybe I'll have another look at Debian kernel and U-Boot. At least for Odroid C2 and N2 they have U-Boot binaries as well. With backports, soon Linux 6.0 is available on Bullseye.

gitmeister commented 1 year ago

I've been looking at this issue to solve the reboot issue on my Odroid C2. I tried the DTS fix to open_drain the TFLASH_VDD_EN (which needs to be adapted for the C2 because the pin assignment is different, but no joy after re-inserting the DTB.deb, HEADERS.deb and IMAGE.deb (maybe I just needed DTB I don't know). What did work is track down the origin of the console log message that indicates "watchdog: watchdog0: watchdog did not stop!" (watchdog_dev.c: watchdog_release()) and brutally force a WDT timeout disabling interrupts/setting a 3s timeout and restarting the WDT to let it time out and reboot the board. This work flawlessly. Of course it's a test because I bypassing some last minute Kernel notifications but it restores the processor to a POR state that is basically what the BL blobs are expecting when they init the devices.

Here's the DTS patch that I tried (for the C2) which didn't succeed:

diff --git a/arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dts b/arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dts
index 7af088c73..f3f31f1ba 100644
--- a/arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dts
+++ b/arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dts
@@ -88,7 +88,7 @@ tflash_vdd: regulator-tflash_vdd {
        /*
         * signal name from schematics: TFLASH_VDD_EN
         */
-       gpio = <&gpio GPIOY_12 GPIO_ACTIVE_HIGH>;
+       gpio = <&gpio GPIOY_12 GPIO_OPEN_DRAIN>;
        enable-active-high;
        /* U16 RT9179GB */
        vin-supply = <&vddio_ao3v3>;

and here's the watchdog hack patch:

diff --git a/drivers/watchdog/watchdog_dev.c b/drivers/watchdog/watchdog_dev.c
index 54903f3c8..83327a66d 100644
--- a/drivers/watchdog/watchdog_dev.c
+++ b/drivers/watchdog/watchdog_dev.c
@@ -936,7 +936,14 @@ static int watchdog_release(struct inode *inode, struct file *file)
    /* If the watchdog was not stopped, send a keepalive ping */
    if (err < 0) {
        pr_crit("watchdog%d: watchdog did not stop!\n", wdd->id);
-       watchdog_ping(wdd);
+       pr_crit("WDWR: Set-up.");
+       local_irq_disable();
+       wdd->ops->set_timeout(wdd, 3);
+       pr_crit("WDWR: Start.");    
+       wdd->ops->start(wdd);
+       pr_crit("WDWR: Now we wait.\n");    
+       wdd->ops->ping(wdd);
+       while(1);
    }
    watchdog_update_worker(wdd);

Not being versed in the Linux kernel I was curious why the watchdog didn't appear to be resetting the board (as I would have thought was the usual way) but it seems there's a Worker task that "kicks the dog" and somehow may be preventing this. In any event as I learn more (or if somebody can point out a better place) . I'm familiar with the original fix from here (https://github.com/armbian/build/pull/4824) but wasn't comfortable enough with the notification handlers to understand what's happening.

MichaIng commented 1 year ago

The patch at Armbian has been merged already, and, AFAIK meson64 current kernel has been moved to v6.1. So a kernel upgrade might solve the issue.