Green LED light doesnt stop blinking when attempting eMMC installation?

[ashleyw-iag]

Using a fresh copy on an SD card with the latest release, I've held the USER button during the orange blinking light on first boot. It does'nt shift to the fast blinking green light, but never seems to stop.

This is an Advanced box and I have tried to reboot after a while without the SD, but it behaves as it would without any OS installed.

Thanks for any help, Ash.

[jan-kiszka]

The pattern is: It first blinks orange to ask you whether to install to eMMC. Then you press the user button, and it switching to red blinking while installing. When done, it stops and reboots, ideally into the freshly installed eMMC image.

[ashleyw-iag]

Ah okay got you, I've just tried that and I briefly get a red blinking light (for two or three pulses) then is goes to solid green and orange again.

[jan-kiszka]

Then there is some problem. Hard to say which one... You could try journalctl -u install-on-emmc-on-first-boot.service when booted the external image after that failed attempt.

[ashleyw-iag]

brill thank you, ill give it a go and let you know.

[ashleyw-iag]

ive just got back to this, after pressing the USER button and allowing it to fail then letting it boot it doesnt appear to store any information in the journal with regards to that failure:

Sep 20 08:40:51 iot2050-debian install-on-emmc.sh[169]: Press USER button to install on eMMC, you have 20 seconds
Sep 20 08:40:51 iot2050-debian install-on-emmc.sh[169]: WARNING: All data on eMMC will be overwritten!
Sep 20 08:41:11 iot2050-debian install-on-emmc.sh[169]: Timeout expired, skipping eMMC installation.
Sep 20 08:41:11 iot2050-debian install-on-emmc.sh[224]: Removed /etc/systemd/system/systemd-remount-fs.service.wants/install-on-emmc-on-first-boot.service.
Sep 20 08:41:12 iot2050-debian systemd[1]: Finished Optional installation on eMMC.

~Could I call this procedure from the cli to recreate the error?~ I am running the script in the terminal, will report back!

[ashleyw-iag]

I appear to be getting stuck writing the bytes, not much thats useful I'm afraid:

root@iot2050-debian:~# /usr/share/install-on-emmc/install-on-emmc.sh
Press USER button to install on eMMC, you have 20 seconds
WARNING: All data on eMMC will be overwritten!
Hold USER button for 5 seconds to confirm
Writing 31211487 sectors to eMMC /dev/mmcblk1...

[jan-kiszka]

There should be "Updating partition UUID of eMMC rootfs" or so after this or at least some error message when dd failed - strange...

Wait! Maybe your device already arms the watchdog during firmware start. We are not feeding it in the install-on-emmc.sh script. That could explain things. I'll try to confirm this and write a patch when needed. Meanwhile, you could update the U-Boot env variable watchdog_timeout_ms to 0 using fw_setenv under Linux.

[ashleyw-iag]

Thanks for that, i can certainly give it a go too!

I have managed to get the script running by removing the conv=fsync argument from the dd command although now im hitting a storage space issue:

Press USER button to install on eMMC, you have 20 seconds
WARNING: All data on eMMC will be overwritten!
Hold USER button for 5 seconds to confirm
Writing 31211487 sectors to eMMC /dev/mmcblk1...
15912092672 bytes (16 GB, 15 GiB) copied, 1514 s, 10.5 MB/s 
dd: writing to '/dev/mmcblk1': No space left on device
31080449+0 records in
31080448+0 records out
15913189376 bytes (16 GB, 15 GiB) copied, 1517.73 s, 10.5 MB/s
Now partprobe...
Error: Invalid argument during seek for read on /dev/mmcblk1
Updating partition UUID of eMMC rootfs
Waiting for /dev/mmcblk1p1 to appear
........................................................................................................................................................................................................

I am using a 16GB SD card and have tried to reduce partition but Im guessing the partition is either expanded or the whole disk is copied? Is there a good way around this you know of?

[jan-kiszka]

This is wrong, and the error is expected. Check if that watchdog was already armed and disable it as described.

[ashleyw-iag]

I created a new SD from the latest build, let it boot from the SD, added the config you recommended, then ran the emmc script from within the terminal. This is what I've got:

[jan-kiszka]

You can't run install-on-emmc.sh from a booted image. It is run from a freshly flashed SD-card or USB stick only.

But I see that watchdog_timeout_ms is 0, thus is not the problem. Or did you have to change it?

[ashleyw-iag]

I had to change it and was checking it had changed in the screenshot. It was blank after the freshly booted SD image had completed.

[ashleyw-iag]

whats the correct way to update this variable in the image prior to installation? im happy to give it a go, we were hoping to use the eMMC for a project we're currently installing.

[jan-kiszka]

The watchdog configuration is stored in the firmware flash, not on the image. So, you can try again after clearing it by using a fresh image and the regular trigger (not by running the script from a booted system).

[ashleyw-iag]

just gave this a go but no luck, just sits flashing the green light and never completes. i'm happy to try any other methods you can think of?

[jan-kiszka]

OK, the watchdog issue was a red herring: I've tried with watchdog armed today, and that will not kill the flashing because the example image allows the kernel driver to keep on pinging the watchdog while the eMMC is written. And flashing worked fine for me, at least when using an image built from latest 'master'.

[jan-kiszka]

BTW, flashing time heavily depends on the access speed of the medium (SD-card or USB stick) that is feeding this. I was even using a slow emulated USB stick, and the process probably took around 20-30 minutes (didn't measure, left it running at some point).

[ashleyw-iag]

hey again, today ive downloaded the latest master in hopes to get a build working similar to you (i was using the last release before from a year ago) but now I am facing this when building:

Not sure if you'd have any thoughts on how to resolve this? Thanks so much for your help again, very grateful!

[jan-kiszka]

Which revision were you building? Note that CI just recently passed this recipe build. I also did a local clean rebuild of this particular recipe and had on such problems.

If you can reproduce this, please share the exact steps and logs about this.

[ashleyw-iag]

just tried this on fresh Debian 10 and before Debian 12 using parallels on M1 Mac, following steps set out by the GitHub repo and this is what i get every time now. Didn't seem to happen with the release from Sept. last year.

[ashleyw-iag]

have now tried on a 64 bit emulated ubuntu 22 and same issue

[jan-kiszka]

Ah, I suspect both your builds were on arm64 Linux: Indeed, we have a native build issue with optee-client-iot2050 on arm64, already in Isar. Patch will follow.

[jan-kiszka]

Temporary patch against meta-iot2050 while waiting for https://groups.google.com/g/isar-users/c/4-YJW8K86UE.

diff --git a/isar-patches/0001-optee-client-Fix-native-arm64-build.patch b/isar-patches/0001-optee-client-Fix-native-arm64-build.patch
new file mode 100644
index 00000000..3a671428
--- /dev/null
+++ b/isar-patches/0001-optee-client-Fix-native-arm64-build.patch
@@ -0,0 +1,31 @@
+From fce7e3667affe5fce82381baeb473dd7342bf00d Mon Sep 17 00:00:00 2001
+From: Jan Kiszka <jan.kiszka@siemens.com>
+Date: Tue, 3 Oct 2023 21:42:55 +0200
+Subject: [PATCH] optee-client: Fix native arm64 build
+
+optee-client sets CROSS_COMPILE to arm-linux-gnueabihf- by default,
+causing breakages when building natively, e.g. on arm64. Fix this by
+simply setting CROSS_COMPILE unconditionally.
+
+Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
+---
+ meta/recipes-bsp/optee-client/files/debian/rules.tmpl | 2 --
+ 1 file changed, 2 deletions(-)
+
+diff --git a/meta/recipes-bsp/optee-client/files/debian/rules.tmpl b/meta/recipes-bsp/optee-client/files/debian/rules.tmpl
+index 1b7920d0..7d1b335b 100755
+--- a/meta/recipes-bsp/optee-client/files/debian/rules.tmpl
++++ b/meta/recipes-bsp/optee-client/files/debian/rules.tmpl
+@@ -7,9 +7,7 @@
+ #
+ # SPDX-License-Identifier: MIT
+ 
+-ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE))
+ export CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)-
+-endif
+ 
+ %:
+   dh $@ --exclude=.a
+-- 
+2.35.3
+
diff --git a/kas/iot2050.yml b/kas/iot2050.yml
index 2191defd..6ea6486a 100644
--- a/kas/iot2050.yml
+++ b/kas/iot2050.yml
@@ -30,6 +30,9 @@ repos:
     commit: ecc7cae423a828d294575301c1584fbe303cba3e
     layers:
       meta:
+    patches:
+      01:
+        path: isar-patches/0001-optee-client-Fix-native-arm64-build.patch

   cip-core:
     url: https://gitlab.com/cip-project/cip-core/isar-cip-core.git

[ashleyw-iag]

thank you so much for the time spent on this @jan-kiszka, i really appreciate it.

~how do I apply this patch?~ applied manually and building now!

[ashleyw-iag]

success! using latest master branch its now running off of emmc.

thank you so much again for your time on this!