Add EEPROM programming support

alexgg commented 7 months ago

Depends on https://github.com/balena-os/balena-raspberrypi/pull/1115

alexgg commented 7 months ago

Manual checks:

[x] EEPROM updated on RPi4 HUP from older version

Before HUP:

Apr 29 2021 17:11:25
version c2f8c388c4ee37ad709ace403467d163e8dd91ce (release)
timestamp 1619712685
update-time 1619712685
capabilities 0x0000001f

After HUP:

2023/05/11 07:26:03
version 4fd8f1f3f7a05f7756edb1d3f15ffd7e428981f5 (release)
timestamp 1683786363
update-time 0
capabilities 0x0000007f

[x] EEPROM not updated on RPI4 HUP from same version

Apr 10 17:16:00 0533069 1de0ac1f3683[3446]: [98-pieeprom][INFO] Updating SPI EEPROM firmware                  
Apr 10 17:16:00 0533069 1de0ac1f3683[3446]: flashrom v1.2 on Linux 5.15.92-v8 (aarch64)                                                                                                                                                     
Apr 10 17:16:00 0533069 1de0ac1f3683[3446]: flashrom is free software, get the source code at https://flashrom.org
Apr 10 17:16:00 0533069 1de0ac1f3683[3446]:                                                                           
Apr 10 17:16:00 0533069 1de0ac1f3683[3446]: Using clock_gettime for delay loops (clk_id: 1, resolution: 1ns).
Apr 10 17:16:00 0533069 1de0ac1f3683[3446]: Found Winbond flash chip "W25X40" (512 kB, SPI) on linux_spi.                                                                                                                                   
Apr 10 17:16:01 0533069 1de0ac1f3683[3446]: Reading flash... done.                                            
Apr 10 17:16:01 0533069 1de0ac1f3683[3446]: [pieeprom-update.sh][INFO] SPI EEPROM fw update is not necessary
Apr 10 17:16:01 0533069 1de0ac1f3683[3446]: [99-resin-uboot][INFO] Switching uboot root partition index to 3...    
Apr 10 17:16:01 0533069 1de0ac1f3683[3446]: [99-resin-uboot][INFO] Done.

[x] EEPROM updated on CM4 HUP from older version

Before HUP:

root@45b953a89e33:/# vcgencmd bootloader_version
2022/08/02 16:55:05
version 91b6280c53c0b6dff9e3d58810f439e577408845 (release)
timestamp 1659455705
update-time 1659632025
capabilities 0x0000007f

HUP:

Apr 10 18:53:29 d246f63 5a7ef44a811e[3624]: flashrom v1.2 on Linux 5.15.92-v8 (aarch64)
Apr 10 18:53:29 d246f63 5a7ef44a811e[3624]: flashrom is free software, get the source code at https://flashrom.org
Apr 10 18:53:29 d246f63 5a7ef44a811e[3624]: 
Apr 10 18:53:29 d246f63 5a7ef44a811e[3624]: Using clock_gettime for delay loops (clk_id: 1, resolution: 1ns).
Apr 10 18:53:29 d246f63 5a7ef44a811e[3624]: Found Winbond flash chip "W25X40" (512 kB, SPI) on linux_spi.
Apr 10 18:53:30 d246f63 5a7ef44a811e[3624]: Reading flash... done.
Apr 10 18:53:30 d246f63 5a7ef44a811e[3624]: [pieeprom-update.sh][INFO] Performing SPI EEPROM fw update...
Apr 10 18:53:30 d246f63 5a7ef44a811e[3624]: flashrom v1.2 on Linux 5.15.92-v8 (aarch64)
Apr 10 18:53:30 d246f63 5a7ef44a811e[3624]: flashrom is free software, get the source code at https://flashrom.org
Apr 10 18:53:30 d246f63 5a7ef44a811e[3624]: 
Apr 10 18:53:30 d246f63 5a7ef44a811e[3624]: Using clock_gettime for delay loops (clk_id: 1, resolution: 1ns).
Apr 10 18:53:30 d246f63 5a7ef44a811e[3624]: Found Winbond flash chip "W25X40" (512 kB, SPI) on linux_spi.
Apr 10 18:53:31 d246f63 5a7ef44a811e[3624]: Reading old flash chip contents... done.
Apr 10 18:53:36 d246f63 5a7ef44a811e[3624]: Erasing and writing flash chip... Erase/write done.
Apr 10 18:53:38 d246f63 5a7ef44a811e[3624]: Verifying flash... VERIFIED.
Apr 10 18:53:38 d246f63 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/mailbox@7e00b840/bcm2835_audio/status
Apr 10 18:53:38 d246f63 5a7ef44a811e[3624]: [99-resin-uboot][INFO] Switching uboot root partition index to 3...
Apr 10 18:53:38 d246f63 5a7ef44a811e[3624]: [99-resin-uboot][INFO] Done.

After HUP:

2023/05/11 07:26:03
version 4fd8f1f3f7a05f7756edb1d3f15ffd7e428981f5 (release)
timestamp 1683786363
update-time 0
capabilities 0x0000007f

[x] EEPROM not updated on CM4 HUP from same version


Apr 11 13:52:13 7ca599e hostapp-update[8649]: [98-pieeprom][INFO] Updating SPI EEPROM firmware
Apr 11 13:52:13 7ca599e hostapp-update[8649]: flashrom v1.2 on Linux 5.15.92-v8 (aarch64)
Apr 11 13:52:13 7ca599e hostapp-update[8649]: flashrom is free software, get the source code at https://flashrom.org
Apr 11 13:52:13 7ca599e hostapp-update[8649]: Using clock_gettime for delay loops (clk_id: 1, resolution: 1ns).
Apr 11 13:52:13 7ca599e hostapp-update[8649]: Found Winbond flash chip "W25X40" (512 kB, SPI) on linux_spi.
Apr 11 13:52:13 7ca599e hostapp-update[8649]: Reading flash... done.
Apr 11 13:52:13 7ca599e hostapp-update[8649]: [pieeprom-update.sh][INFO] SPI EEPROM fw update is not necessary
Apr 11 13:52:13 7ca599e hostapp-update[8649]: [99-resin-uboot][INFO] Switching uboot root partition index to 2...
Apr 11 13:52:13 7ca599e hostapp-update[8649]: [99-resin-uboot][INFO] Done.

- [x] Force EEPROM programming failure and check HUP still works - for backwards compatibility

ID="balena-os" NAME="balenaOS" VERSION="2.112.12" VERSION_ID="2.112.12" PRETTY_NAME="balenaOS 2.112.12" DISTRO_CODENAME="kirkstone" MACHINE="raspberrypicm4-ioboard" META_BALENA_VERSION="2.112.12" BALENA_BOARD_REV="e9308f8" META_BALENA_REV="2e4f0c95" SLUG="raspberrypicm4-ioboard"

balena run -it --privileged --device=/dev/vchiq ubuntu:latest bash -c 'apt update && apt install -y libraspberrypi-bin && vcgencmd bootloader_version' 2022/08/02 16:55:05 version 91b6280c53c0b6dff9e3d58810f439e577408845 (release) timestamp 1659455705 update-time 1659632025


HUP with a broken update script:

Apr 12 14:58:33 d7f7f8a f911b35176ce[3090]: [1-bootfiles][INFO] Deploying /mnt/boot/balenaos.fingerprint Apr 12 14:58:33 d7f7f8a f911b35176ce[3090]: [1-bootfiles][INFO] overwriting modified file /mnt/boot/balenaos.fingerp. Apr 12 14:58:33 d7f7f8a f911b35176ce[3090]: [80-rollback][INFO] Running rollback hook Apr 12 14:58:33 d7f7f8a f911b35176ce[3090]: [98-pieeprom][INFO] Updating SPI EEPROM firmware Apr 12 14:58:33 d7f7f8a f911b35176ce[3090]: [98-pieeprom][ERROR] Failed to update SPI EEPROM firmware Apr 12 14:58:33 d7f7f8a f911b35176ce[3090]: [99-resin-uboot][INFO] Switching uboot root partition index to 3... Apr 12 14:58:33 d7f7f8a f911b35176ce[3090]: [99-resin-uboot][INFO] Done.


After reboot:

ID="balena-os" NAME="balenaOS" VERSION="5.2.2" VERSION_ID="5.2.2" PRETTY_NAME="balenaOS 5.2.2" DISTRO_CODENAME="kirkstone" MACHINE="raspberrypicm4-ioboard" META_BALENA_VERSION="5.2.4" BALENA_BOARD_REV="0e62a0da4b21" META_BALENA_REV="906a26c224a3" SLUG="raspberrypicm4-ioboard"

And

2022/08/02 16:55:05 version 91b6280c53c0b6dff9e3d58810f439e577408845 (release) timestamp 1659455705 update-time 1659632025 capabilities 0x0000007f


- [x] EEPROM update on a RPI4/CM4 when kernel is updated - used to fail before as modules would not load (see 16ac8ab30bdb698d2f886eb6796ce413df3bd74e)

Updating from v2.112.12 with kernel v5.15 to `alexgg/eeprom-flash` over balenaOS v5.2.2 and kernel version 5.15.92.

ID="balena-os" NAME="balenaOS" VERSION="2.112.12" VERSION_ID="2.112.12" PRETTY_NAME="balenaOS 2.112.12" DISTRO_CODENAME="kirkstone" MACHINE="raspberrypicm4-ioboard" META_BALENA_VERSION="2.112.12" BALENA_BOARD_REV="e9308f8" META_BALENA_REV="2e4f0c95" SLUG="raspberrypicm4-ioboard"

balena run -it --rm --privileged --device=/dev/vchiq ubuntu:latest bash -c 'apt update && apt install -y libraspberrypi-bin && vcgencmd bootloader_version' 2022/08/02 16:55:05 version 91b6280c53c0b6dff9e3d58810f439e577408845 (release) timestamp 1659455705 update-time 1659632025 capabilities 0x0000007

Linux aaf4dc0 5.15.34-v8 #1 SMP PREEMPT Tue Apr 19 19:21:26 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux


HUP:

Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: [98-pieeprom][INFO] Updating SPI EEPROM firmware Apr 12 14:17:58 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/mailbs Apr 12 14:17:58 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/spi@7s Apr 12 14:17:58 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/spi@7s Apr 12 14:17:58 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/gpio@s Apr 12 14:17:58 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/gpio@n Apr 12 14:17:58 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/gpio@s Apr 12 14:17:58 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/gpio@s Apr 12 14:17:58 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/gpio@n Apr 12 14:17:58 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/gpio@s Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: flashrom v1.2 on Linux 5.15.34-v8 (aarch64) Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: flashrom is free software, get the source code at https://flashrom.org Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: Using clock_gettime for delay loops (clk_id: 1, resolution: 1ns). Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: Found Winbond flash chip "W25X40" (512 kB, SPI) on linux_spi. Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: Reading flash... done. Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: [pieeprom-update.sh][INFO] Performing SPI EEPROM fw update... Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: flashrom v1.2 on Linux 5.15.34-v8 (aarch64) Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: flashrom is free software, get the source code at https://flashrom.org Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: Using clock_gettime for delay loops (clk_id: 1, resolution: 1ns). Apr 12 14:17:58 aaf4dc0 e7a0fb3de036[2965]: Found Winbond flash chip "W25X40" (512 kB, SPI) on linux_spi. Apr 12 14:17:59 aaf4dc0 e7a0fb3de036[2965]: Reading old flash chip contents... done. Apr 12 14:18:05 aaf4dc0 e7a0fb3de036[2965]: Erasing and writing flash chip... Erase/write done. Apr 12 14:18:06 aaf4dc0 e7a0fb3de036[2965]: Verifying flash... VERIFIED. Apr 12 14:18:06 aaf4dc0 kernel: OF: overlay: WARNING: memory leak will occur if overlay removed, property: /soc/mailbs Apr 12 14:18:07 aaf4dc0 e7a0fb3de036[2965]: [99-resin-uboot][INFO] Switching uboot root partition index to 3... Apr 12 14:18:07 aaf4dc0 e7a0fb3de036[2965]: [99-resin-uboot][INFO] Done.

After HUP:

Linux aaf4dc0 5.15.92-v8 #1 SMP PREEMPT Wed Feb 8 16:47:50 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

balena run -it --rm --privileged --device=/dev/vchiq ubuntu:latest bash -c 'apt update && apt install -y libraspberrypi-bin && vcgencmd bootloader_version' 2023/05/11 07:26:03
version 4fd8f1f3f7a05f7756edb1d3f15ffd7e428981f5 (release) timestamp 1683786363
update-time 0
capabilities 0x0000007f



Note that rollbacks of EEPROM updates are not supported.

alexgg commented 7 months ago

@resin-jenkins retest this please

alexgg commented 7 months ago

lgtm

alexgg commented 7 months ago

Passed tests:

test-raspberrypi
test-raspberrypi3
test-raspberrypi3-64
test-raspberrypi4-64
test-revpi-core-3
test-raspberrypi0-2w-64

alexgg commented 7 months ago

@resin-jenkins retest this please

alexgg commented 7 months ago

@resin-jenkins retest this please

alexgg commented 7 months ago

@resin-jenkins retest this please

alexgg commented 7 months ago

@resin-jenkins retest this please

alexgg commented 7 months ago

Passed tests:

test-raspberrypi
test-raspberrypi3
test-raspberrypi3-64
test-raspberrypi4-64
test-revpi-core-3
test-rt-rpi-300
test-raspberrypi0-2w-64

alexgg commented 7 months ago

@resin-jenkins retest this please

alexgg commented 7 months ago

@resin-jenkins retest this please

alexgg commented 6 months ago

Devices with required tests that didn't pass:

test-raspberrypi4-unipi-neuron
test-revpi-connect

balena-os / balena-raspberrypi

Add EEPROM programming support #1118