search

OE4T / meta-tegra

BSP layer for NVIDIA Jetson platforms, based on L4T
MIT License
410 stars 225 forks source link

OTA Blob update failed. Status: 3 #367

Closed lfdmn closed 4 years ago

lfdmn commented 4 years ago

Hi,

I had bl_update_payload deployment working on warrior, now moving to zeus 32.3.1.

I've updated the partition template to match the new one and added back my custom partitions.

I use tegra-bup-payload_1.0.bb and I strip the filename tags from the partition template for the when signing the bootloader payload.

I have not set any encryption key but I noticed that the bootloader partitions now all have .encrypt suffix. I don't know if this affects anything.

The image builds fine and I can flash the device.

I can find the /opt/ota_package/bl_update_payload size 44607776B.

Slot info before nv_update_engine call

jetson-xavier ota_package # nvbootctrl dump-slots-info
magic:0x43424e00,             version: 3             features: 3             num_slots: 2
slot: 0,             priority: 15,             suffix: _a,             retry_count: 7,             boot_successful: 1
slot: 1,             priority: 14,             suffix: _b,             retry_count: 7,             boot_successful: 1

Then the update fails with status 3

jetson-xavier ota_package # nv_update_engine -i
Nvidia A/B-Redundancy Update tool Version 1.2
current slot 0
Start updating slot 1
Tegra User Block Device: /dev/disk/by-partlabel
Tegra Boot Block Device: /dev/mmcblk0boot0
HEADER: MAGIC NVIDIA__BLOB__V2
HEX_VALUE 131072
BLOB_SIZE 44607776
HEADER_SIZE 48
NUMBER_OF_ELEMENTS 36
HEADER_TYPE 0
UNCOMP_SIZE 44607776
MB1_RATCHET_LV 0
MTS_RATCHET_LV 0
ROLLBACK_FUSE_LV 0
Device TN Spec: 2888-400-0004-G.0-1-2-jetson-xavier-mmcblk0p1
OTA Blob update failed. Status: 3
/usr/sbin/nv_bootloader_payload_updater --no-dependent-partition failed.
Slot 1 update failed

And the slot info shows

jetson-xavier ota_package # nvbootctrl dump-slots-info
magic:0x43424e00,             version: 3             features: 3             num_slots: 2
slot: 0,             priority: 15,             suffix: _a,             retry_count: 7,             boot_successful: 1
slot: 1,             priority: 0,              suffix: _b,             retry_count: 0,             boot_successful: 0

Any idea what could be the error? Status: 3 doesn't tell much and there's nothing I can find in the l4T Development Guide

madisongh commented 4 years ago

Try running the update under strace to see what it's trying to do when it reports the 'Status: 3'. Something like:

# strace -ff -onvup -s256 /usr/sbin/nv_update_engine --install no-reboot

Then look for the 'Status: 3' in the nvup.<PID> files that get generated. That's what I've done to troubleshoot these problems.

lfdmn commented 4 years ago

Thanks!

I've attached the nv_bootloader_payload_updater pid file. nvup.6793.txt if someone could send me one for a successful run I'd appreciate and could compare.

I don't find anything. /opt/ota_package/bl_update_payload is read /etc/nv_boot_control.conf is read and nv_bootloader_payload_updater exits without other external calls :/

madisongh commented 4 years ago

Hmm. Very mysterious. You did a full flash with R32.3.1 first, right?

lfdmn commented 4 years ago

Yep I did. I'll try your test distro to see if I get the same.

Signing off for summer break, back in 4 week.

lfdmn commented 4 years ago

I get the same with your test distro.

I used zeus-32.3.1 branch, fixed some missing symlinks in the layers directory and added tegra-bup-payload to IMAGE_INSTALL in tegra-sysinstall.bb

IMAGE_INSTALL = "packagegroup-core-boot tegra-sysinstall-tools haveged tegra-bup-payload"

Flashed, enabled AB with nv_update_engine and tried nv_update_engine -i no-reboot.

Same Status: 3 :/

I'll try building from a different PC now see if there's any difference.

madisongh commented 4 years ago

I just took a closer look at the strace output you sent me, and I think the problem is that your Xavier has a board revision that isn't covered by the update payload.

In R32.3.1, the only board revision covered by the BUP generator is F.0, whereas your board is revision G.0. That's probably what's triggering the error.

In L4T R32.4.2, NVIDIA has changed the specs for the 2888-400-0004 models to drop the board revision as a match criterion completely.

Try changing this section in meta-tegra/conf/machine/include/tegra194.inc:

TEGRA_BUPGEN_SPECS ?= "fab=400;boardsku=0001;boardrev=H.0 \
                       fab=400;boardsku=0001;boardrev=J.0 \
                       fab=400;boardsku=0001;boardrev=L.0 \
                       fab=400;boardsku=0004;boardrev=F.0"

so that last line ends with boardrev= (i.e., just delete the F.0).

lfdmn commented 4 years ago

Spot on!

That works :) Thanks a lot!

I go on holiday before I start testing something else now :D

madisongh commented 4 years ago

I've pushed that change to zeus-l4t-r32.3.1 and dunfell.