Image padding contains invalid bytes

[joshuaspence]

Firmware updates aren't working for me for certain Hue devices after #212. The error message is Image padding contains invalid bytes

• LTD011
• LCA001
• LCA008
• LTG002
• 1743530P7

[stigvi]

I have the same issue. I don't wan't to destroy my light at christmas eve so I will not try anymore until next week :-)

[caelor]

Likewise with the following:

• 9290022169 (Ambiance E27)
• 929001953301 (Ambiance GU10)

[stickpin]

Same issue here with 8718699673147.

Here is the log:

Debug 2022-12-24 08:28:25getNewImage for '0x0017880108276c15', meta {"fileVersion":16786432,"fileSize":439622,"url":"https://otau.meethue.com/storage/ZGB_100B_0112/3ddd2b84-5fa0-4c7f-a695-e238c85ccaef/100B-0112-01002400-ConfLightBLE-Lamps-EFR32MG13.zigbee","sha512":"f30e0633129ff4569b644ac1edccd6bfe22e872e641520665c945fac6f41f3e757b333206e5d2c9ad0f2be042c0992159d55800c7c71bcb0c0514da877a206cf"}
Debug 2022-12-24 08:28:25ZigbeeOTA: downloading firmware image from https://otau.meethue.com/storage/ZGB_100B_0112/3ddd2b84-5fa0-4c7f-a695-e238c85ccaef/100B-0112-01002400-ConfLightBLE-Lamps-EFR32MG13.zigbee
Debug 2022-12-24 08:28:25OTA update checksum validation succeeded for '0x0017880108276c15'
Debug 2022-12-24 08:28:25getNewImage for '0x0017880108276c15', image header {"otaUpgradeFileIdentifier":{"type":"Buffer","data":[30,241,238,11]},"otaHeaderVersion":256,"otaHeaderLength":56,"otaHeaderFieldControl":0,"manufacturerCode":4107,"imageType":274,"fileVersion":16786432,"zigbeeStackVersion":2,"otaHeaderString":"\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000","totalImageSize":439622}
Debug 2022-12-24 08:28:25Update of 'Living Room Main WW Lamp 1' failed (AssertionError [ERR_ASSERTION]: Image padding contains invalid bytes)
Info 2022-12-24 08:28:25MQTT publish: topic 'zigbee2mqtt/Living Room Main WW Lamp 1', payload '{"brightness":254,"color":{"x":0.459,"y":0.408},"color_mode":"xy","color_temp":372,"last_seen":"2022-12-24T07:28:25.607Z","linkquality":98,"power_on_behavior":"off","state":"ON","update":{"installed_version":"1.93.11","state":"available"},"update_available":false}'
Info 2022-12-24 08:28:25MQTT publish: topic 'zigbee2mqtt/bridge/response/device/ota_update/update', payload '{"data":{"id":"Living Room Main WW Lamp 1"},"error":"Update of 'Living Room Main WW Lamp 1' failed (Image padding contains invalid bytes)","status":"error","transaction":"o43yv-3"}'
Error 2022-12-24 08:28:25Update of 'Living Room Main WW Lamp 1' failed (Image padding contains invalid bytes)
Debug 2022-12-24 08:28:25AssertionError [ERR_ASSERTION]: Image padding contains invalid bytes at validateSilabsGbl (/app/node_modules/zigbee-herdsman-converters/lib/ota/common.js:149:13) at validateImageData (/app/node_modules/zigbee-herdsman-converters/lib/ota/common.js:93:13) at getNewImage (/app/node_modules/zigbee-herdsman-converters/lib/ota/common.js:450:5) at runMicrotasks (<anonymous>) at processTicksAndRejections (node:internal/process/task_queues:96:5) at Object.updateToLatest (/app/node_modules/zigbee-herdsman-converters/lib/ota/common.js:286:19) at OTAUpdate.onMQTTMessage (/app/lib/extension/otaUpdate.ts:255:21)

Thanks a lot in advance and Merry Christmas! :)

[cturra]

i’m seeing the same with 929002478401.

[TheWanderingTurkey]

I'm also seeing this error with the following bulbs:

9290022166 9290022267 9290022266A

[Koenkk]

@TheJulianJES I think this is caused by https://github.com/Koenkk/zigbee-OTA/commit/d573deee8396192089a3db9d54088c00633804f3 . I will temporarily revert it.

[joshuaspence]

The revert worked, in that I am no longer seeing errors or any firmware updates available. Happy to help debug whatever is the problem with the new firmware versions

[TheJulianJES]

Hmm, I did update one of those listed models with ZHA and had no issues, but I think zigpy has a special case for certain Hue images which it normally didn't need to use.

So I guess Hue somewhat changed their format

I'll try to have a closer look later/in the next few days.

(The "affected images" are all Silabs images from what I can see. Signify updated those newer lights to ZCL R8 and maybe those EZSP versions use a different format.)

[dawiinci]

Seems to only affect updates from 1.93.11. Other updates went smoothly.

[red-island]

Have the same for 929001821618 and 9290020399.

Tried to update 929001821618 from app using bluetooth: works. Now getting the expected "firmware is newer" error when trying with zigbee ota.

Maybe the app uses different image or bluetooth/app makes the difference.

[TheJulianJES]

The "Hue Bluetooth" images are the exact same images. It's likely just that Z2M isn't parsing this new format correctly. The updates installed just fine when using ZHA.

[TheJulianJES]

(Although the issue seems to be in zigbee-herdsman-converters, I'm posting here now to avoid duplicating this issue.) The newer Hue images (which seem to report zcl_version=8) aren't correctly parsed by Z2M.

zigpy (used by ZHA) validates the image successfully:

% zigpy ota info 100B-0112-01002400-ConfLightBLE-Lamps-EFR32MG13.zigbee

Header: OTAImageHeader(upgrade_file_id=200208670, header_version=256, header_length=56, field_control=<FieldControl.0: 0>, manufacturer_id=4107, image_type=274, file_version=16786432, stack_version=2, header_string='', image_size=439622, *device_specific_file=False, *hardware_versions_present=False, *key=ImageKey(manufacturer_id=4107, image_type=274), *security_credential_version_present=False)
Number of subelements: 33
Validation result: ValidationResult.VALID

zigpy's validation for GBL images is done here: https://github.com/zigpy/zigpy/blob/5d05b68b4ebd157aaa4e61ea84e6944fcb530d64/zigpy/ota/validators.py#L73-L111

Z2M's code fails here when checking for padding: https://github.com/Koenkk/zigbee-herdsman-converters/blob/e17363841acc5cbd6314b7cc0a669b6b349b3d7e/lib/ota/common.js#L149

zigpy only seems to check padding for EBL images (and not GBL images): https://github.com/zigpy/zigpy/blob/5d05b68b4ebd157aaa4e61ea84e6944fcb530d64/zigpy/ota/validators.py#L58-L59

So I guess Z2M could just drop the padding check for GBL images?

Image used for testing (that apparently fails Z2M's current validation): https://otau.meethue.com/storage/ZGB_100B_0112/3ddd2b84-5fa0-4c7f-a695-e238c85ccaef/100B-0112-01002400-ConfLightBLE-Lamps-EFR32MG13.zigbee

[Koenkk]

@TheJulianJES thanks for your analysis!

@pedrolamas since you added the GBL validation, is this something you can check? (the above post from @TheJulianJES contains all the info needed).

[pedrolamas]

You definitely got my attention!! I'll take a look ASAP, possibly even today! 🙂

[pedrolamas]

I've taken a first look, seems this file is using 0xFF as padding instead of 0x0!

I'm trying to find where did I see on the standard that these should be 0x0 or if I just assumed it to be the case as with EBL... if this was just an incorrect assumption of mine, seems safe to just remove that padding assertion and call it a day!

I'll run a few more tests and then send a PR with the conclusions!

[pedrolamas]

PR created to remove the block of code that validates the GBL padding.

The PR includes the OTA file as a new test case.

[Koenkk]

@pedrolamas great

The new images that were added by @TheJulianJES are available again. Update to the latest dev or wait for the 1 Jan release to apply the updates.

[MattWestb]

@pedrolamas Then padding with 0xFF and flashing the file its not changing the default not written state of the flash that is 0xFF after flash erase if not deleting it. I can being that If using 0x00 the file is writing the last unused bits to the flash and its not wanted / needed and can doing problems in the device.

Then HUE is encrypting there OTA files it shall being OK as long the device is accepting it it shall being safe then the device is not flashing it if not can paring it OK.

I have not reading the latest Silabs papers (GSDK 4.X vesions) but i think HUE is having the same problems as IKEA is having then moving to GSDK 4.X from EZSP SDK 3.2 for new devices, then IKEA have releasing the last OTA files (Styrbar) with wrong version in the OTA meta data but the correct version inside the GBL / EBL file.

[pedrolamas]

Thanks for adding that @MattWestb (I have to admit took me a bit to understand what you wrote), I tried checking the latest Silabs docs but their website is reporting errors (502 mainly) all over the place...

I'm hoping that devices processing OTA data will just ignore anything after the end tag with the CRC check, but obviously there are no guarantees of that.

[MattWestb]

I was finding one interesting thing in https://www.silabs.com/documents/public/user-guides/ug489-gecko-bootloader-user-guide-gsdk-4.pdf how Silabs is doing there GBL files and the tagging. GBL End Tag 0xFC0404FC This tag indicates the end of the GBL file. It contains a 32-bit CRC for the entire file as an integrity check. The CRC is a non cryptographic check. This must be the last tag.

So if have configured the bootloader OK it shall not flashing the GBL file inside the OTA container if the CRC i not OK.

But reading the bootloader config papers is not one must checking the CRC in the bootloader code so in the end its depending if HUE have setting it or not. But i think is not relevant then using encrypted OTA files then one more check is being done then decrying it before the bootlooader is flashing it but i have reading that part.

Perhaps its getting more information in the above and below linked docks how its working with encrypted OTA / GBL files.

For making and parsing GBL files https://www.silabs.com/documents/public/user-guides/ug162-simplicity-commander-reference-guide.pdf.

[pedromrg]

Issue here with https://www.zigbee2mqtt.io/devices/929002241201.html#philips-929002241201

[red-island]

@pedrolamas great

The new images that were added by @TheJulianJES are available again. Update to the latest dev or wait for the 1 Jan release to apply the updates.

Just wanted to confirm the fix from end-user perspective ... running docker zigbee2mqtt:latest-dev ... version is 1.28.4-dev commit: d8f1f3e.

OTA returns "no updates available" for e.g. 929001821618

Logs show new image detected, but still result in no update available

[red-island]

@pedrolamas great The new images that were added by @TheJulianJES are available again. Update to the latest dev or wait for the 1 Jan release to apply the updates.

Just wanted to confirm the fix from end-user perspective ... running docker zigbee2mqtt:latest-dev ... version is 1.28.4-dev commit: d8f1f3e.

OTA returns "no updates available" for e.g. 929001821618

Logs show new image detected, but still result in no update available

Just wanted to confirm that I can start the update to the new image by putting manually a message into MQTT zigbee2mqtt/bridge/request/device/ota_update/update - but not working from the UI

[pedrolamas]

For making and parsing GBL files silabs.com/documents/public/user-guides/ug162-simplicity-commander-reference-guide.pdf.

For anyone interested, I have a fully working EBL/GBL parser/validator (including CRC checks) written in TypeScript, available at https://github.com/pedrolamas/silabs-image-parser

[bilbon0bud]

hello

same problem for the hue 4090331p9 (4090331P9

Best Regards

[Mosher23]

Same problem here with 3216331p6

[TheJulianJES]

Like previously mentioned here and here, Zigbee2MQTT version 1.29 will come out 2023/01/01 and include the bugfix for updating newer Hue lights to the latest firmware.

For now, just don't apply the update to the affected bulbs.