Koenkk / zigbee-herdsman-converters

Collection of device converters to be used with zigbee-herdsman
MIT License
924 stars 3.07k forks source link

Osram/Ledvance OTA updates support #1000

Closed pedrolamas closed 4 years ago

pedrolamas commented 4 years ago

I've been taking a look at the Osram/Ledvance OTA updates and how we can implement this and it does seem quite trivial as they provide a fully open and very well documented API!

I've started a branch in my fork here: https://github.com/PedroLamas/zigbee-herdsman-converters/tree/pedrolamas/osram-ota

Problem is that I currently don't have any Osram device, so I'm asking anyone interested in helping to try my branch (quite trivial if you are using Docker) and report back the debug values from a zigbee2mqtt/bridge/ota_update/check call.

Thanks in advance!

kiall commented 4 years ago

I have a couple of the SMART+ Motion Sensors so just checked:

zigbee2mqtt:debug 2020-02-20 09:57:31: ++ meta to send '{"fieldControl":0,"manufacturerCode":4174,"imageType":48,"fileVersion":503337744}'

{"id":60,"type":"EndDevice","ieeeAddr":"0x000d6f0011087674","nwkAddr":13006,"manufId":4174,"manufName":"CentraLite","powerSource":"Battery","modelId":"Motion Sensor-A","epList":[1,2],"endpoints":{"1":{"profId":260,"epId":1,"devId":1026,"inClusterList":[0,1,3,32,1026,1280,2821],"outClusterList":[25],"clusters":{"genBasic":{"attributes":{"modelId":"Motion Sensor-A","manufacturerName":"CentraLite","powerSource":3,"zclVersion":1,"stackVersion":2,"hwVersion":1,"dateCode":"����������������"}},"msTemperatureMeasurement":{"attributes":{"measuredValue":1562}},"genPowerCfg":{"attributes":{"batteryAlarmState":0}}},"binds":[{"cluster":1026,"type":"endpoint","deviceIeeeAddress":"0x00124b001caabd5d","endpointID":1},{"cluster":1,"type":"endpoint","deviceIeeeAddress":"0x00124b001caabd5d","endpointID":1}]},"2":{"profId":260,"epId":2,"devId":263,"inClusterList":[0,1,3,1030,2821],"outClusterList":[3],"clusters":{},"binds":[]}},"stackVersion":2,"hwVersion":1,"dateCode":"����������������","zclVersion":1,"interviewCompleted":true,"meta":{"configured":1},"lastSeen":1582101517569} pi@zigbee2mqtt:~/zigbee2mqtt $

andreasbrett commented 4 years ago

Thanks for implementing this. I have some E27 bulbs (warm white and rgbw) and smart plugs that I will test this weekend.

pedrolamas commented 4 years ago

Thanks for sending the data @kiall!

The values for manufacturerCode and imageType match my expectations, now I just need to see how the fileVersion translates to a major.minor.build type version!

pedrolamas commented 4 years ago

@kiall seems your device is made by CentraLite while the reference API we currently have is for devices made by Ledvance... we will need to find an API for those in order to support them. I'm updating the title of this thread to be more accurate.

The values returned from https://api.update.ledvance.com/v1/zigbee/products seems to indicate that only manufacturerCode with 4489 or 4364 are supported by this API!

kiall commented 4 years ago

@kiall seems your device is made by CentraLite while the reference API we currently have is for devices made by Ledvance... we will need to find an API for those in order to support them. I'm updating the title of this thread to be more accurate.

Poking through the available firmwares, it seems your right! I guess this complicates the devices.js updates as they are all jumbled up :) e.g. this is one of these:

   {
        zigbeeModel: ['Motion Sensor-A'],
        model: 'AC01353010G',
        vendor: 'OSRAM',
        description: 'SMART+ Motion Sensor',
        supports: 'occupancy, tamper and temperature',
        fromZigbee: [
            fz.temperature,
            fz.iaszone_occupancy_2,
        ],
        toZigbee: [],
        meta: {configureKey: 1},
        ota: ota.osram,
        configure: async (device, coordinatorEndpoint) => {
            const endpoint = device.getEndpoint(1);
            await bind(endpoint, coordinatorEndpoint, ['msTemperatureMeasurement', 'genPowerCfg']);
            await configureReporting.temperature(endpoint);
            await configureReporting.batteryPercentageRemaining(endpoint);
        },
    },
pedrolamas commented 4 years ago

I've had some success searching for older issues with either 4489 or 4364!

Specifically, this issue has a nice database entry:

{
  "id": 21,
  "type": "Router",
  "ieeeAddr": "0xf0d1b800001345e6",
  "nwkAddr": 34924,
  "manufId": 4489,
  "manufName": "LEDVANCE",
  "powerSource": "Mains (single phase)",
  "modelId": "Plug Z3",
  "epList": [
    1,
    242
  ],
  "endpoints": {
    "1": {
      "profId": 260,
      "epId": 1,
      "devId": 266,
      "inClusterList": [
        0,
        3,
        4,
        5,
        6,
        2821,
        4096,
        64527
      ],
      "outClusterList": [
        25
      ],
      "clusters": {
        "genBasic": {
          "attributes": {
            "modelId": "Plug Z3",
            "manufacturerName": "LEDVANCE",
            "powerSource": 1,
            "zclVersion": 2,
            "appVersion": 1,
            "stackVersion": 2,
            "hwVersion": 1,
            "dateCode": "Dec 10 2018\u0000\u0000\u0000\u0000\u0000",
            "swBuildId": "00103101"
          }
        }
      },
      "binds": [
        {
          "cluster": 6,
          "type": "endpoint",
          "deviceIeeeAddress": "0x00124b0014b92997",
          "endpointID": 1
        }
      ]
    },
    "242": {
      "profId": 41440,
      "epId": 242,
      "devId": 97,
      "inClusterList": [],
      "outClusterList": [
        33
      ],
      "clusters": {},
      "binds": []
    }
  },
  "appVersion": 1,
  "stackVersion": 2,
  "hwVersion": 1,
  "dateCode": "Dec 10 2018\u0000\u0000\u0000\u0000\u0000",
  "swBuildId": "00103101",
  "zclVersion": 2,
  "interviewCompleted": true,
  "meta": {
    "configured": 1
  }
}

The two values to retain from this is "modelId": "Plug Z3" and "swBuildId": "00103101".

I cross-checked that modelId with the values returned from https://api.update.ledvance.com/v1/zigbee/products, and found this:

{"id":{"company":4489,"product":45},"name":"Plug Z3"}

The company code above matches the expected 4489, but now I also know the product code is 45.

I then ran the following request:

POST https://api.update.ledvance.com/v1/zigbee/difference HTTP/1.1
User-Agent: Fiddler
Host: api.update.ledvance.com
Content-type: text/json
Content-Length: 131

[
  {
   "company": 4489,
   "product": 45,
   "version": {
     "major": 1,
     "minor": 0,
     "build": 0
    }
  }
]

Amongst the information returned, I found this:

"version":{"major":1,"minor":3,"build":101}}

This is basically the current matches the swBuildId we previously had, just by doing a string with the 3 values!

So swBuildId is "AAABBCC" where AAA is major, BB minor, and CC build!

Calimerorulez commented 4 years ago

If I have time this weekend, I'm glad to assist with three Smart+ Plugs.

pedrolamas commented 4 years ago

The API is so simple that... it's done!

I locally tested as much as possible and everything seems to be fine, but it would be great to have someone confirm that it actually works on retrieving the files!

@Koenkk should I create the PR now or wait for someone to actually confirm it works first?

There's also the problem @kiall above exposed: I have no idea how to identify which Osram devices have manufacturer code 4489 or 4364 by looking on the devices.js file... so I don't know which actually support this (though in the end, they will just receive a message saying that no firmware was found)

kiall commented 4 years ago

@Koenkk should I create the PR now or wait for someone to actually confirm it works first?

There's also the problem @kiall above exposed: I have no idea how to identify which Osram devices have manufacturer code 4489 or 4364 by looking on the devices.js file... so I don't know which actually support this (though in the end, they will just receive a message saying that no firmware was found)

It's probably no big deal, especially if there are some comments in the osram.js file to explain the mess OSRAM created 🤣 it'll get cleaned up if/when a firmware source is found.

Though, it might make sense to update Z2M to have OTA sources expose the manufacturer IDs they support, and remove the hardcoded OTA choice from devices.js.

Calimerorulez commented 4 years ago

Hi all,

I'm trying to help, and I am using @Koenkk's latest-dev docker image. I have created a volume with this repository, but now when I launch the docker container, I get the following error:

Using '/app/data' as data directory

> zigbee2mqtt@1.10.0-dev start /app

> node index.js

internal/modules/cjs/loader.js:984


  throw err;

  ^

Error: Cannot find module 'axios'

Require stack:

- /app/node_modules/zigbee-herdsman-converters/ota/salus.js

- /app/node_modules/zigbee-herdsman-converters/ota/index.js

- /app/node_modules/zigbee-herdsman-converters/devices.js

- /app/node_modules/zigbee-herdsman-converters/index.js

- /app/lib/util/utils.js

- /app/lib/util/settings.js

- /app/lib/util/logger.js

- /app/lib/mqtt.js

- /app/lib/controller.js

- /app/index.js

    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:981:15)

    at Function.Module._load (internal/modules/cjs/loader.js:863:27)

    at Module.require (internal/modules/cjs/loader.js:1043:19)

    at require (internal/modules/cjs/helpers.js:77:18)

    at Object.<anonymous> (/app/node_modules/zigbee-herdsman-converters/ota/salus.js:1:15)

    at Module._compile (internal/modules/cjs/loader.js:1157:30)

    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1177:10)

    at Module.load (internal/modules/cjs/loader.js:1001:32)

    at Function.Module._load (internal/modules/cjs/loader.js:900:14)

    at Module.require (internal/modules/cjs/loader.js:1043:19) {

  code: 'MODULE_NOT_FOUND',

  requireStack: [

    '/app/node_modules/zigbee-herdsman-converters/ota/salus.js',

    '/app/node_modules/zigbee-herdsman-converters/ota/index.js',

    '/app/node_modules/zigbee-herdsman-converters/devices.js',

    '/app/node_modules/zigbee-herdsman-converters/index.js',

    '/app/lib/util/utils.js',

    '/app/lib/util/settings.js',

    '/app/lib/util/logger.js',

    '/app/lib/mqtt.js',

    '/app/lib/controller.js',

    '/app/index.js'

  ]

}

npm ERR! code ELIFECYCLE

npm ERR! errno 1

npm ERR! zigbee2mqtt@1.10.0-dev start: `node index.js`

npm ERR! Exit status 1

How do I add the missing module 'axios' to @Koenkk's docker image?

pedrolamas commented 4 years ago

@Calimerorulez that is actually expected, you need to do npm ci or nom install after cloning the repository (doesn't matter is from Koenkk or from me, you need to do a fresh install of the dependencies)

BudBundi commented 4 years ago

@Calimerorulez that is actually expected, you need to do npm ci or nom install after cloning the repository (doesn't matter is from Koenkk or from me, you need to do a fresh install of the dependencies)

Same for me, but I can't run npm ci in the docker container because it restarts all the time

pedrolamas commented 4 years ago

No, don't do that on the container, do it on the host!

So just clone the repo, open a terminal on the root of the repo and run npm install.

twsl commented 4 years ago

If you created the container yourself, add RUN npm ci in the docker file before the entrypoint command

BudBundi commented 4 years ago

no npm on the host and do not feel like to mess with that

jasperro commented 4 years ago

I've tried checking for updates for two different OSRAM devices, but they both said there was no firmware file available. Failed to call 'OTAUpdate' 'onMQTTMessage' (AssertionError [ERR_ASSERTION]: No image available for manufacturerCode '4364' imageType '98' and Failed to call 'OTAUpdate' 'onMQTTMessage' (AssertionError [ERR_ASSERTION]: No image available for manufacturerCode '4489' imageType '17'

zigbee2mqtt:debug 2020-02-20 21:19:58: Got OTA request '{"fieldControl":1,"manufacturerCode":4489,"imageType":17,"fileVersion":1061121}'

With traceback: Feb 20 21:11:11 raspberry npm[9983]: at getImageMeta (/opt/zigbee2mqtt-dev/node_modules/zigbee-herdsman-converters/ota/osram.js:35:5) Feb 20 21:11:11 raspberry npm[9983]: at processTicksAndRejections (internal/process/task_queues.js:97:5) Feb 20 21:11:11 raspberry npm[9983]: at async isNewImageAvailable (/opt/zigbee2mqtt-dev/node_modules/zigbee-herdsman-converters/ota/osram.js:63:18) Feb 20 21:11:11 raspberry npm[9983]: at async Object.isUpdateAvailable (/opt/zigbee2mqtt-dev/node_modules/zigbee-herdsman-converters/ota/common.js:145:23) Feb 20 21:11:11 raspberry npm[9983]: at async OTAUpdate.onMQTTMessage (/opt/zigbee2mqtt-dev/lib/extension/otaUpdate.js:28:33) Feb 20 21:11:11 raspberry npm[9983]: at async Controller.callExtensionMethod (/opt/zigbee2mqtt-dev/lib/controller.js:329:21))

Calimerorulez commented 4 years ago

Running npm install worked :)

Now when I run the check for upgrades, there is no update for my Smart+ Plug apparently, or it is not supported?

zigbee2mqtt:info 2020-02-20 21:14:37: MQTT publish: topic 'zigbee2mqtt30/bridge/log', payload '{"type":"ota_update","message":"Device '0x7cb03eaa0a001856' does not support OTA updates","meta":{"status":"not_supported","device":"0x7cb03eaa0a001856"}}'

It is a "model":"Smart+ plug (AB3257001NJ)","manufacturer":"OSRAM"}

Calimerorulez commented 4 years ago

Ok, I must have done something wrong. I checked out your repository, but I'm missing the osram.js:

/opt/zigbee-herdsman-converters/ota $ ll
total 32
drwxr-xr-x 2 root root  4096 Feb 20 18:50 .
drwxr-xr-x 8 root root  4096 Feb 20 20:04 ..
-rw-r--r-- 1 root root 10734 Feb 20 18:50 common.js
-rw-r--r-- 1 root root    89 Feb 20 18:50 index.js
-rw-r--r-- 1 root root  2844 Feb 20 18:50 salus.js
-rw-r--r-- 1 root root  2180 Feb 20 18:50 tradfri.js

I have to switch to your branch. I'll test further :)

Calimerorulez commented 4 years ago
zigbee2mqtt:info  2020-02-20 21:50:57: Checking if update available for '0x7cb03eaa0a001856'

zigbee2mqtt:info  2020-02-20 21:50:57: MQTT publish: topic 'zigbee2mqtt30/bridge/log', payload '{"type":"ota_update","message":"Checking if update available for '0x7cb03eaa0a001856'","meta":{"status":"checking_if_available","device":"0x7cb03eaa0a001856"}}'

zigbee2mqtt:error 2020-02-20 21:50:58: Failed to check if update available for '0x7cb03eaa0a001856' (No image available for manufacturerCode '4364' imageType '39')

zigbee2mqtt:info  2020-02-20 21:50:58: MQTT publish: topic 'zigbee2mqtt30/bridge/log', payload '{"type":"ota_update","message":"Failed to check if update available for '0x7cb03eaa0a001856' (No image available for manufacturerCode '4364' imageType '39')","meta":{"status":"check_failed","device":"0x7cb03eaa0a001856"}}'
pedrolamas commented 4 years ago

@jasperro Thanks for testing!

I've checked your device with details {"fieldControl":1,"manufacturerCode":4489,"imageType":17,"fileVersion":1061121}, the message is correct, that is the latest version of the firmware for that device.

You can check by doing the following:

I think I need to improve the error messages...

pedrolamas commented 4 years ago

@Calimerorulez using the same approach as on my previous post, I can see that the latest available version for the manufacturerCode '4364' imageType '39' is version 0.20.509, which translates to a fileVersion 132361, can you check your logs for that number, please?

jasperro commented 4 years ago

@PedroLamas My other device (man: 0x110c | img: 0x0062) has version 1020510, also the latest version, so no update needed

pedrolamas commented 4 years ago

I've made some adjustments to ensure we always retrieve metadata for existing firmware and then check locally if this is a new version or not.

This will improve the debug messages so that "No new image available" will be returned when an image has been found but it is the same version as the current one on the device, and "No image available..." when there is no image at all.

pedrolamas commented 4 years ago

I've taken the list from https://api.update.ledvance.com/v1/zigbee/products and refactored it to what I believe is a simpler view:

{
  '4364': {
    '3': 'PAR16 50 TW',
    '4': 'Surface Light TW',
    '5': 'GARDENSPOT RGB',
    '6': 'CLASSIC A60 RGBW',
    '7': 'GARDENSPOT W',
    '8': 'CLA60 TW',
    '9': 'Surface Light W',
    '17': 'PAR16 RGBW',
    '19': 'CLA60 W CLEAR',
    '20': 'CLASSIC B40 TW',
    '39': 'Plug01 OnOff MK',
    '46': 'SubstiTube W MK',
    '90': 'GARDENPOLE RGBW LIGHTIFY',
    '91': 'LIGHTIFY OUTDOOR FLEX',
    '92': 'LIGHTIFY INDOOR FLEX',
    '98': 'CLA60 RGBW OSRAM',
    '99': 'CLA60 TW OSRAM',
    '101': 'MR16 TW OSRAM',
    '103': 'GARDENPOLE MINI RGBW OSRAM',
    '104': 'OUTDOOR LANTERN W RGBW OSRAM',
    '105': 'OUTDOOR LANTERN B50 RGBW OSRAM',
    '106': 'PANEL RGBW OSRAM',
    '107': 'CEILING TW OSRAM',
    '108': 'FLOOD LIGHT RGBW OSRAM',
    '110': 'OUTDOOR LANTERN B90 RGBW OSRAM'
  },
  '4489': {
    '12': 'A19 W 10 year',
    '13': 'A19 TW 10 year',
    '15': 'BR30 W',
    '16': 'PAR38 W 10 year',
    '17': 'CLA60 RGBW Z3',
    '25': 'A19 RGBW',
    '26': 'BR30 TW',
    '27': 'BR30 RGBW',
    '28': 'RT TW',
    '29': 'RT RGBW',
    '30': 'FLEX RGBW',
    '31': 'FLEX Outdoor RGBW',
    '32': 'Outdoor Accent Light RGB',
    '33': 'Convertible Undercabinet Light TW',
    '34': 'Flushmount TW',
    '35': 'Edge-Lit Under Cabinet',
    '42': 'Flex RGBW Z3',
    '44': 'Tibea TW Z3',
    '45': 'Plug Z3',
    '46': 'PAR16 TW Z3',
    '48': 'PAR16 RGBW Z3',
    '49': 'PAR16 DIM Z3',
    '51': 'B40 TW Z3',
    '52': 'B40 DIM Z3',
    '59': 'Gardenpole RGBW Z3',
    '60': 'A60 TW Z3',
    '61': 'A60 DIM Z3',
    '64': 'GardenpoleMini RGBW Z3',
    '70': 'Undercabinet TW Z3',
    '90': 'Panel TW Z3',
    '92': 'Outdoor FLEX RGBW Z3',
    '99': 'Panel TW HCL',
    '101': 'Downlight TW HCL'
  }
}

As such, these are the only devices supported by this API.

andreasbrett commented 4 years ago

@Calimerorulez using the same approach as on my previous post, I can see that the latest available version for the manufacturerCode '4364' imageType '39' is version 0.20.509, which translates to a fileVersion 132361, can you check your logs for that number, please?

Not sure if file versions are correctly interpreted for all devices. I already noticed some days ago that for some Osram/Ledvance devices file version are somewhat strange when comparing real world vs API numbering. See https://github.com/Koenkk/zigbee2mqtt/issues/2921#issuecomment-587152731

My "Plug 01" is on version v1.04.12 and latest version is V0.20 Build 509 My "CLA60 TW OSRAM" is on version v1.05.10 and latest version is V0.20 Build 510

So from the CLA60 I would assume that they're equal and the major version number "somehow" needs to be neglected or interpreted in a different way. Not sure why v1.x.y is v0.20 Build xy.

Here's my log output when checking for the "Plug 01".

info 2020-02-21 01:35:15: Checking if update available for 'kt_power_outlet' debug 2020-02-21 01:35:15: Check if update available for '0x84182600000xxxxx' (Plug 01) debug 2020-02-21 01:35:15: Using endpoint '3' debug 2020-02-21 01:35:15: Got OTA request '{"fieldControl":0,"manufacturerCode":4364,"imageType":39,"fileVersion":16909330}' debug 2020-02-21 01:35:15: Is new image available for '0x84182600000xxxxx', current '{"fieldControl":0,"manufacturerCode":4364,"imageType":39,"fileVersion":16909330}', latest meta '{"fileVersion":132361,"fileSize":121680,"url":"https://api.update.ledvance.com/v1/zigbee/firmwares/download?company=4364&product=39&version=0.20.509"}' debug 2020-02-21 01:35:15: Updata available for '0x84182600000xxxxx': NO info 2020-02-21 01:35:15: No update available for 'kt_power_outlet'

Would be good if in cases like these where the device has a firmware that is newer than the newest firmware provided by the manufacturer there'd be a info/warning in the log.

BTW: I can only help you out with the Plug01 as all my other Osram devices are already up2date.

Calimerorulez commented 4 years ago

I did not know that I had to enable debug-logging to see the version numbers, here is my log for my plug:

zigbee2mqtt:debug 2020-02-21 08:38:37: Received MQTT message on 'zigbee2mqtt30/bridge/ota_update/check' with data '0x7cb03eaa0a001856'

zigbee2mqtt:info  2020-02-21 08:38:37: Checking if update available for '0x7cb03eaa0a001856'

zigbee2mqtt:info  2020-02-21 08:38:37: MQTT publish: topic 'zigbee2mqtt30/bridge/log', payload '{"type":"ota_update","message":"Checking if update available for '0x7cb03eaa0a001856'","meta":{"status":"checking_if_available","device":"0x7cb03eaa0a001856"}}'

zigbee2mqtt:debug 2020-02-21 08:38:37: Check if update available for '0x7cb03eaa0a001856' (Plug 01)

zigbee2mqtt:debug 2020-02-21 08:38:37: Using endpoint '3'

zigbee2mqtt:debug 2020-02-21 08:38:37: Received Zigbee message from '0x7cb03eaa0a001856', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":0,"manufacturerCode":4364,"imageType":39,"fileVersion":16909330}' from endpoint 3 with groupID 0

zigbee2mqtt:debug 2020-02-21 08:38:37: No converter available for 'AB3257001NJ' with cluster 'genOta' and type 'commandQueryNextImageRequest' and data '{"fieldControl":0,"manufacturerCode":4364,"imageType":39,"fileVersion":16909330}'

zigbee2mqtt:debug 2020-02-21 08:38:37: Got OTA request '{"fieldControl":0,"manufacturerCode":4364,"imageType":39,"fileVersion":16909330}'

zigbee2mqtt:debug 2020-02-21 08:38:38: Is new image available for '0x7cb03eaa0a001856', current '{"fieldControl":0,"manufacturerCode":4364,"imageType":39,"fileVersion":16909330}', latest meta '{"fileVersion":132361,"fileSize":121680,"url":"https://api.update.ledvance.com/v1/zigbee/firmwares/download?company=4364&product=39&version=0.20.509"}'

zigbee2mqtt:debug 2020-02-21 08:38:38: Updata available for '0x7cb03eaa0a001856': NO

zigbee2mqtt:info  2020-02-21 08:38:38: No update available for '0x7cb03eaa0a001856'

zigbee2mqtt:info  2020-02-21 08:38:38: MQTT publish: topic 'zigbee2mqtt30/bridge/log', payload '{"type":"ota_update","message":"No update available for '0x7cb03eaa0a001856'","meta":{"status":"not_available","device":"0x7cb03eaa0a001856"}}'

zigbee2mqtt:info  2020-02-21 08:38:38: MQTT publish: topic 'zigbee2mqtt30/0x7cb03eaa0a001856', payload '{"state":"OFF","linkquality":102,"update_available":false}'

So no update available :)

pedrolamas commented 4 years ago

Thanks @andreasbrett for sending that, and you are absolutely right!

Your Plug01 device is returning "manufacturerCode":4364,"imageType":39,"fileVersion":16909330.

On the Levance website I can see that the available firmware is version 0.20.509, and by downloading the image and analysing it, it reports a fileVersion of 16909577, so the good news is that an update is available for that device (and the same here applies to @Calimerorulez as your logs show the exact same values!)

So the question is how we translate fileVersion 16909577 to version 0.20.509. My current approach was to convert 16909577 to hex ("01020509") and substring this as "AAABBCCC", but that approach does NOT work here as this would return version 10.20.509 when we already know it is actually 0.20.509... so the real question is, what is that 1 and how do we get to it?

I'm going to download a few more images and compare the fileVersions and indicated versions to see if I can find a pattern...

pedrolamas commented 4 years ago

Seems that the extra "1" is for the 4364 manufacturer code only... In any case, I found another way around this!

{
    "blob": null,
    "identity": {
    "company": 4364,
    "product": 39,
    "version": {
        "major": 0,
        "minor": 20,
        "build": 509
    }
    },
    "releaseNotes": "1. Stack upgrade;\r\n2. Improve network performance.",
    "shA256": "c70126e43666f0a83077e8d93c587b7c0cefcd37d4719ebbeab1e33090b7c3b5",
    "name": "ZLL_Plug01_OnOff_MK_0x01020509.ota",
    "productName": "Plug01 OnOff MK",
    "fullName": "Plug01 OnOff MK/01020509/ZLL_Plug01_OnOff_MK_0x01020509.ota",
    "extension": ".ota",
    "released": "2019-03-14T06:21:58",
    "salesRegion": "eu",
    "length": 121680
},

The above is the metadata returned for manufacturerCode 4364, imageType 39, and I can see clearly that the fileVersion is actually part of the fullName field returned, so I'm going to start using that instead of trying to figure out how to go from version->fileVersion

I've confirmed this is the case in all other results from the service!

kiall commented 4 years ago

@PedroLamas maybe this helps? No guarantees anyone is actually following it though.

Screenshot_20200221-100308

Screenshot from https://zigbeealliance.org/wp-content/uploads/2019/12/07-5123-06-zigbee-cluster-library-specification.pdf

pedrolamas commented 4 years ago

Thanks @kiall unfortunately I can see it doesn't match as the spec recommends the last 2 bytes to be the stack version and that bit I know it actually matches the build.

In any case, I just pushed an update that will parse the fileVersion from the fullName element so I'd expect anyone with Plug01 "manufacturerCode":4364,"imageType":39,"fileVersion":16909330 to have an update available!

If you have one of these plugs, please pull the latest and run a "check", if it returns true, then do the "update" next!

Be aware that I also renamed any "osram" reference on my code to the more accurate "ledvance".

BudBundi commented 4 years ago

I tested the new build, with two Plug 01, both updated without a problem. For me it looks like you can open a PR

Calimerorulez commented 4 years ago

Mine just updated fine too! Thank you all!

pedrolamas commented 4 years ago

My PR just got merged into the dev branch, so closing this now as resolve!

Kosta-Github commented 4 years ago

I am seeing this error regularly now (within zigbee2mqtt):

Failed to call 'OTAUpdate' 'onZigbeeEvent' (AssertionError [ERR_ASSERTION]: No image available for manufacturerCode '4174' imageType '48'
     at getImageMeta (/app/node_modules/zigbee-herdsman-converters/ota/ledvance.js:15:5)
     at runMicrotasks (<anonymous>)
     at processTicksAndRejections (internal/process/task_queues.js:97:5)
     at async isNewImageAvailable (/app/node_modules/zigbee-herdsman-converters/ota/ledvance.js:48:18)
     at async Object.isUpdateAvailable (/app/node_modules/zigbee-herdsman-converters/ota/common.js:168:23)
     at async OTAUpdate.onZigbeeEvent (/app/lib/extension/otaUpdate.js:37:31)
     at async Controller.callExtensionMethod (/app/lib/controller.js:352:21))

Is this something I can help debugging/fixing or is this an already known issue?

I have 2 types of OSRAM Smart+/ledvance devices:

ArcherSimple commented 4 years ago

Seeing same issue regularly in zigbee2mqtt 1.16.1 log.

Failed to call 'OTAUpdate' 'onZigbeeEvent' (AssertionError [ERR_ASSERTION]: No image available for manufacturerCode '4489' imageType '37'
     at getImageMeta (/opt/zigbee2mqtt/node_modules/zigbee-herdsman-converters/ota/ledvance.js:17:5)
     at runMicrotasks (<anonymous>)
     at processTicksAndRejections (internal/process/task_queues.js:97:5)
     at async isNewImageAvailable (/opt/zigbee2mqtt/node_modules/zigbee-herdsman-converters/ota/common.js:263:18)

I have 2 types of OSRAM Smart+/ledvance devices:

My first time to post so please let me know if there is more info I should post or if this is in correct place. Thank you!

ZZJHONS commented 1 year ago

Experiencing the same red message when checking OTA:

2022-12-02 23:31:49 Failed to check if update available for 'Osram 1' (No image available for manufacturerCode '4489' imageType '17')

pedrolamas commented 1 year ago

@ZZJHONS the complete URL seems to work correctly: https://api.update.ledvance.com/v1/zigbee/firmwares/newer?company=4489&product=17&version=0.0.0

Check again, and if this is still happening, we might need to take a closer look!

Pferdebockwurst commented 1 year ago

I've just tried to update an older OSRAM Smart+ Plug (Model AB3257001NJ). The OTA failed with the error message "No image available for manufacturerCode '4364' imageType '39' " The device is on firmware version 1.04.12.

pedrolamas commented 1 year ago

@Pferdebockwurst in terms of URL, that also seems to work fine: https://api.update.ledvance.com/v1/zigbee/firmwares/newer?company=4364&product=39&version=0.0.0

Can you try to do a curl "https://api.update.ledvance.com/v1/zigbee/firmwares/newer?company=4364&product=39&version=0.0.0" directly from where you are running Z2M?

You should get something looking like this:

image

Pferdebockwurst commented 1 year ago

@Pferdebockwurst in terms of URL, that also seems to work fine: https://api.update.ledvance.com/v1/zigbee/firmwares/newer?company=4364&product=39&version=0.0.0

Can you try to do a curl "https://api.update.ledvance.com/v1/zigbee/firmwares/newer?company=4364&product=39&version=0.0.0" directly from where you are running Z2M?

You should get something looking like this:

image

I did, and this is the result, on the machine that is running Home Assistant and Zigbee2mqtt:

Screenshot_20221203-175038

pedrolamas commented 1 year ago

Thanks for checking @Pferdebockwurst, given the above I fail to see how this assert is going false...

Pferdebockwurst commented 1 year ago

Thanks for checking @Pferdebockwurst, given the above I fail to see how this assert is going false...

I don't know either. @Koenkk is there a recent change that might affect how these devices try to check for an firmware update? I have never had this kind of error when trying to update these devices before.

ZZJHONS commented 1 year ago

Hello I tried the above in HA Terminal but with my data for the bulb changed and got this:

image

The firmware version showing is 01066400 but shows black not clickable:

image

PD: I think I added this bulb a few days ago and it did update from a 2017 firmware to a new one of Nov 26 2021, but yesterday I added a philips bulb and when checking OTA this error for the Osram+ appeared.

EDIT: In between Z2M updated twice 1.28.2 to 1.28.3 and to 1.28.4, so in one of those might be a breaking change?

pedrolamas commented 1 year ago

I wonder if this problem is now fixed with #5069... 🤔

Pferdebockwurst commented 1 year ago

I wonder if this problem is now fixed with #5069... 🤔

I'd like to know too, but I can't check it because I'm not using the dev branch.

pedrolamas commented 1 year ago

Well, I now know that #5069 did not fix the problem as that fix is only applied if a proxy is in use, however I managed to confirm that it is in fact the problem!

I added a "quick & dirty" test locally to just download from one of the above URL's and can see the data is broken, seems to be this issue with axios, but the proposed solution of sending Accept-Encoding: * did not fix the problem for me!

image

@Koenkk axios v1.x has shown for a while that it has some serious issues (including this last one), so I would seriously consider downgrading back to v0.x

This is the result of running exactly the same test (no code change) with axios v0.27.2:

image

Though the test failed (expected), I can clearly see all the data there.

Koenkk commented 1 year ago

@pedrolamas reading https://github.com/axios/axios/issues/5328 it seems this problem has been introduced in 1.2.0, I've locked it to 1.1.3 now. Can you check if this fixed the issue?

Changes will be available in the dev branch in a few hours from now. (https://www.zigbee2mqtt.io/advanced/more/switch-to-dev-branch.html)

pedrolamas commented 1 year ago

I ran a quick test with 1.1.3 and indeed it seems to work fine with that version!

stonie81 commented 1 year ago

Hi, after updating to 1.29.0 the OTA error of 1.28.4 is gone, but my light won't update. I have a FLEX RBGW Z3 and the test per URL brings up 2 newer versions (the z2m frontend shows me Nov 27 2018 as date of the firmware) https://api.update.ledvance.com/v1/zigbee/firmwares/newer?company=4489&product=17&version=0.0.0

Maybe the frontend lie? Help would be appreciated. :-)

zigbee2mqtt | Zigbee2MQTT:debug 2023-01-03 11:36:48: Received MQTT message on 'zigbee2mqtt/bridge/request/device/ota_update/check' with data '{"id":"Lichtschlauch_Durchgangszimmer","transaction":"lomgz-11"}' zigbee2mqtt | Zigbee2MQTT:info 2023-01-03 11:36:48: Checking if update available for 'Lichtschlauch_Durchgangszimmer' zigbee2mqtt | Zigbee2MQTT:debug 2023-01-03 11:36:48: Check if update available for '0xf0d1b800001b0362' (FLEX RGBW Z3) zigbee2mqtt | Zigbee2MQTT:debug 2023-01-03 11:36:48: Using endpoint '1' zigbee2mqtt | Zigbee2MQTT:debug 2023-01-03 11:36:50: Received Zigbee message from 'Lichtschlauch_Durchgangszimmer', type 'commandQueryNextImageRequest', cluster 'genOta', data '{"fieldControl":1,"fileVersion":1061121,"im> zigbee2mqtt | Zigbee2MQTT:debug 2023-01-03 11:36:50: Got OTA request '{"fieldControl":1,"manufacturerCode":4489,"imageType":42,"fileVersion":1061121}' zigbee2mqtt | Zigbee2MQTT:debug 2023-01-03 11:36:50: Is new image available for '0xf0d1b800001b0362', current '{"fieldControl":1,"manufacturerCode":4489,"imageType":42,"fileVersion":1061121}', latest meta '{"fileVersion"> zigbee2mqtt | Zigbee2MQTT:debug 2023-01-03 11:36:50: Update available for '0xf0d1b800001b0362': NO zigbee2mqtt | Zigbee2MQTT:info 2023-01-03 11:36:50: No update available for 'Lichtschlauch_Durchgangszimmer'

stonie81 commented 1 year ago

The update could be started by Home Assistant and went smoothly

info 2023-01-03 16:42:05Device 'Lichtschlauch_Durchgangszimmer' was updated from '{"dateCode":"Nov 27 2018\u0000\u0000\u0000\u0000\u0000","softwareBuildID":"00103101"}' to '{"dateCode":"Feb 16 2022\u0000\u0000\u0000\u0000\u0000","softwareBuildID":"01066400"}'