Can i OTA upgrade IKEA lights from deConz ?

[donnib]

Hi, Time has come where IKEA did an firmware upgrade of the lights. I can see newer version now. Can i upgrade with deConz ? If so how ? Just download the http://fw.ota.homesmart.ikea.net/Tradfri_OTA_release_signed_2017_07_12_161101/bin/159694-TRADFRI-bulb-ws-gu10-1.2.217.ota.ota.signed then somehow use OTAU on the cluster ? How do i choose the image ? I am reluctant trying until someone states this does not brick my device.

[manup]

The IKEA files are not embedded in ZigBee standard OTA format container so it won't work out of the box, maybe it's possible with the deCONZ OTAU File Editor and the JSON info from IKEA to create the files.

[ebaauw]

The firmware files probably need to be packaged into OTA files before deCONZ can use them, see https://github.com/dresden-elektronik/deconz-rest-plugin/issues/23#issuecomment-303816290.

Once it's online (and downloadable like the current version) I'll give it a go for an update test.

@manup, can we take you up on your promise?

[manup]

@manup, can we take you up on your promise?

Yes I'll check it out during the next days :)

[donnib]

That would be great since the release notes states following ZigBee interoperability with 3rd party devices improved. whatever that means and for me i am interested in Stability improvements which might solve my problem with the remote being really bad at detecting push on the button.

[ebaauw]

Yeah, one would hope that the lights become ZLL compliant with the firmware update, including mesh routing (see https://github.com/dresden-elektronik/deconz-rest-plugin/issues/33#issuecomment-318813819).

[donnib]

In the meantime can i upgrade with IKEA Hub or i have to remove the items from deConz first (in that case i probably won't do it) ?

[ebaauw]

You would have to pair a light with the IKEA Hub before it can upgrade its firmware. In this case, I would remove the light from deCONZ before resetting it (to pair it with the IKEA hub), since I expect it to have a different fingerprint after the upgrade. Then reset the light again, and pair it with deCONZ.

I would love to see what the upgraded light looks like in deCONZ (endpoints, clusters). If you have the time and courage to upgrade at least one light, I'd really appreciate a screenshot.

[donnib]

I would love to see what the upgraded light looks like in deCONZ (endpoints, clusters). If you have the time and courage to upgrade at least one light, I'd really appreciate a screenshot.

I have MANY lights that i haven't used yet. I'll take one and probably a remote to upgrade. I am still learning the terminology so where do i find endpoints ?

Similar to this you want to see ? If not what maybe an example would help then i'll provide tonight when i get the chance to do it.

[manup]

Update starts and is running, lets see if it goes through ..

[ebaauw]

Fingers crossed...

[manup]

Nope, update did go through (20 min) but in the upgrade end request the bulb asks for more data.

That means the IKEA update with their gateway must be sniffed in order to figure out what's the difference.

[ebaauw]

Pity.

I'm afraid I won't be able to help here, as I don't have an IKEA Hub. Not yet sure if I want to invest in one having just one IKEA bulb and one remote.

[manup]

That's no problem I have on available, will take a while though, maybe as weekend hacking :)

[donnib]

I am trying to update as we speak but the app just hangs on Update pending, tried repairing, killing the app, rebooted the ikea hub but still pending.

UPDATE: the bulb, updated, now waiting for the remote

[ebaauw]

On the Hue developers forum, they did manage to upgrade the firmware (I assume through the IKEA Hub). With the new firmware, the Trådfri light connects to the Hue bridge, so it must indeed have a ZLL endpoint. See https://developers.meethue.com/content/philips-hue-and-ikea-trådfri?page=2#comment-3701

[donnib]

That's good news, i am still waiting for the remote to update, the feedback is really bad in the app just stating Update pending...... and i am waiting.....and waiting

[donnib]

The remote update never succeeded

[manup]

Updating remotes can be tricky, these devices sleep most of the time and the little batteries usually survive only one update and must be fully charged before.

[donnib]

@manup wow yeah maybe that was the case, i suceeded in the end unfortunately the problems with the remote are the same, they suck big time. They are actually not usable in the way they work. I did upgrade all those 10 GU10 spots that i have problems with so now to try pairing them again with deconz.

[donnib]

I would love to see what the upgraded light looks like in deCONZ (endpoints, clusters). If you have the time and courage to upgrade at least one light, I'd really appreciate a screenshot.

@ebaauw Is this what you want to see ? If not let me know what you want to see on the upgraded lamps.

[ebaauw]

Yes, thanks! It confirms that the light now has a ZLL endpoint instead of ZHA. No changes to the clusters, by the looks of it.

[manup]

Digging and sniffing deeper into IKEA gateway OTA updates. I've created a new plugin which extracts OTA data from the gateway directly via ZigBee. This is kinda nice because now every OTA update firmware can be extracted from any gateway. Works so far with IKEA but should work with Philips and OSRAM gateway too. I'm afraid it's not legal to redistribute the files though .

I was wrong about the public IKEA update files, they do indeed contain the standard ZigBee OTA format files, but they are packed into a further container format (starting with bytes NGIS, whatever this is...).

Next I'll need to test if the extracted OTA part will get accepted by the devices, from what I've seen so far it should work.

[ebaauw]

I've created a new plugin which extracts OTA data from the gateway directly via ZigBee.

I assume the RaspBee needs to join the network created by the gateway, before the firmware can be extracted? I've done so successfully for the Philips Hue bridge and the OSRAM Lightify Gateway (using the ZLL link key), but I haven't been able to make the RaspBee join the network by the innr gateway (tried both the ZLL and ZHA link keys). Then again, their lights reportedly aren't firmware upgradable in the first place (although the UC 110 and PL 110 do offer an OTAU cluster). Maybe I should spend a rainy day sniffing while an innr light joins the innr gateway...

This is kinda nice because now every OTA update firmware can be extracted from any gateway.

Cool. I've got some Philips and one OSRAM light on older firmware that their respective bridges/gateways wouldn't update for some reason. Neither app provides a means to start a firmware upgrade of the light manually.

On the other hand, one would still need a gateway for each vendor. I hope we can also use the files published by vendors (like IKEA).

I'm afraid it's not legal to redistribute the files though .

I wouldn't ask for that. I'm happy to exact them from the bridges/gateways myself.

[manup]

Good news: OTA update works with official IKEA files now, will be available in the next version. Hopefully IKEA keeps the files public in future.

Cool. I've got some Philips and one OSRAM light on older firmware that their respective bridges/gateways wouldn't update for some reason. Neither app provides a means to start a firmware upgrade of the light manually.

If the original bridges won't update the lights, it might be that the firmware can't be extracted at all.

[donnib]

@manup that's cool, what does one have to do ? Manually get the file then what ?

[ebaauw]

Cool. I've got some Philips and one OSRAM light on older firmware that their respective bridges/gateways wouldn't update for some reason. Neither app provides a means to start a firmware upgrade of the light manually. If the original bridges won't update the lights, it might be that the firmware can't be extracted at all.

I stand corrected, all my Philips lights (of the same type) are now on the same firmware, after the latest Philips firmware update. I do have two ORSAM E14 bulbs (Classic B40 TW - LIGHTIFY) with the same Date Code (20140331CNEF****), one on V1.04.12, and one on V1.03.20. I upgraded at least the one with the higher firmware from the Lightify gateway. This is also the version reported by the Lightify app under System Update. Maybe I'm too impatient and should have kept the lights connected to the Lightify gateway for a couple of days? Normally they're connected to deCONZ; I only had them connected to the Lightify gateway for a couple of hours the other day, while sniffing setting the power-on defaults through the Lightly app (see #102).

[ebaauw]

Good news: OTA update works with official IKEA files now, will be available in the next version.

Good news indeed. To celebrate, I've ordered a Xiaomi Magic Cube as well... at less than a third of the price of the Trådfri hub.

[manup]

@manup that's cool, what does one have to do ? Manually get the file then what ?

I'll write a small step-by-step guide tomorrow, it's pretty easy, OTA files will be downloaded automatically from a small python script.

[manup]

I do have two ORSAM E14 bulbs (Classic B40 TW - LIGHTIFY) with the same Date Code (20140331CNEF****), one on V1.04.12, and one on V1.03.20. I upgraded at least the one with the higher firmware from the Lightify gateway.

I'll test this too soon, have a LIGHTIFY home gateway here.

[donnib]

I'll write a small step-by-step guide tomorrow, it's pretty easy, OTA files will be downloaded automatically from a small python script.

Terrific, looking forward to that, resetting lamps and using the IKEA gateway is a pain. I've done it on few lights but this would make my life so much easier.

[manup]

New version is online, more details in the release notes:

https://github.com/dresden-elektronik/deconz-rest-plugin/releases/tag/V2_04_64

[donnib]

@manup cool stuff, can't wait !!!

[donnib]

@manup my web interface does not say anything about an update to 0x26120500, i pulled latest changes from master and build them webinterface says 2.04.64 but deConz reports 2.04.63

[manup]

Ah I see, since you have pulled changes for 2.04.64 the updater won't trigger (it doesn't know that core is still on 2.04.63).

Please update the core manually: http://www.dresden-elektronik.de/rpi/deconz/beta/deconz-2.04.64-qt5.deb

$ sudo dpkg -i deconz-2.04.64-qt5.deb

It should then show the Update Firmware button in the webapp.

[donnib]

.....something is definitely happening now :)

About upgrade nothing is happening in the plugin, i don't see any progress or anything starting to upgrade, have to wait a while ? I did run the python file and it did get the files then i did a restart of deconz. Anything else needs to be done ? I do also have teh OTAU server enabled in the webinterface.

UPDATE: someting happened when i clicked Query in the OTAU plugin then it started to update on of the lights. Maybe it would have started by itself ?

[manup]

Jep that looks better :)

The lights will ask the gateway from time to time if there is an update available. You can accelerate that by power off/on a lights (mains power) or by select a light in the OTA plugin and click query button.

[donnib]

You can accelerate that by power off/on a lights (mains power) or by select a light in the OTA plugin and click query button.

Yeah exactly what i noticed, can the gateway update only one at a time ? I have tried Query on multiple lights while one device is updating already that i know needs an update but nothing happens.

[ebaauw]

OK, installed v2.4.64, had some issues updating the RaspBee firmware, but after power cycling the Raspberry Pi, the firmware upgrade went through allright.

I ran the Python script. In ~/otau, I now have ten .ota.ota.signed files, which have been unpacked/copied into ten 117C-xxxx-yyyyyyyy.zigbee files (I assume that's the manufacturer id (IKEA), image code, and file version?). Feeling lucky, I downloaded the latest firmware for my ubisys D1 and placed it in ~/otau. It looks like it's been copied as well, to 10F2-7B01-010E015E.zigbee. There's also a 1014-0005-14010400.zigbee file. Philips? How on earth did it get there?

Ik looks like the OTAU plugin is visiting all nodes, very, very slowly, populating the Version and Image fields, at least for the Philips nodes. The OSRAM, IKEA and ubisys nodes still show all zeroes for these field, even though Last Query Time has been updated. All node continue to show "No file" under Progress.

Do I need to select the OTAU files manually and press Update? If so, how do I match the right file to the right node? Or should I wait for the Image be populated? Should I specify the mmmm-xxxx-yyyyyyyy.zigbee file, or is either file OK?

I also see that the Cluster Info shows info on the OTAU cluster. Is that new? I don't think I've seen that before.
Some Philips lights show an innr PL 110/UC 110 control box as Upgrade server. It has both a server and a client OTAU cluster (see https://github.com/ebaauw/homebridge-hue/wiki/ZigBee-Devices#innr-control-board), but the client cluster doesn't support any of the attributes - guess it's not upgradeable after all.
The OSRAM light on the older firmware shows the Hue bridge as Upgrade server; the other OSRAM light shows the OSRAM gateway. Both OSRAMs report the same current file version. Is that the version uploaded to the light or the version seen on the gateway?

[manup]

Yes one at a time, takes 20-25 minutes per light. Note The light must be selected in the table in OTA plugin (not the node view).

After click on query they should respond with their current firmware immediately, and if a new version is available upload should start after a few seconds.

[donnib]

Ok well then it behaves correctly, i just have to be patient and wait for it to be finished before next can start.

[manup]

There's also a 1014-0005-14010400.zigbee file. Philips? How on earth did it get there?

It was there for a long time I guess, that's the update files from dresden elektronik FLS. These were downloaded by an ancient OTAU download script from deCONZ, I will remove that soon since it doesn't make sense to download stuff likely not needed.

Ik looks like the OTAU plugin is visiting all nodes, very, very slowly, populating the Version and Image fields, at least for the Philips nodes. The OSRAM, IKEA and ubisys nodes still show all zeroes for these field, even though Last Query Time has been updated. All node continue to show "No file" under Progress.

It's the other way around, lights are asking the gateway for updates (often after powerup or after some hours). For some specific devices deCONZ will also automatically query the devices.

You can manually speed that up by select a device in the table and click query.

Do I need to select the OTAU files manually and press Update? If so, how do I match the right file to the right node? Or should I wait for the Image be populated? Should I specify the mmmm-xxxx-yyyyyyyy.zigbee file, or is either file OK?

deCONZ will select the newest file automatically (if available), there new is determined by the filename, which by the way is also part of the ZigBee OTA standard.

I also see that the Cluster Info shows info on the OTAU cluster. Is that new? I don't think I've seen that before.

Its been there for a while, the OTA client cluster (gray) was extended by a few attributes recently.

The OSRAM light on the older firmware shows the Hue bridge as Upgrade server; the other OSRAM light shows the OSRAM gateway. Both OSRAMs report the same current file version. Is that the version uploaded to the light or the version seen on the gateway?

It's the version of the light. The devices will pick the OTA server on their own after power up, some are smart and try multiple servers for updates... some. ;)

[donnib]

The first light has completed, i tried Query next lamp but nothing happens, something is funky.....don't know what.

[manup]

My advice is patience ;) or better just let OTA run in the background, it will start and complete automatically at some time. We update large networks with up to 160 nodes very often, it's supposed to just run in the background.

[donnib]

@manup sure will do, i'll go to bed now and see tomorrow what the version of the lights is, it should be the new one. will report tomorrow.

[ebaauw]

After click on query they should respond with their current firmware immediately You can manually speed that up by select a device in the table and click query.

Doesn't seem to do anything. Or are some lights supposed to report all zeroes?

[manup]

Have you selected the light in the OTA table view? Its row needs to be selected there.

[ebaauw]

Yes I have. Maybe it's because they're still looking at the wrong Update server?

Another question: how do I extract the firmware files from my OSRAM gateway?

[donnib]

So it seems that some updates have been done but the node list version is not updated. If i look at the lights attribute the version seems new. Also in the OTA list the light also shows the new version. The node list would be nice to have updated sine it's a good overview of the nodes and their version.

[ebaauw]

My Trådfri bulb finally updated. Indeed the Node List hasn't been updated. The update removed the weird character from the model identifier.

The update of the ubisys seems to stall at 0.09%. Also, the Duration isn't updated and stalls at 0:26. There's no direct line between the ubisys and the RaspBee, would I need to move them closer?

[ebaauw]

Powered down the innr UC 110 and PL 110. Restarted deCONZ. The OTA table now shows the Version and Image on clicking Query. Except for a few Philips nodes - I suspect these are the Hue dimmer switches - and the Trådfri remote.