search

dresden-elektronik / deconz-ota-plugin

Server side implementation of the standard Zigbee OTA protocol.
BSD 3-Clause "New" or "Revised" License
47 stars 13 forks source link

Hue Firmware Update #10

Closed Krocko closed 4 years ago

Krocko commented 6 years ago

Today Philips published firmware updates for various lights. Any chance to update the lights with deCONZ?

ebaauw commented 6 years ago

Today Philips published firmware updates for various lights

Which lights? Which versions? I see no mention of this on the release notes, nor in the Hue developer forum.

snozzlebert commented 6 years ago

On the Dutch website: http://www2.meethue.com/nl-nl/ondersteuning/versie-informatie/lamps-luminaires I've upgraded my LCT015, LTW012, LWB010 yesterday. All to version 1.29.0_r21169 I think

Security improvement: proximity based linking with a Hue dimmer switch is now only possible for 30 minutes after the light is powered on.

also affects the Touchlink function in deCONZ. Possibly it contains changes for Hue Entertainment? (#167)

Krocko commented 6 years ago

@ebaauw i have read about it in a German forum.

http://forum.smartapfel.de/forum/thread/675-heute-großes-hue-lampenupdate/

manup commented 6 years ago

Currently it is not possible to update Hue lights with deCONZ due the lack of official available firmware files.

The changelog suggests that there are no functional improvements, but the Touchlink restriction for 30 min is quite useful and prevents a evil neighbour from stealing your lights :)

ebaauw commented 6 years ago

Cool, thanks!

Any chance to update the lights with deCONZ?

Yes, please. Really not looking forward at resetting all of my 29 Hue lights, connecting them to the Hue bridge for updating the firmware, resetting them, connecting them back to deCONZ, hoping the groups and scenes will be restored correctly. I'm really tempted to try if the Hue bridge will update the light firmware when it's joined to the deCONZ network (see dresden-elektronik/deconz-rest-plugin#266).

I like the security improvement, but it really shows Philips expect the lights to be powered all the time. In other words: they don't plan on supporting state.reachable for traditional wall switches, nor providing sensible power-on defaults. I wonder if the 30min limitation also holds for the reset through the Hue dimmer switch (holding On and Off buttons for 10 seconds)?

Possibly it contains changes for Hue Entertainment?

Would they report that as Reachability improvements of the lights.? I would expect a bridge firmware update and app update for that as well.

snozzlebert commented 6 years ago

Possibly it contains changes for Hue Entertainment?

Would they report that as Reachability improvements of the lights.? I would expect a bridge firmware update and app update for that as well.

I'm pretty sure the firmware is related to Hue Entertainment. There is a difference in capabilities in the bridge response before and after the firmware update of a bulb. Before:

{
    "state": {
        "on": true,
        "bri": 254,
        "hue": 8418,
        "sat": 140,
        "effect": "none",
        "xy": [
            0.4573,
            0.41
        ],
        "ct": 366,
        "alert": "none",
        "colormode": "ct",
        "mode": "homeautomation",
        "reachable": true
    },
    "swupdate": {
        "state": "noupdates",
        "lastinstall": null
    },
    "type": "Extended color light",
    "name": "Hue color candle 1",
    "modelid": "LCT012",
    "manufacturername": "Philips",
    "capabilities": {
        "streaming": {
            "renderer": false,
            "proxy": false
        }
    },
    "uniqueid": "00:17:88:01:03:00:0a:0a-0b",
    "swversion": "1.15.2_r19181",
    "swconfigid": "79A02AB9",
    "productid": "Philips-LCT012-1-E14ECLv1"
}

After:

{
    "state": {
        "on": false,
        "bri": 254,
        "hue": 0,
        "sat": 0,
        "effect": "none",
        "xy": [
            0.3804,
            0.3768
        ],
        "ct": 366,
        "alert": "none",
        "colormode": "ct",
        "mode": "homeautomation",
        "reachable": true
    },
    "swupdate": {
        "state": "noupdates",
        "lastinstall": "2017-12-06T20:42:04"
    },
    "type": "Extended color light",
    "name": "Hue color candle 1",
    "modelid": "LCT012",
    "manufacturername": "Philips",
    "capabilities": {
        "streaming": {
            "renderer": true,
            "proxy": true
        }
    },
    "uniqueid": "00:17:88:01:03:00:0a:0a-0b",
    "swversion": "1.29.0_r21169",
    "swconfigid": "EA9279F9",
    "productid": "Philips-LCT012-1-E14ECLv1"
}

streaming.renderer and streaming.proxy are true after updating

ebaauw commented 6 years ago

Yes. What Hue bridge firmware are you on? See also https://github.com/ebaauw/homebridge-hue/issues/217#issuecomment-345854006.

snozzlebert commented 6 years ago

Version is 1.22.0

ebaauw commented 6 years ago

Ah, I see they published it Nov 27.

mathos77 commented 6 years ago

Not that I have any issues with my current Hue bulbs, but it would be nice if we would somehow be enabled to update the lights via deCONZ.

If we would know where the firmware files are, would we then be able to update the bulbs ?

ebaauw commented 6 years ago

That depends on whether they’re encrypted.

qraynaud commented 5 years ago

That would indeed be a very cool feature to have. It should be possible to proxy the Philips hue app to see where it checks for updates and downloads them. Right now mine says there is no update available so it's too late for that though (I updated everything using the hue app and am just switching over to deconz).

timbru31 commented 5 years ago

If it helps I could do some network sniffing and provide the request and responses from the Hue App / Bridge, I have to update some hue bulbs and can borrow a bridge.

qraynaud commented 5 years ago

@timbru31 yes, now you're saying I feel like it's a lot more sensitive to sniff the bridge :+1:

manup commented 5 years ago

Would be interesting to access the files not sure if the url can be sniffed when https is used and it might be within the content body.

stale[bot] commented 5 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Hypfer commented 5 years ago

This bot does more harm than good tbh

timbru31 commented 5 years ago

Sorry for the silence - I had no luck back in November (had to bring the bridge back the day after) but purchased a used Hue bridge for small money, so I have more time to sniff the network traffic now :)

timbru31 commented 5 years ago

It seems that the Hue App is actually downloading the firmware, at least this is the only interesting package I've found when my Hue app was searching for new software updates:

Screen Shot 2019-04-14 at 22 47 54

Edit: I believe this is a captured package from the bridge which would make more sense.

ebaauw commented 5 years ago

It's the Hue bridge that downloads the firmware, not the app. The app just issues a REST API call to the bridge to start the search.

timbru31 commented 5 years ago

Yeah captured that now, too: {"swupdate":{"checkforupdate":true}} is the payoad in the PUT request.

timbru31 commented 5 years ago

My bridge made a request to http://fds.dc1.philips.com/firmware/ZGB_100B_010C/16783872/100B-010C-01001A00-ConfLight-Lamps_0012.zigbee (267kb)

No auth needed for the request. Any idea what this file could be? strings and a HexEditor did not yield any useful information.

ebaauw commented 5 years ago

Judging by the filename and size, that's a firmware image alright.

100B-010C-01001A00-ConfLight-Lamps_0012.zigbee
|    |    + image file version
|    +- image type (device type)
+- manufacturer code 0x100b = Philips

These match the fields in the STD OTAU Plugin panel of the deCONZ GUI, see e.g. https://github.com/dresden-elektronik/deconz-rest-plugin/issues/96#issuecomment-321022808.

The image type and file version match the current (1.46.13_r26312) firmware for the Color Temperature bulbs and spots (LTWxxx).

Now the $100M question is whether this file is encrypted. If it isn't, you should be able to download it, feed it do the OTAU plugin and update the firmware from deCONZ. If it is, the file is useless without the decryption key.

timbru31 commented 5 years ago

I have sniffed the NWK of the Hue bridge so I can capture the Zigbee packages, too. And the ZLL OTA cluster is (AFAIK) without encrypted firmware?

Edit: By the way this is the current state of the /config response from the Hue bridge. Notice the text property:

    "swupdate": {
        "updatestate": 2,
        "checkforupdate": false,
        "devicetypes": {
            "bridge": false,
            "lights": [
                "1"
            ],
            "sensors": []
        },
        "url": "",
        "text": "ZGB_100B_010C - ConfLight Lamps SAMR21 - 26.0",
        "notify": false
    },
manup commented 5 years ago

Awesome!

Yes this is a valid OTA image and encrypted but that's ok. OTA files should be encrypted since they are transferred transparently.

image

manup commented 5 years ago

Note decryption usually is done on the receiving device.

Some vendors don't encrypt their files though (not saying any names here..).

ebaauw commented 5 years ago

Note decryption usually is done on the receiving device.

Cool, didn't know that.

The URI seems to redirect to the AWS cloud. They don't support browsing the directory structure (if the URI path even reflects one), so we might have to re-sniff the Hue bridge for every type and version. Unless there's like an index file containing links to the firmware files, like IKEA has.

I suppose the Hue bridge only requests the firmware for connected lights?

manup commented 5 years ago

I suppose the Hue bridge only requests the firmware for connected lights?

Likely, maybe something like pihole in between can help to figure out more URLs.

timbru31 commented 5 years ago

The URI seems to redirect to the AWS cloud. They don't support browsing the directory structure (if the URI path even reflects one), so we might have to re-sniff the Hue bridge for every type and version. Unless there's like an index file containing links to the firmware files, like IKEA has.

The only URL that's not returning a 404 is http://fds.dc1.philips.com/ with a (strange?) response of ecdprd
Is it actually an S3 bucket? There are some tools to scan them :)

I suppose the Hue bridge only requests the firmware for connected lights?

Seems to be true. On the other hand I've only got the E14 lights.

Likely, maybe something like pihole in between can help to figure out more URLs.

I've got one running, too. I can take some deeper looks into the logged queries.

timbru31 commented 5 years ago

The image type and file version match the current (1.46.13_r26312) firmware for the Color Temperature bulbs and spots (LTWxxx).

Firmware matches my updated LCT012, too

timbru31 commented 5 years ago

Other noteworthy philips domains (sadly pihole logs only the domain names not paths/queries since it's only my DNS resolver)

ebaauw commented 5 years ago

LCT012 is a gamut-C extended colour light, which, I understand (I don't have any of these myself) has 5-channels: RGB+CW/WW. So it would make sense to use the same firmware for the colour temperature lights (with two channels: CW/WW), where the RGB part is used only for lights that support this.

From what I've seen, the gamut-B extended colour lights (RGB but different colours, optimised for colour temperature), the gamut-A colour lights (also RGB, with other different colours, optimised for colours), and the on/off lights all have different firmware builds.

timbru31 commented 5 years ago

Back to the network traffic: after resetting the bridge to factory state and after the PUT to check for updates it always connects to:

After factory reset it also connects to:

timbru31 commented 5 years ago

Given that Philips encrypted the OTA files for older firmware, I think that one I sniffed is encrypted too :/ See https://www.blackhat.com/docs/us-16/materials/us-16-OFlynn-A-Lightbulb-Worm.pdf (start at page 24 and 64)

timbru31 commented 5 years ago

Would a sniffed OTA upgrade (Zigbee/IEEE 802.15.4 traffic) help any further? I've captured the OTA process - if wanted/required I could send the pcap file and NWK key. @manup @ebaauw

manup commented 5 years ago

I think on Zigbee level all should be fine as the data is transferred transparently, only need more URLs for various firmware files :)

manup commented 5 years ago

@timbru31 how do you sniff the traffic of the Hue App? Tried some apps like "Packet Capture", but couldn't figure out the snooping in ssl connections.

ebaauw commented 5 years ago

To find the firmware location, you need to sniff the (apparently unencrypted) traffic from the Hue bridge to the cloud, not the (https) traffic from the Hue app to the Hue bridge.

The old (round) bridge doesn’t support https, so the Hue app can still use plain http. I suspect/hope that if you block port 443 in your firewall, the app will fallback to plain http for the new (square) bridge as well. Haven’t tried that myself, though.

timbru31 commented 5 years ago

@manup My sniffing setup is the following:

@ebaauw True with HTTP, but the payload is still AES encrypted and I assume we won't find the key that easily. See attached pcap: bridge_dump_factory_reset.pcapng.zip (package 43)

manup commented 5 years ago

Interesting approach with the Internet sharing, gonna try this next. Looks like only the bridge needs to be sniffed.

I've brought some of my Hue devices to a round v1 Hue bridge, and started update search yesterday. Looked like nothing happened, but after a few hours updates there shown for all devices:

When I powered off all devices to not start the update :)

Meanwhile I found another way to sniff the bridge traffic via the Telekom Speedport W 724V router capture feature http://192.168.2.1/html/capture.html.

The bridge update state is back to 1 "Downloading available updates" and no updates are shown yet. The router sniffer is running, hopefully today brings some new URLs.

Also added the Hue Dimmer switch to the bridge.

timbru31 commented 5 years ago

Meanwhile I found another way to sniff the bridge traffic via the Telekom Speedport W 724V router capture feature http://192.168.2.1/html/capture.html.

Nice! The FRITZ!Box supports such a feature, too - but it's well hidden :)

I've got my hands on a new Hue Bridge v2 (square), on the first connection to the internet it immediately downloaded this file: http://fds.dc1.philips.com/firmware/BSB002/1931140050/BSB002_1931140050.product.RSA_prod_01.fw2, I assume that's the most recent Bridge firmware (not yet with a changelog, the only (German) source regarding this version is: https://hueblog.de/community/frage/gibt-es-infos-zum-update-der-bridge-auf-version-1931140050/)

Cool thing: the naming follows the scheme http..../BSB002/<fw-version>/BSB002_<fw-version>.product.RSA_prod_01.fw2 (for example an older firmware link: http://fds.dc1.philips.com/firmware/BSB002/1931069120/BSB002_1931069120.product.RSA_prod_01.fw2)

Besides that I've gained root on the box (http://colinoflynn.com/2016/07/getting-root-on-philips-hue-bridge-2-0/, https://blog.andreibanaru.ro/2018/03/27/philips-hue-2-1-enabling-wifi/) and have access to the unencrypted ipbridge binary (the one binary that does everything from being a webserver to sending commands to the Zigbee Soc) - now we need to understand how the AES key is constructed to decrypt the connection to www.ecdinterface.philips.com (where I assume the firmware info is stored/returned in the body payload)

ebaauw commented 5 years ago

Nice! The FRITZ!Box supports such a feature, too - but it's well hidden

Brilliant!

Markkuuss commented 5 years ago

@manup: Have you found any new URLs to the following hue devices?

  • Motion sensor SML001
  • Hue beyond LLM001
  • E27 Lux LWB004
  • E27 color light LCT001
  • GU10 color light LCT003
robertlandes commented 5 years ago

Btw. I can confirm, that zigbee firmware file 100B-010C-01001A00-ConfLight-Lamps_0012.zigbee from above updated the following lamps to latest firmware version through OTA:

The strange thing is, that I thought it didn't work properly, because Phoscon / REST API was still showing old firmware version for my LWB010 lamps, which I didn't pair with my Hue Bridge for FW update before pairing it to deCONZ. So last weekend I reset one of the bulbs paired it to my Hue Bridge to do the FW update, just to see, that it already has latest firmware. So repaird it back to deCONZ (without deleting it) and it still showed old FW version. After deleting it and pairing it again, it showed up as new light with the correct firmware version. So it seems, that there is an Issue with updating FW versions after OTA update.

Probably will need to open a separate issue for that.

manup commented 5 years ago

@manup: Have you found any new URLs to the following hue devices?

Sadly not yet, my last attempt a while ago with the Hue bridge v1, over a few days, didn't yield any new URLs, probably my Hue devices are too old or too new to get updated.

Btw. I can confirm, that zigbee firmware file 100B-010C-01001A00-ConfLight-Lamps_0012.zigbee from above updated the following lamps to latest firmware version through OTA:

Congrats this is the first time a Hue device was updated via deCONZ :)

So it seems, that there is an Issue with updating FW versions after OTA update.

Once the firmware version was queried it isn't refreshed for Hue lights. We can active this in future releases, it's already active for certain other devices which could be updated.

To manually query the firmware version you can read the Basic cluster attributes in the deCONZ GUI.

ebaauw commented 5 years ago

After deleting it and pairing it again, it showed up as new light with the correct firmware version. So it seems, that there is an Issue with updating FW versions after OTA update.

You need to force-read the Basic cluster attributes before the GUI and REST API show the new firmware (and date code?). If the new firmware exposes a different profile or device id (e.g. upgrade from ZLL to Zigbee 3.0, which uses ZHA), you need to delete and re-pair the device. Just force-reading the Simple Descriptors isn't enough to update the GUI.

l-mb commented 5 years ago

Hey, I wonder if someone has actually tried reaching out to Philips. Sorry if this is a dumb question, but have they officially denied publishing their firmware images more readily? I need to update my bulbs and this does seem like a project I could try to play with next week, but first I'd like to know whether the easy path has been tried ;-)

timbru31 commented 5 years ago

Personally I haven't asked Philips 🙈

rb2k commented 5 years ago

@timbru31 any chance you ever tried to use DNS poisoning to redirect the hue requests to a local MITM proxy?

robertlandes commented 5 years ago

@timbru31 any chance you ever tried to use DNS poisoning to redirect the hue requests to a local MITM proxy?

Yes, that would be interesting. I think https://mitmproxy.org/ can probably be a tool to achieve this. I would test it, I just don’t have any Hue hardware which is not the current firmware version...