Support request for BTicino DIN power consumption module

[meco489]

Hi, I have installed a RaspBee for my home automation. I can connect some BTicino products, using Legrand integration .... under the list of functioning products. I can't connect a DIN module for power measurement. I attach below the screenshots. Thanks in advance and congratulations for the work you are doing. Domenico

Working Products:

• BTicino K4003C equal to Legrand 067773
• BTicino 3584C equal to Legrand 064888
• BTicino K4531C equal to Legrand 067775

Non-functioning products:

• BTicino F20T60A

[rotragit]

Does your 3rd attempt also contain the successful sensor creation later?

@SwoopX no, at the end the sensor was not created, no light was created and the module left the network remaining with the red led on.

[SwoopX]

@rotragit Could you also collect that information, when it does successfully? Would love to see what's going over the air then.

[rotragit]

Yes, I'll do that this afternoon (hope before your evening work ;-) )

[Smanar]

Sidenote: The device has a manufacturer specific attribute 0xF000 in the basic cluster, Uint32 with value 0000005D

Yes, this is normal, it's the Legrand protection.

I m still reading the rest, so muh thing to read ...

BTW, if the device is not included during the test, you can give up logs, totally unusable, the legrand device need to pass the legrand protection to be usable, else it will don't respect protocol and leave the network itself.

Maybe the 0xFC01 cluster provides something, but the Electrical Measurement cluster should be exposed as a ZHAPower /sensors resource. The code is already there, you’d only need to whitelist the device to create the resource and to scale the reported values.

It's already done, the device is working fine. But on his situation the sensor is not created using normal pairing.

I m looking the debug and there is some things I don't understand.

• Why in the fonction DeRestPluginPrivate::handleDeviceAnnceIndication(const deCONZ::ApsDataIndication &ind) the light part is skipped ? I don't see the condition for that ?

Edit: In rest_sensor.cpp, The device wil trigger void DeRestPluginPrivate::handleIndicationSearchSensors(const deCONZ::ApsDataIndication &ind, deCONZ::ZclFrame &zclFrame) line 2338 But never finish it (never reach line 2378)

We can make a test with forcing it to test with

        else if (macCapabilities & deCONZ::MacDeviceIsFFD)
        {
            if (checkMacVendor(ext, VENDOR_LDS))
            { //  Fix to allow Samsung SmartThings plug sensors to be created (7A-PL-Z-J3, modelId ZB-ONOFFPlug-D0005)
            }
            else if (checkMacVendor(ext, VENDOR_JASCO))
            { //  Fix to support GE mains powered switches
            }
            else if (checkMacVendor(ext, VENDOR_LEGRAND))
            { //  Fix to test
            }
            else
            {
                return;
            }
        }

I say that according the 2 attemp, I don't see in log "set fast probe address to" and he have said the pairing have worked.

[rotragit]

4attempt, module paired pressing reset button after it appears in deconz and applying the tips. In Phoscon the joining procedure ends with "Ready". The sensor is created in sensors table:

sqlite> select * from sensors; 1|Daylight|Daylight|PHDL00|Philips|00:21:2e:ff:ff:05:03:6f-01|1.0|{"dark":false,"daylight":true,"lastupdated":"2020-03-30T11:53:08","status":170,"sunrise":"2020-03-30T04:58:37","sunset":"2020-03-30T17:40:10"}|{"configured":true,"lat":"45.545479","long":"11.535421","on":true,"sunriseoffset":30,"sunsetoffset":-30}||normal|1 2|DIN power consumption module|ZHAPower|DIN power consumption module|Legrand|00:04:74:00:00:a1:32:f6-01-0b04|01b|{"lastupdated":"2020-03-30T11:54:40","power":0}|{"on":true,"reachable":true}|{"d":13,"ep":1,"in":[2820],"p":260}|normal|1 sqlite>

There is not any entry in nodes table (lights):

sqlite> select * from nodes; 00:21:2e:ff:ff:05:03:6f-01|1|normal|Configuration tool 1|65520|1|ConBee II|dresden elektronik|0x264a0700|{"attr/manufacturername":"dresden elektronik","attr/modelid":"ConBee II","attr/name":"Configuration tool 1","attr/swversion":"0x264a0700","attr/type":"Configuration tool","attr/uniqueid":"00:21:2e:ff:ff:05:03:6f-01","state/reachable":true} 00:12:4b:00:0b:e8:6e:0e-08|2|normal|On/Off light 2|65520|8||LUMI||{"attr/manufacturername":"LUMI","attr/modelid":null,"attr/name":"On/Off light 2","attr/swversion":null,"attr/type":"On/Off light","attr/uniqueid":"00:12:4b:00:0b:e8:6e:0e-08","state/alert":null,"state/on":null,"state/reachable":false} ec:1b:bd:ff:fe:31:95:fe-01|3|normal|Dimmable light 3|65520|1|TRADFRI bulb E27 WW 806lm|IKEA of Sweden|2.1.022|{"attr/manufacturername":"IKEA of Sweden","attr/modelid":"TRADFRI bulb E27 WW 806lm","attr/name":"Dimmable light 3","attr/swversion":"2.1.022","attr/type":"Dimmable light","attr/uniqueid":"ec:1b:bd:ff:fe:31:95:fe-01","config/powerup":7,"state/alert":null,"state/bri":254,"state/on":true,"state/reachable":false} sqlite>

Attached deconz.log and whireshark capture. 4attempt.zip

[SwoopX]

I say that according the 2 attemp, I don't see in log "set fast probe address to" and he have said the pairing have worked.

@Smanar Right, that doesn't really fit the picture. And the code location you mentioned above is the one I meant.

[Smanar]

Yep, so you want to make modification to test, or I do it ?

[SwoopX]

Must be done by someone owning the device. I'd prefer not yet to add it to my repo. Would that be ok?

[rotragit]

We can make a test with forcing it to test with

        else if (macCapabilities & deCONZ::MacDeviceIsFFD)
        {
            if (checkMacVendor(ext, VENDOR_LDS))
            { //  Fix to allow Samsung SmartThings plug sensors to be created (7A-PL-Z-J3, modelId ZB-ONOFFPlug-D0005)
            }
            else if (checkMacVendor(ext, VENDOR_JASCO))
            { //  Fix to support GE mains powered switches
            }
            else if (checkMacVendor(ext, VENDOR_LEGRAND))
            { //  Fix to test
            }
            else
            {
                return;
            }
        }

No success. First the timeout was gone and "Connection failed". Then I repeat the join, pressed the reset button. No sensor created anyway. Attached 5attempt.

EDIT: I've also checked I'm running the "right" deconz:

(base) root@romano-SATELLITE-PRO-L850:/# find . -name libde_rest_plugin.so ./opt/libde_rest_plugin.so ./usr/share/deCONZ/plugins/libde_rest_plugin.so

The first is the compiled plugin, the second where it has been copied to. Because I had a doubt the plugin was copied in the wrong directory...

5attempt.zip

[Smanar]

You haven't sensor created, but the sensor was included in deconz ? Else the code can't be usefull.

BTW this log is more usefull. It's realy better

20:29:05:954 ZDP indication search sensors 0x0004740000A132F6 (0x51EF) cluster 0x8002 20:23:34:203 [3] get simple descriptor 0x01 for 0x0004740000a132f6

20:23:34:290 [444] Model ID: à5œ)š 20:23:34:290 [444] Model ID: 2E8A8D90 20:23:34:290 [444] Manufacturer: à5œ)š 20:23:34:290 [444] Manufacturer: 2E8A8D90 20:23:34:359 [555] Creating sensors

What is that ? There is a memory corruption somewhere ?

But idk if the log are usefull because we can see the device join again later, so it mean the first announce was useless.

20:23:34:102 DB sql exec UPDATE devices SET nwk = 23945 WHERE mac = '00:04:74:00:00:a1:32:f6';INSERT INTO devices (mac,nwk,timestamp) SELECT '00:04:74:00:00:a1:32:f6', 23945, strftime('%s','now') WHERE (SELECT changes() = 0);
20:29:05:943 DB sql exec UPDATE devices SET nwk = 20975 WHERE mac = '00:04:74:00:00:a1:32:f6';INSERT INTO devices (mac,nwk,timestamp) SELECT '00:04:74:00:00:a1:32:f6', 20975, strftime('%s','now') WHERE (SELECT changes() = 0);
20:29:59:584 DB sql exec UPDATE devices SET nwk = 17921 WHERE mac = '00:04:74:00:00:a1:32:f6';INSERT INTO devices (mac,nwk,timestamp) SELECT '00:04:74:00:00:a1:32:f6', 17921, strftime('%s','now') WHERE (SELECT changes() = 0);

[rotragit]

For "no sensor created" I mean there is no entry in the sensor table in zll.db. If the sensor is not listed in the table, I think is not in the API, also. What do you mean with "sensor created in deconz"? There is anything else can I check?

[SwoopX]

@rotragit Did you use my de_web_plugin.cpp for attempt #4? Asking because it doesn't look like that because the extended debug output is missing. If yes, could you repeat it once again? Deconz log would be sufficient this time. Thanks!

[SwoopX]

@Smanar If I understand all information correctly, we may change the code a bit to have the required Legrand attribute read automatically during pairing. Eventually, a simple check, well placed, for the manufacturer code, followed by triggering "handleBasicClusterIndication" you previously introduced (or at least the initial version) by maybe reading one or more basic attributes could do the trick.

20:29:05:954 ZDP indication search sensors 0x0004740000A132F6 (0x51EF) cluster 0x8002
20:23:34:203 [3] get simple descriptor 0x01 for 0x0004740000a132f6

20:23:34:290 [444] Model ID: à5œ)š�
20:23:34:290 [444] Model ID: 2E8A8D90
20:23:34:290 [444] Manufacturer: à5œ)š�
20:23:34:290 [444] Manufacturer: 2E8A8D90
20:23:34:359 [555] Creating sensors

What is that ? There is a memory corruption somewhere ?

Good question, but it kept me busy a few days back. No idea, maybe an unresolved address? At least, it doesn't look right.

[rotragit]

@SwoopX OK, of course if you say the debug info are missing the only reason could be the plugin was not the right one. I've checked the command history but I have to admit I cannot give a prove the plugin used was that with your file because there is a apt update; apt upgrade in the middle and I used several console, so the history is not consistent among them. I'm really sorry for that.

In this moment the MD5 file signature are:

554a921982c5464286455c91dc20e51d de_web_plugin.cpp 018fcfdf2053b1e53d3572217a312684 de_web_plugin.cpp.original

the first one is your file.

These are the command used today:

2009 qmake && make -j2 2010 sudo cp ../libde_rest_plugin.so /usr/share/deCONZ/plugins 2011 /usr/bin/deCONZ --dbg-info=2 --dbg-error=1 > deconz.log 2012 mv deconz.log /home/romano/6attempt-deconz.log 2013 cd /home/romano 2014 zip 6attempt.zip 6attempt-* 2015 history

rest_sensors.cpp is the package one without the modification.

After the module appears in deCONZ I pressed the reset button and the led gone green. Within the timeout of Phoscon (add sensor, 3 minutes timeout) I applied the tips and Phoscon ended with "Ready".

In zll.db, in the table sensors, the row is inserted only after I press "Ready" in Phoscon. And now the sensors table is:

1|Daylight|Daylight|PHDL00|Philips|00:21:2e:ff:ff:05:03:6f-01|1.0|{"dark":true,"daylight":false,"lastupdated":"2020-03-31T04:26:50","status":130,"sunrise":"2020-03-31T04:56:43","sunset":"2020-03-31T17:41:27"}|{"configured":true,"lat":"45.545479","long":"11.535421","on":true,"sunriseoffset":30,"sunsetoffset":-30}||normal|1 2|DIN power consumption module|ZHAPower|DIN power consumption module|Legrand|00:04:74:00:00:a1:32:f6-01-0b04|01b|{"lastupdated":"2020-03-31T04:27:00","power":0}|{"on":true,"reachable":true}|{"d":13,"ep":1,"in":[2820],"p":260}|normal|1

There is not any entry related to the module in nodes table.

Attached the log and wireshark. 6attempt.zip

[rotragit]

This is the same as #6 but with the modified rest_sensor.cpp. Deconz log + wireshark.

7attempt.zip

[Smanar]

For "no sensor created" I mean there is no entry in the sensor table in zll.db. If the sensor is not listed in the table, I think is not in the API, also. What do you mean with "sensor created in deconz"? There is anything else can I check?

If the device not appear in deconz, it mean the pairing have failed, so the log is useless. To check the problem, we need the device was correctly included in deconz, to check why it s not in the API too. And we need to device correctly included, it mean stable, if it leave 2m after because the protection, it's a failed try too. And without using the "special trick"

ATM i m looking the 5 attemps, the device was included in deconz in this one ?

[rotragit]

No, in attempt 5 the sensor was not created. You can look at #6 and #7

But I think there was a misunderstanding. If the sensor is not in the sensors table, of course is not in the API / json. If the sensor is in the sensors table then is always also in the API / json. I never met the case that is in the sensors table but is not in the API / json.

I'm sorry if I was not able to explain myself. I think there is a misunderstanding on this point. One problem is that the sensors table is not populated if I don't apply the tips (re-read the attributes from the basic cluster using the deconz GUI). But first the problem is that the module is not complete paired if I don't press the reset button, and I think the two problems are related: pressing the reset button "forces" the module to consider the pairing OK so it will not leave the network after a while because a reply message is missing. But stopping the process with the reset button leaves deconz in the half of its own process and the row in sensors table is not executed.

[SwoopX]

@rotragit Thanks for your efforts. Attempt 6 (deconz log) now confirms what I expected. I'd like to try something: by adding some additional code to read an attribute of the basic cluster, maybe that helps to do the automagic. I'll let you know when I got the changes ready.

[rotragit]

@SwoopX thank to you for your time trying to resolve this issue. I'm not sure that if you read an attribute of the basic cluster the problem is solved, because if I read the basic cluster attributes with the deConz GUI but I have not pressed the reset button then the module leave the network immediately. The mission should be to understand how to avoid pressing the reset button to make the join of the module to the network. And I believe that if this solution is found, then the second problem will disappear.

EDIT: or we have to setup a process like: 1) in Phoscon go to add a sensor 2) while Phoscon is scanning, give power to the module 3) wait 20/30 seconds 4) deconz is "pairing" the module but the module will leave the network if reset button is not pressed 5) press the reset button 6) wait for the green led, normally in 10/20 seconds from the reset button released 7) read the basic cluster attribute after a while but before the Phoscon scanning ends 8) "ready" button in Phoscon should appear, press it! 9) done

....mumble....

[SwoopX]

Ah, so pushing the reset switch seems to be key here, not reading the cluster attributes?!

Weired, I don't see any hints in the traffic sniff what the difference could be based on the reset button press. Maybe another Legrand security feature?

[rotragit]

Yes, I think that if that step is avoided all the problems are solved.

It would be nice to sniff the network when pairing with the Legrand gateway, but I don't have it (and it's not cheap to buy....). We have to find someone who own it, and able to capture the traffic....

[SwoopX]

Yeah, indeed.

Just to be sure here. Have you already tried a slight modification of your proposed steps above? Namely the following:

0. Reset the device
1. Give power to the module
2. Don't do anything, just wait for like 30 secs.
3. Start Phoscon sensor search
4. Immediately reset the device
5. Read just one attribute of the basic cluster (I would propose modelID) if necessary

[rotragit]

Not exactly the same. I try this evening because I'm "at work" now (well.... smart working of course, but I have to give precedence to work of course ;-) )

[Smanar]

But I think there was a misunderstanding. If the sensor is not in the sensors table, of course is not in the API / json. If the sensor is in the sensors table then is always also in the API / json. I never met the case that is in the sensors table but is not in the API / json.

The device can be in deconz but not in the API, I mean you can see it in the GUI, but no sensor is created. For me the problem is not the pairing process, if you need to press the reset buton, just press it, it 's here for that.

My problem is why the device not appear in the API when it s correctly added in deconz.

BTW, thx for for your patience and attemps logs number 7 and 6 ^^, new reading for me too, I hope you don't need to make the attemps number 47.

Edit But you have used the "special trick" in attemps 6 and 7 ? On the 7 try, there is one where the device was correctly included and without the "special trick" ?

[rotragit]

The device can be in deconz but not in the API, I mean you can see it in the GUI, but no sensor is created.

Yes, in that case I wrote "no sensor created" (no entry in sensors table) and the device is created (there is an entry in devices table, but this is not a prove because if I delete the node in deconz the entry in devices table is not removed. For this reason every time I repeat the process I manually perform a "delete from devices where id=...." and, if it's the case "delete from sensors where sid=...").

For me the problem is not the pairing process, if you need to press the reset buton, just press it, it 's here for that.

Could be, but deconz is able to detect that the button is pressed? Else it doesn't know when is the time to read the attributes.

My problem is why the device not appear in the API when it s correctly added in deconz.

because the sensors table in the db is not populated with a row related to the device :-) Why the table is not populated? Because pressing the button the pairing process is broken, imho :-)

BTW, thx for for your patience and attemps logs number 7 and 6 ^^, new reading for me too, I hope you don't need to make the attemps number 47.

ahah, no problem for me :-)

But you have used the "special trick" in attemps 6 and 7 ?

Yes

On the 7 try, there is one where the device was correctly included and without the "special trick" ?

No.

The only situation where the device is "included correctly without the special trick" is when the same device was first included WITH the special trick AND then is reset (red led) WITHOUT cleaning from deconz (node removed from the GUI AND entries removed from the tables in zll.db). I think this is because the network still remember the pairing some way.

[ebaauw]

See my comment in the other issue: https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2629#issuecomment-606704568

The only situation where the device is "included correctly without the special trick" is when the same device was first included WITH the special trick AND then is reset (red led) WITHOUT cleaning from deconz (node removed from the GUI AND entries removed from the tables in zll.db). I think this is because the network still remember the pairing some way.

The pairing info (i.e. the network key, id, channel, etc) is stored on the device, and lost when you reset it, or when it leaves the network. On network level, there's no database of paired devices: if a device knows the key, it has access to the network.

The device descriptors are saved in the deCONZ database (devices and device_descriptors tables); the Basic attributes might be saved in the database as well (zcl_values table); the resource attributes are saved in the nodes (for lights) and sensors tables. If you pair the same device again, while this info is still available, some of the steps in the pairing process might be skipped, since the info is already available.

[rotragit]

@ebaauw yes, here something is missing between point 3 and 4.... We think some security response to the device, because the device leaves the network itself after a while.

Is the source code of deconz open?

[SwoopX]

For the deconz REST API, yes (this is where we are). For deconz core, (unfortunately) no.

[ebaauw]

Is the source code of deconz open?

No, it's not, and it won't be, see https://github.com/dresden-elektronik/deconz-rest-plugin/issues/1688#issuecomment-531196164. No big deal, though, any security response should be handled by the REST API plugin anyways.

yes, here something is missing between point 3 and 4....

I've seen some intermittent issues that sometimes deCONZ won't read the Simple Descriptors while the network is open (it simply doesn't send the request). Restarting deCONZ might help here. Otherwise, close the network (PUT {"permitjoin": 0} to /config), manually read the descriptors, re-open the network (PUT {"permitjoin": 120} to /config), read the Basic attributes to continue the pairing process.

Not sure what's causing this; I haven't tried de-activating the REST API plugin when this happens (in the REST API plugin popup window under the Plugins menu), to see if that's somehow interfering.

[rotragit]

See my comment in the other issue: #2629 (comment)

@ebaauw have you a sequence diagram with deconz core, phoscon and rest api entities? I have to admit that some steps are obscure to me. For example I think the problem is between step 3 and 4 in your comment, and that steps are deconz core, but you say the security must be handled by rest api. So I understand that in the middle rest api comes somehow in action.... In source code of rest api I see "INSERT INTO devices" lines, but there is not any "INSERT INTO sensors". I'm a little bit confused :-)

[ebaauw]

have you a sequence diagram with deconz core, phoscon and rest api entities

No such thing, I'm afraid. The objects managed by the deCONZ core all start with deCONZ::. They're defined in /usr/include/deconz/*.h which is installed by the deconz-dev package. There used to be a documentation site for this C++ API used by plugins to communicate with the core, but that's gone MIA. I think it was probably generated from the comments in the header files, so most info is still available (although not easily readable).

Phoscon is a separate web application, that communicates to deCONZ (or rather to the REST API plugin) over the REST API. Like the core, it's not open source and not part of this GitHub library. In essence it's a collection of html5 and client-side javascript files, that might be served from any web server (no server-side logic). These files are included in the deconz package and installed into /usr/share/deCONZ/webapp/pwa (/usr/share/deCONZ/webapp contains the old web app). There's a server with dresden elektronik somewhere that serves the latest developer version of Phoscon, but I don't know the address.

The core handles the interaction with the radio device (RaspBee or ConBee), using the device's serial interface. The device itself handles the coordinator function in firmware; the core provides access to the ZigBee devices in the coordinator's network through the GUI and through the C++ plugin API. ZigBee messages are received by the device, passed to the core over the serial interface, passed to the REST API over the C++ plugin API, passed to Phoscon over the REST API. Commands from Phoscon are passed to the REST API plugin over the REST API, passed to the core using the C++ plugin API, passed to the device using the serial interface.

For example I think the problem is between step 3 and 4 in your comment, and that steps are deconz core

Could be, but then it's the intermittent problem I've described above. There's no other interaction with the device needed nor possible between these "steps". Any gateway/hub/bridge pairing devices needs to read the descriptors first, before it knows what device it's dealing with and how to configure it.

but you say the security must be handled by rest api.. So I understand that in the middle rest api comes somehow in action

No, the "security" (I understand some magic configuration command, so the device knows it's paired to the right network) comes after reading the descriptors.

see "INSERT INTO devices" lines, but there is not any "INSERT INTO sensors". I'm a little bit confused

They use REPLACE INTO, see https://www.sqlitetutorial.net/sqlite-replace-statement/.

[rotragit]

Just to be sure here. Have you already tried a slight modification of your proposed steps above? Namely the following:

1. Reset the device
2. Give power to the module
3. Don't do anything, just wait for like 30 secs.
4. Start Phoscon sensor search
5. Immediately reset the device
6. Read just one attribute of the basic cluster (I would propose modelID) if necessary

@SwoopX Here you are. The module leaves the network as soon as I read a attribute. I don't know how to read ONLY modelID. I've selected modelID and pressed READ, but I think deconz has read all the aattributes. Attached 8attempt.zip, as usual. 8attempt.zip

[rotragit]

This 9attempt is..... just to prove the reset button here makes the difference. The module was already paired, then I reset the module BUT I don't clean deconz (do not delete the node and do not clean the db). Then I try to pair again and here comes the difference: in Phoscon the scanning goes "Ready" without I have to press the button. I know that's because some info are already in deconz/rest api/db so some steps are skipped. BUT the led remains red and after a while the module itself leaves the network (node goes gray in deconz). Attached 9attemp as usual.

9attempt.zip

[rotragit]

@ebaauw thank you for the explanation :-)

Could be, but then it's the intermittent problem I've described above. There's no other interaction with the device needed nor possible between these "steps". Any gateway/hub/bridge pairing devices needs to read the descriptors first, before it knows what device it's dealing with and how to configure it.

Well, in my case is not intermittent the problem. Any attempt to pair the module has the problem. Why is not possible an interaction between that steps? I agree the gateway has to read the descriptors before dealing and configuring the module, but why cannot be there a security step? Is in the zigbee specs there could be not?

I say this because from my point of view, If I want to "secure" the network (i.e. make able only my devices to connect to my gateway) I could check only the mac addr of the module before reading the descriptors. And if I want to add the "feature" so that my module can be paired only with my gateway I can install a firmware on the module so that it leaves the network if a "special message" is not received......

(.......mmmhhhh, looks like the situation we have.......)

[ebaauw]

I say this because from my point of view, If I want to "secure" the network (i.e. make able only my devices to connect to my gateway) I could check only the mac addr of the module before reading the descriptors.

Sure, you only need to change the ZigBee protocol, providing custom firmware for all your devices.

And if I want to add the "feature" so that my module can be paired only with my gateway I can install a firmware on the module so that it leaves the network if a "special message" is not received......

The coordinator needs to know which device it’s dealing with so it knows to send the special message.

[rotragit]

I belive you (and I'm not and expert of zigbee protocol). My question was because we are between NWK and APL layers. I'm not sure the OSI layers are broken if somebody puts a security control between them. In TCP/IP is usual to do that (packet filters, ip filters, ....) but nobody declare out of standard or the protocol is changed if a network has a firewall implemented. But we are off topic for sure. And also if that's the case (something as Legrand/bTicino has implemented some sort of knockp protocol) we don't know the key to access it. And I have tons of ideas on how they could have implemented an unbreakable handshake based on pub/priv keys.

[ebaauw]

I'm not and expert of zigbee protocol

I’m no expert either, but I have been using deCONZ for a couple of years. I’m quite familiar with the upper layers in the stack (where REST API support for new devices happens), but I don’t know the NWK and MAC details (e.g. the routing issues are beyond me).

My question was because we are between NWK and APL layers.

There is no such in between. The radio firmware handles MAC and NWK and passes the APS payload “up” to the serial client (i.e. the deCONZ core). The core handles the GUI based on the ZDO or ZCL payload in the APS, and passes the APS with the ZDO or ZCL payload to its plugins. If you want to do stateful packet inspection at NWK level, this needs to happen in the device firmware. Or in IP terms: on the router or switch. Afaik there is no ZigBee standard for that (and even if there were, I’m pretty sure each manufacturer would come up with brilliant new interpretations of this standard, making cross-manufacturer compatibility an illusion).

And I have tons of ideas on how they could have implemented an unbreakable handshake based on pub/priv keys.

Unlikely, as they advertise the ZHA (Zigbee Home Automation) profile. I don’t know the details, but professional grade ZigBee networks (based on other profiles like smart energy, building automation, ...) use a more advanced security model, not based on a factory pre-configured shared “secret” to exchange the network key. Each device needs to be configured with a network-specific secret, before it can join the network. I suspect “my” smart electricity meter and “my” smart gas meter are using that (i.e. the smart meters that the network company installed in my home). I think the tool to configure the devices alone will set you back several hundreds of euros; not at all suited for the consumer market.

[rotragit]

Yes, I agree. This evening / night I'll try to do a script to join this module, but first I have to implement an API in rest_api to have the devices table out (I've seen only sensors and lights in json). In this way we can prompt the user to press the button and then go on reading the descriptors. I think this will be the easiest way to implement the module without a fuzzy press/long press/short/press/medium press/hope/hope/hope process :-)

[Smanar]

Too much to read, but I have a question. If I m right you have already successfull pairing the device without the trick ? By sucesssfull pairing I mean in deconz not in the API, and stable, without leaving.

It s the first thing I have asked at the start. First, have the device included and after use the trick.

BTW legrand pairing will be always random, because it depend the device you already have on the netowrk, and the neighboard the device will have.

[rotragit]

@Smanar no, never. I have always pressed the reset button.

[ebaauw]

but first I have to implement an API in rest_api to have the devices table out (I've seen only sensors and lights in json). In this way we can prompt the user to press the button and then go on reading the descriptors.

I don't think that'll work. The database is only written later, so when the row in the devices table appears, the device will have long left the network. You would need to hook onto the NodeAdded event, where the core informs the plugin that it has created a new node. https://github.com/dresden-elektronik/deconz-rest-plugin/blob/ca72c46fd147dcf5c0b04304a96316d2c97ff477/de_web_plugin.cpp#L10944

[Smanar]

@Smanar no, never. I have always pressed the reset button.

No when I say special trick, I mean set phoscon in permet join and read attribute.

The button reset is not a trick, it is here for pairing, you need to use it.

Before all, you need to include the device only with the reset button and setting permit join in phoscon, without using deconz, like for other zigbee devices.

Like that https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2456#issuecomment-603322966

And using the log taked in this try, we can check why deconz don't add the device in api (and I speak about API database , not Deconz database).

If the device is not included in deconz during the procedure, all you can do is useless. If the device don't pass the legrand check, it will make craps as request.

[rotragit]

@Smanar ok, the log are in there two comments:

(https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2456#issuecomment-605681818)

(https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2456#issuecomment-606168336)

[SwoopX]

@Smanar @rotragit If you take a look at my open pull request, the changes for the iHorn sensors, that could eventually do the trick without explicitly reading the cluster. Although it kinda feels like a dirty hack (it probably is), @manup directly intruduced this himself above for the Trust device.

[Smanar]

So on 5th attemp (the 2nd is useless, was before the modification)

20:23:33:914 device announce 0x0004740000A132F6 (0x5D89) mac capabilities 0x8E 20:23:33:914 set fast probe address to 0x0004740000A132F6 (0x5D89) 20:23:34:130 [2] get active endpoints for 0x0004740000a132f6 20:23:34:197 ZDP indication search sensors 0x0004740000A132F6 (0x5D89) cluster 0x8005 20:23:34:197 ZDP indication search sensors 0x0004740000A132F6 (0x5D89) clear timeout on cluster 0x8005 20:23:34:203 [3] get simple descriptor 0x01 for 0x0004740000a132f6 20:23:34:272 void deCONZ::zmNode::setFetched(deCONZ::RequestId, bool) fetched item: 5, node: 0x5D89 20:23:34:272 ~Resource() /lights 0x7fff2e8a87f8 20:23:34:272 [555] Checking for new sensors 20:23:34:290 [000] simple descriptor - EP 0x01 20:23:34:290 [000] simple descriptor - SD 0x00 20:23:34:290 [000] simple descriptor - BREAK 20:23:34:290 [444] Model ID empty 20:23:34:290 [444] Model ID length: 0 20:23:34:290 [444] Model ID: à5œ)š 20:23:34:290 [444] Model ID: 2E8A8D90 20:23:34:290 [444] Manufacturer: à5œ)š 20:23:34:290 [444] Manufacturer: 2E8A8D90 20:23:34:290 [444] unsupp pushback 20:23:34:359 [555] Creating sensors <<< According to your debug file it mean modelid exist. 20:23:34:360 Clear fast probe timeout for cluster 0x0000, 0x0004740000A132F6 20:23:34:365 [000] simple descriptor - EP 0x01 20:23:34:366 [000] simple descriptor - SD 0x00 20:23:34:366 [000] simple descriptor - BREAK 20:23:34:366 [444] return 1 << According to your debug file It mean device is supported.

And we have other sensor (already included ?) that are (re)trying to be included too

20:24:39:397 Node data 0x0004740000a132f0 profileId: 0x0104, clusterId: 0x0B04 20:24:39:398 [555] Checking for new sensors 20:24:39:398 [555] Model ID still empty 20:24:39:398 [555] Device not supported 20:24:39:398 ZCL attribute report 0x0004740000A132F0 for cluster: 0x0B04, ep: 0x01, frame control: 0x18, mfcode: 0x0000

Deconz inclusion is event based, so I m searching wich one event is the last and provoke device memorisation in database and see why it's not trigger by this device but

20:25:00:468 node 00004740000A132F6 leave wait state

So bad inclusion, useless too.

I can't say how using deconz can help the device to be included, but for the moment, the inclusion is don't made correctly.

If someone else with the device can say how he have included it without deconz, I will be happy to know his procedure. @meco489 ?

BTW @SwoopX your file with better debug is so usefull, I think we can add some of your debug line in the official version.

[rotragit]

@Smanar yes, of course. The module we are trying to pair has mac addr 00:04:74:00:00:a1:32:f6.

[SwoopX]

@Smanar Erik pointed me on a mistake I made regarding the garbage in the modelid and manufacturer. Will try to get that sorted out quickly. Also though about pulling the additional debug output. Not sure if it would have any performance impact while not being used.

[SwoopX]

Guys, I've updated my debug version of de_web_plugin.cpp a bit and tweaked sensor creation as well. Worked like a charm for the humidity sensor which gave me a hard time back in the days. It's again to be found here: https://github.com/SwoopX/st_playground/blob/master/de_web_plugin.cpp

@rotragit if it's not too much for you, could you please compile it and join the device once more with that version? Use toe reset switch to have a successful pairing but omit reading the basic attributes. If all goes well, that won't be necessary any more. Thanks!

[rotragit]

MD5 signature of your file used: 9a6a5f54c6ef972f0697ebbdf60c79da de_web_plugin.cpp

Sorry, the sensor was not created. I start sensor scanning in Phoscon, power the module, when the module is in deConz (checking the deConz GUI, else I don't know how) I press the reset button, the led goes green, Phoscon goes timeout with "Failed to connect". Module is in devices table, no sensor is in sensor table. Attached as usual 10attempt :-) 10attempt.zip

P.S: two warning when compiling your file (nothing important anyway):

de_web_plugin.cpp: In member function ‘void DeRestPluginPrivate::addSensorNode(const deCONZ::Node, const deCONZ::NodeEvent)’: de_web_plugin.cpp:3772:13: warning: unused variable ‘inClusterCount’ [-Wunused-variable] int inClusterCount = i->inClusters().size(); ^~~~~~ de_web_plugin.cpp:3773:13: warning: unused variable ‘outClusterCount’ [-Wunused-variable] int outClusterCount = i->outClusters().size();

[Smanar]

You have keep this modification https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2456#issuecomment-605940522 on rest_sebsor.cpp ?

I m going to breakfast, so I will check log later, but I think the procedure will be fine, if phoscon say "failed to connect" but the device appear in deconz (and stay on place) it will be ok for me.