[Request device support] Develco EMI Norwegian HAN

[Jopinder]

Develco Norwegian HAN External Meter Interface

Real-time measurement and reporting of household power.

Product page: https://www.develcoproducts.com/products/meter-interfaces/emi-norwegian-han/

Technical manual with detailed cluster information: https://www.develcoproducts.com/media/3747/emizb-132-technical-manual-emi-norwegian-han.pdf

Node:

Node Info:

Basic Node 0000:

Simple Metering 0702:

Electrical Measurement 0B04:

[ebaauw]

Did you read the attributes of the 0x0702 and 0x0b04 clusters? The 0x0b04 seems to show only default values. The 0x0702 suggests you’re currently consuming 8.3 kW. Can you relate this to what the meter is displaying? Life-time consumption is again the default value.

[Jopinder]

Yes, all values are after clicking "read" and the meter shows ~1-1.2kW atm. To open the HAN-port on Norwegian meters we have to ask the power company to do it, and it says on "My pages" that the port is open. So I'm not sure why it shows default values.

I will request a close and re-opening of the port to see if that helps with the values.

[gantonjo-tnm]

Thank you, @Jopinder, for requesting support for the Develco EMI Norwegian HAN. I have one myself, but have not been able to add it to my Conbee II USB adapter, hence I am not able to get any readings into my HomeAssistant server. I have asked my power provider (LOS) to open my HAN port, and they confirmed that they had enabled my HAN port. Even so, I am not able to see my Develco sensor in ConBee II. I tried using the PhosCon web interface as well as the VNC GUI.

Can you tell me (details) how you managed to bind your Develco HAN to your Deconz adapter?

Thanks in advance and Happy New Year.

[Jopinder]

My power company has confirmed the HAN port is opened after some mailing with their support. I'm going to contact the seller over new year (Wattle/Home Control AS) to ask if they have done anything to the firmware as they claim it only works with their gateway.

@gantonjo-tnm I tried to add it via the web-gui and Sensors -> Add new sensor and chose "Other", but it did not show up. I then reset the Develco and added it again via the web ui. It still didn't show up in the web ui, but this time in the VNC UI as connected (but unable to get any readings from the power meter). So nothing special I did I'm afraid.

Btw, I have Skagerak Energi and the Aidon meter, which meter does LOS use?

[gantonjo-tnm]

@Jopinder LOS uses Kamstrup meters.

[einarjh]

I have the same reader, but my Manufacturer Code comes up as 0x117c as opposed to 0x1015. Very strange, this seems to be the code for IKEA? The sensor is not talking directly to my hub, it's only within range of one of my IKEA bulbs.

My reader is bought from Elektroimportøren, and the web shop lists it as compatible with "Homely": https://www.elektroimportoren.no/emi-norsk-han/4514731/Product.html?Event=livesearch

My meter is also from Aidon. I've gotten a confirmation from my power company that my HAN port is enabled.

@gantonjo-tnm I had to upgrade to the latest deCONZ beta before I was able to get cluster information etc. from my HAN sensor. Got it from here: http://deconz.dresden-elektronik.de/raspbian/beta/

Here are my screenshots:

Node info:

0x000:

0x0702:

0x0b04:

[einarjh]

I just realized that the meter is already turning up in Home-Assistant, probably because I'm on the current deCONZ beta?

But alas, it's not showing anything useful yet (the second screenshot is particularly hilarious)

[gantonjo-tnm]

@einarjh: Very interesting. Thanks for the information, allthough I am not sure how to upgrade my Conbee II to the latest Beta version. I am running my Home Assistant in Docker on an old Apple Mac Mini 2.5 that is running Ubuntu 18.04.3 LTS. Deconz is running as Docker image "marthoc/deconz".

[einarjh]

Derp! I just started reading the documentation for EMIZB-132 a bit closer, and found this:

Seems there is an extra step needed to enable metering data from Aidon meters, which both @Jopinder and I have.

[ebaauw]

Best define these attributes in general.xml and set them thru the deCONZ GUI. You would need to know the type of each attribute.

[gantonjo-tnm]

@einarjh Ok, I managed to update my Docker image to latest "marthoc/deconz", but even so I am not able to find my Develco EMI Norwegian HAN adapter. I have tried to factory reset it and unplugging it after the reset. No luck binding it to my ConBee II :-(

(Since I used docker-compose, "docker-compose down && docker-compose pull && docker-compose up -d" gave me the latest version of the "marthoc/deconz" Docker image):

[einarjh]

I messed around with this some more tonight, but can't quite get it to work.

As far as I can tell from the manufacturer documentation, the manufacture specific attribute ID 0x0302 needs to be added within the 0x0702 cluster, with the manufacturer code set to 0x1015, so I tried adding it like this (with an educated guess for the type, since this is omitted from the documentation):

<attribute-set id="0x0300" description="Develco Specific" mfcode="0x1015">
  <attribute id="0x0302" name="Interface Mode" type="enum16" access="rw" default="0x0200" showas="hex" required="m" mfcode="0x1015">
    <value value="0x0200" name="Norwegian HAN"></value>
    <value value="0x0201" name="Norwegian HAN - Enable extra load. This is need to enable Adion meter communication"></value>
    <value value="0x0202" name="Norwegian HAN - Aidon Meter supporting Norwegian HAN HW interface"></value>
    <value value="0x0203" name="Norwegian HAN - Kaifa meter and Kamstrup meters running old firmware"></value>
  </attribute>
</attribute-set>

That didn't turn up at all, but when I used the IKEA mfcode, it showed up. Why is my Develco device cloning the manufacturer code for IKEA?

Next problem: the attribute id 0x0302 seems to be conflicting with the attribute id for divisor, when I double click on my new attribute, it brings up the widget for divisor, ref. screenshot:

If I give a fictional, unused attribute set ID, it shows the expected widget with a dropdown, but that won't let me read or write, of course. I get UNSUP_MANUF_GENERAL_COMMAND …

[einarjh]

@gantonjo-tnm Sorry, I can't help you with the connectivity issues. I have a feeling that the meter struggles with the range, though, so make sure your nearest node is close enough. I had to move some of my IKEA lights from the IKEA hub over to deCONZ in order to extend the range. Even now it drops out from time to time.

[Jopinder]

That didn't turn up at all, but when I used the IKEA mfcode, it showed up. Why is my Develco device cloning the manufacturer code for IKEA?

@einarjh That's very strange. My Develco is currently connected to ConBee via an IKEA Trådfri spot, and it keeps all the info, 0x1015 etc.

@gantonjo-tnm Yeah, the range on the Develco is kind of crappy, especially if it's mounted on the inside of the fuse box.

[ebaauw]

so I tried adding it like this

That looks good to me, but you might want to set required to o.

That didn't turn up at all, but when I used the IKEA mfcode, it showed up. Why is my Develco device cloning the manufacturer code for IKEA?

I’m afraid you’re running into a limitation of the GUI: it only shows manufacturer-specific clusters and attributes for the same manufacturer code as the device reports in the Node Descriptor. You see the attribute, but it won’t work, as the device expects a different manufacturer code in the read and write commands.

Why they use the IKEA code is beyond me, I’ve seen iCasa use the Philips code, so their lights are exposed natively to HomeKit by the Hue bridge.

the attribute id 0x0302 seems to be conflicting with the attribute id for divisor

That’s another limitation in the GUI: it doesn’t support manufacturer-specific attributes with the same ID as a standard attribute. Ran into this for the poweron colour settings of Hue lights. If you comment out the standard attribute, you should be able to interact with the manufacturer-specific attribute (if it weren’t for the previous issue).

You could try and access the attributes thru the deconz-cli-plugin, but that’s not for the faint-hearted.

[gantonjo-tnm]

Thanks @einarjh and @Jopinder fro your answers. The distance between my Develco and my ConBee II is not more than 20 cm, so I cannot believe distance is the reason for my problems. In addition I powered off all other Zigbee devices in my home to avoid interference.

[Jopinder]

@einarjh How are you able to edit general.xml? I'm using Hass.io/HassOS, and each time I copy the edited file back to the deCONZ container and restart it, the file goes back to default.

Is there some way of forcing a reload of general.xml without restarting the container?

I also tried creating a file with the values and adding it as a second source for the ZCLDB with no luck.

[einarjh]

@Jopinder I'm not using docker, I'm using the .deb file directly on a raspbian install.

[einarjh]

I tried resetting and re-adding the meter. It still came back with the wrong manu ID. Then I tried turning off every router and moving my raspberry pi within range of the reader. Same problem, it still came back with the IKEA ID.

Then I went for broke, reset everything, and started from scratch with a blank gateway and then adding it. I had to reset everything about ten times, because the device kept coming up with blank manu ID and no attributes. When I finally got it synced up and showing all the attributes again, the manufacture ID was set to 0x1135, which is the same as the raspBee hat has. It seems to clone the manufacturer ID of whatever router it talks to when it pairs up.

So I'm pretty much stuck. I can't get it to show up with the correct manufacturer ID, which means I can't get it to use the correct measuring mode for my power meter.

Is this a bug in deCONZ or do I have a faulty zigbee device?

[einarjh]

Why they use the IKEA code is beyond me, I’ve seen iCasa use the Philips code, so their lights are exposed natively to HomeKit by the Hue bridge.

Just to re-iterate. My testing indicates that DeConz is not seeing the correct manufacturer code, it's seeing the code from whatever router it is talking to. I have left it connected for a few days now, with no change.

[SwoopX]

@einarjh Ok, I managed to update my Docker image to latest "marthoc/deconz", but even so I am not able to find my Develco EMI Norwegian HAN adapter. I have tried to factory reset it and unplugging it after the reset. No luck binding it to my ConBee II :-(

@gantonjo-tnm not sure if I understand what you're missing. From the screenshot you posted, the EMI is visible (device 0x0CF2). EMI is supported since deconz version .72, so the update was necessary.

@einarjh The reason why you have previously seen a different manufacturer code is 99% a node descriptor issue I also had to fight with on the develco smoke sensors (#2154, #2052, #1653). Long story short: in deconz core, the node descriptor is NULL, although a valid node descriptor response was transmitted from the device. This is still not resolved however, my sensors work now with the changes comitted.

Regarding the metering cluster manufacturer specific attribute: I've seen that while amending general.xml for the EMI, but that one felt fishy. Like @ebaauw already mentioned, it might be hard to have it set. Maybe that's doable with ma-ca's deconz cli plugin. Anyhow, not haveing set this attribute appropriately might be the reason why the measurement values show defaults?

@einarjh happy to include your changes of general.xml in a pull request, if you want.

EDIT @einarjh I might be able to help you with correcting node info data. Let me test that on my sensors first. If it works, it would also require some help from @Jopinder or @gantonjo-tnm.

[SwoopX]

@einarjh the trick works. You need to open daconz' database with sqlite3 and shut down deconz before. While open, issue the following query and make sure you amend the MAC address to yours:

UPDATE device_descriptors SET data = x'02408015105050000000500000' WHERE device_id = (SELECT id FROM devices WHERE mac = '00:15:bc:AA:BB:CC:DD:EE') AND endpoint = 0 AND type = 2;

I know that data set is not 100% accurate (as it should be DC powered instead of battery), but it will do.

[einarjh]

@SwoopX Perfect! I suspected that messing with some database or cache file would allow me to replace the manufacturer ID somehow, but I didn't know where to start. I am now able to read the value, but not able to write it. Hmmm.

[einarjh]

As to committing my suggested config change, that wouldn't make much sense since one needs to comment out/remove different parts of the file in order to get it to work. Assuming I can make it work.

[SwoopX]

Ah yeah, I see. Well, as @ebaauw suggested, you might wanna give the deconz cli plugin from ma-ca a go. If I recall correctly, it might be able to handle that. I may be able to assist.

[einarjh]

That thing looks daunting. I'll see if I'm able to set it up correctly first.

[einarjh]

OK, I have it set up now, seeing lots of traffic in the nc "shell". Awaiting further instructions :)

[gantonjo-tnm]

Thanks @SwoopX. Being new to this, I failed to see the Develco HAN actually being attached to the ConBee II. Now I can manually read the data from my adapter, but I still cannot see it inside my Home Assistant. I will follow this thread and see what comes out, as I understand I am not the only one trying to get this working for Docker installs (like the one I run on my Ubuntu server). So, thanks to all of you who do the hard work.

[einarjh]

@gantonjo-tnm Out of curiosity, since you don't have the troublesome Aidon meter, could you paste a screenshot of the 0x0702 cluster (Simple Metering) after pressing the read button (the one above the attributes)?

[gantonjo-tnm]

Of course: @einarjh :-) I will screenshot all possible cluster info parts so you all have it at hand:

[einarjh]

I'm kinda jealous now. Nice to see what I should expect to see once I get this working.

Are you sure you're not seeing it in Home-Assistant? Have you checked under Configuration -> Integrations -> deCONZ? A couple of sensors are showing up as the two eye icons as seen here:

[gantonjo-tnm]

@einarjh: No, unfortunately, I am not seeing any sensors in the integrations->deCONZ :-( And it is not visible in the Phoscon APP either.

[SwoopX]

@gantonjo-tnm your reading look reasonable, so those seem to be indeed dependant on the interface mode. We would require to find out how to expose them with correct formatting in deconz. Also, I feel some updates to the REST API would make sense for those devices since it is the first time we have them.

@einarjh I'd have said you should use one of those:

zclattrmanu 0x2931 02 0x0702 0x1015 03020202

The last values, that should be the payload. We need to find out how to correctly build it. Don't have time right now, maybe in the evening I can have another look and verify with a sniff.

[einarjh]

@SwoopX I was able to poll some basic data (manufacturer etc.) from it last night, but the tricky part would be figuring out what to send to it, yes. Thanks for all your help so far!

[einarjh]

If it's useful, this is what is spit out when I do a read of the 0x0302 attribute from the GUI:

<-APS attr 0xA2C7 2 0x0702 0x0302 0x31 00 02

(For future reference, my new shortaddr is 0xA2C7, I've done a few resets since the initial screenshot)

[SwoopX]

Give that one a try. I'm just not sure if that's the correct data type. Should write value "0202".

zclattrmanu 0xA2C7 02 0x0702 0x1015 020203210202

[einarjh]

I tried sending it a few times, mostly I get no response, but I'm getting this about 10% of the time:

<-APS-DATA.confirm FAILED status 0xE9, id = 0x5A, srcEp = 0x01, dstcEp = 0x02, dstAddr = 0xA2C7

[SwoopX]

Hm, tried the command just now and it appears that it should be correct. Just to have a better understanding what's send here:

zclattrmanu 0xA2C7 02 0x0702 0x1015 020203210202

  0xA2C7     02     0x0702 0x1015    02      0203     21     0202
shortaddr endpoint cluster mfcode writeattr attrid datatype value

Please take note that the attribute ID and value must be written in reverse byte order!

So it should have written value 0202 to manufacturer specific attribute 0302. Again, not exactly sure if that's the correct datatype, but it matches the one in your previous screenshot.

Do you still have your custom general.xml in action? Eventually, the datatype is not appropriately there. You used enum16 (which is 31), I used 21 in the command (which is uint16). Could also be 29, which is int16...

I'd rather start with this in the general.xml and then try different datatypes accordingly, if it doesn't work:

<attribute-set id="0x0300" description="Develco Specific" mfcode="0x1015">
  <attribute id="0x0302" name="Interface Mode" type="u16" access="rw" required="m" mfcode="0x1015">
  </attribute>
</attribute-set>

[einarjh]

At some point during my tests today, I was able to write to it, it's now it's set to 0x0202. Still getting no data, though, I'll try setting it to 0x0201.

I need to figure out which version of the command that worked, or whether it was because I pressed the physical button on the meter first.

[einarjh]

If I'm reading your latest question right, @SwoopX, then deconz-cli cares about what's set as the data type in the XML?

[einarjh]

I got it. Looking closer at your annotated explanation, I realized that the attrid was written in reverse byte order, so the proper command is:

zclattrmanu 0xA2C7 02 0x0702 0x1015 020203310202

No fiddling with the actual hardware is required.

And that answers my own question, deconz-cli doesn't seem to care what's set as the data type in the XML.

[einarjh]

I haven't seen any change in the data that is being reported, still getting only the default values. I've tested with both 0x0201 and 0x0202. Very strange.

[SwoopX]

I got it. Looking closer at your annotated explanation, I realized that the attrid was written in reverse byte order

Damn, thought of mentioning it, but actually forgot it. Sorry for that and glad you figured it out. I'll update it above for future reference.

deconz-cli cares about what's set as the data type in the XML?

As to my experience, it does. Played with it on the manufacturer specific attributes for basic cluster yesterday. However, it looks like this is only for value representation in deconz GUI. A read seems to provide the assiciated data type.

I haven't seen any change in the data that is being reported, still getting only the default values. I've tested with both 0x0201 and 0x0202. Very strange.

Sorry to read that, strange indeed. I'm afraid I cannot be of much help in that case. Do you have my node descriptor tweak applied and re-read the values?

@gantonjo-tnm To catch up on your earlier question regarding visibility of the device. Deconz should expose at leat two sensors, but only via REST API. They are not visible in Phoscon (closed source). As this is a yet unique device, some further amendments on the API will be required to provide further values of interest.

[gantonjo-tnm]

@SwoopX : Thanks for your catch up.Accessing the port where the Phoscon is exposed only shows me the Phoscon App. I guess I need a special URL to access the REST API. Are you able to tell me how to access the REST API?

[SwoopX]

Hope you're running on Linux without docker ;)

You can try this https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2121#issuecomment-564770517

However, a shortcut while having deconz DB open would be to run the following query

select * from sensors;

Then, look out for any ZHAconsumption and ZHApower sensors.

[gantonjo-tnm]

Thanks, @SwoopX. Well, I am running the marthoc/deconz container in Docker on my Ubuntu server ;-) I actually found that the persistant storage for deconz on my server is at /opt/deconz/, so I could actually query the database on my server;-)

root@macubuntu:~/docker/dconz# cd /opt/deconz/ root@macubuntu:/opt/deconz# sqlite3 zll.db SQLite version 3.22.0 2018-01-22 18:45:57 Enter ".help" for usage hints. sqlite> select * from sensors; 2|TRÅDFRI remote control|ZHASwitch|TRADFRI remote control|IKEA of Sweden|00:0d:6f:ff:fe:5e:a7:3a-01-1000|1.2.223|{"buttonevent":2002,"lastupdated":"2019-12-06T19:08:26"}|{"alert":"none","battery":87,"group":"57270","on":true,"reachable":true}|{"d":2096,"ep":1,"in":[0,1,4096],"out":[5,6,8],"p":49246}|normal|3 3|Kjøkkenkontroll|ZHASwitch|TRADFRI remote control|IKEA of Sweden|d0:cf:5e:ff:fe:42:3f:c6-01-1000|1.2.214|{"buttonevent":3002,"lastupdated":"2020-01-13T18:57:11"}|{"alert":"none","battery":60,"group":"56285","on":true,"reachable":true}|{"d":2096,"ep":1,"in":[0,1,4096],"out":[5,6,8],"p":49246}|normal|3 1|Daylight|Daylight|PHDL00|Philips|00:21:2e:ff:ff:04:89:64-01|1.0|{"dark":true,"daylight":false,"lastupdated":"2020-01-14T19:33:22","status":230,"sunrise":"2020-01-14T08:04:47","sunset":"2020-01-14T15:05:00"}|{"configured":true,"lat":"58.4343","long":"8.7466","on":true,"sunriseoffset":30,"sunsetoffset":-30}||normal|1

As you can see, there are no ZHAxxx sensors in my DB :-( But in the VNC GUI, the Develco EMI is visible and updates with correct energy values when I order a read from the HAN.

I guess the absence of the ZHAxxx sensors in the database is the reason why they are not visible in Hass.io

[SwoopX]

@gantonjo-tnm Congratz, you promoted yourself to be my new lab rat ;)

Can you please try the following:

1. Shutdown deconz
2. Update the database as mentioned here: https://github.com/dresden-elektronik/deconz-rest-plugin/issues/2127#issuecomment-573210412
3. Start deconz again
4. Wait a moment, shut down deconz again and check the sensors

If that doesn't change anything, try reading the simple descriptors while deconz is running, wait a moment,shut down deconz again and check the sensors.

[gantonjo-tnm]

@SwoopX Done. No new sensors appearing in the zll.db :-(

Device_Descriptors for my device are as follows: 27,14,0,2,4,X'0204015300000600000300200002070407040b0303000a001900',1575475907 28,14,0,0,2,X'02408015105050000000500000',1575659017

[SwoopX]

Hmmm. Could you also please try to search for new sensors in Phoscon? No device reset, no pairing, just the search and cross fingers...

[gantonjo-tnm]

@SwoopX , search for new sensors in Phoscon did not reveal any new sensors, neither in HA nor in the zll.db :-(