krahabb / meross_lan

Home Assistant integration for Meross devices
MIT License
415 stars 45 forks source link

connecting Refoss EM06 via local MQTT #476

Open cbergmann opened 1 month ago

cbergmann commented 1 month ago

Hi,

I have a Refoss Smart Energy Monitor EM06 to monitor the power usage of some of my appliances. It seems that Refoss and Meross are somehow connected/the same company therefore I tried this integration but only with limited success. Following is my writeup of what I did and what works.

When using the Refoss app I can pair the device with the Refoss cloud and see the collected data. On my HA setup I strive to not have any dependencies to external cloud services. Therefore I wanted to pair this device locally to my local mosquitto MQTT server. As my mosquitto Server normally does not allow anonymous connections I added another listener with the following config:

listener 8884
allow_anonymous true
require_certificate false
use_username_as_clientid true
acl_file /etc/mosquitto/meross.acl
certfile /etc/mosquitto/certs/tls.crt
cafile /etc/mosquitto/certs/tls.crt
keyfile /etc/mosquitto/certs/tls.key

The DHCP server is configured to give the EM06 a fixed IP and the firewall is configured to only allow that IP to connect to that port. /etc/mosquitto/meross.acl reads as follows:

user 48:e1:e9:e9:2d:1d
topic readwrite /appliance/#

Next I did a hardware-reset on the EM06 by pressing the button for around 5 seconds. The device confirmed that by a red LED and after restarting blinking green.

Thereafter I installed Version v5.3.0 of the meross_lan Integration from via HACS and restarted Home Assistant.

I then installed the MerossBLE app onto my phone. The device was found and correctly reported in the Discovered Device secion as followed.

device_info

I entered the hostname of my MQTT Server (redacted as mqtt.my.local) in the "Server" field and 8884 in the "Port" field. I kept 0 the UserId field and entered a short passphrase as "Key". Then I selected "Write". In the "Time Configuration" section I selected "Europe/Berlin" as my "Timezone" and also selected "Write". As a last step in the app I selecte "Scan" in the "WIFI Configuration" section. Then selecting my wifi and entering the password before also confirming with "Write". The LED of the device then turns to solid green.

In the mosquitto log I see the device connecting with username 48:e1:e9:e9:2d:1d. It subscribes to /appliance/2401066372070174070148e1e9e92d1d/subscribe and the following three messages are exchanged:

/appliance/2401066372070174070148e1e9e92d1d/publish {"header":{"messageId":"39fb7d784b361cdca356b4c184962afc","namespace":"Appliance.Control.Bind","triggerSrc":"DevBoot","method":"SET","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/subscribe","uuid":"2401066372070174070148e1e9e92d1d","timestamp":1721545595,"timestampMs":371,"sign":"33b22ff8d7889391f54b5dc12de28a6a"},"payload":{"bind":{"bindTime":1721545595,"time":{"timestamp":1721545595,"timezone":"Europe/Berlin","timeRule":[[1729990800,3600,0],[1743296400,7200,1],[1761440400,3600,0],[1774746000,7200,1],[1792890000,3600,0],[1806195600,7200,1],[1824944400,3600,0],[1837645200,7200,1],[1856394000,3600,0],[1869094800,7200,1],[1887843600,3600,0],[1901149200,7200,1],[1919293200,3600,0],[1932598800,7200,1],[1950742800,3600,0],[1964048400,7200,1],[1982797200,3600,0],[1995498000,7200,1],[2014246800,3600,0],[2026947600,7200,1]]},"hardware":{"type":"em06","subType":"eu","version":"2.0.0","chipType":"esp32-c3","uuid":"2401066372070174070148e1e9e92d1d","macAddress":"48:e1:e9:e9:2d:1d"},"firmware":{"version":"2.3.8","compileTime":"May 14 2024 -- 15:59:34","encrypt":1,"wifiMac":"48:e1:e9:e9:2d:1d","innerIp":"192.168.2.174","server":"mqtt.my.local","port":8884,"userId":0}}}}
/appliance/2401066372070174070148e1e9e92d1d/subscribe {"header":{"messageId":"39fb7d784b361cdca356b4c184962afc","namespace":"Appliance.Control.Bind","method":"SETACK","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/publish","triggerSrc":"CloudControl","timestamp":1721545596,"timestampMs":0,"sign":"be70c34a6db56609e2e9d3ade91d335e"},"payload":{}}
/appliance/2401066372070174070148e1e9e92d1d/publish {"header":{"messageId":"39fb7d784b361cdca356b4c184962afc","namespace":"Appliance.Control.Bind","triggerSrc":"CloudControl","method":"ERROR","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/publish","uuid":"2401066372070174070148e1e9e92d1d","timestamp":1721545595,"timestampMs":431,"sign":"33b22ff8d7889391f54b5dc12de28a6a"},"payload":{"error":{"code":5001,"detail":"sign error"}}}

This seems to repeat every few seconds.

Next I go to Home Assistant and it discovers the Meros Lan Integration (my HA is in german so some labels might be not correct here). I select "Configure" and enter the same passphrase as device key. It discovers the em06 device immediately and I can configure it.

While doing that the following MQTT messages are exchanged:

/appliance/2401066372070174070148e1e9e92d1d/publish {"header":{"messageId":"447d01e1e2ac841a327279a4598fe69a","namespace":"Appliance.Control.Bind","triggerSrc":"DevBoot","method":"SET","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/subscribe","uuid":"2401066372070174070148e1e9e92d1d","timestamp":1721545846,"timestampMs":141,"sign":"6bb4b4c076b21ee8e9133a3758b2fc0b"},"payload":{"bind":{"bindTime":1721545846,"time":{"timestamp":1721545846,"timezone":"Europe/Berlin","timeRule":[[1729990800,3600,0],[1743296400,7200,1],[1761440400,3600,0],[1774746000,7200,1],[1792890000,3600,0],[1806195600,7200,1],[1824944400,3600,0],[1837645200,7200,1],[1856394000,3600,0],[1869094800,7200,1],[1887843600,3600,0],[1901149200,7200,1],[1919293200,3600,0],[1932598800,7200,1],[1950742800,3600,0],[1964048400,7200,1],[1982797200,3600,0],[1995498000,7200,1],[2014246800,3600,0],[2026947600,7200,1]]},"hardware":{"type":"em06","subType":"eu","version":"2.0.0","chipType":"esp32-c3","uuid":"2401066372070174070148e1e9e92d1d","macAddress":"48:e1:e9:e9:2d:1d"},"firmware":{"version":"2.3.8","compileTime":"May 14 2024 -- 15:59:34","encrypt":1,"wifiMac":"48:e1:e9:e9:2d:1d","innerIp":"192.168.2.174","server":"mqtt.my.local","port":8884,"userId":0}}}}
/appliance/2401066372070174070148e1e9e92d1d/subscribe {"header":{"messageId":"447d01e1e2ac841a327279a4598fe69a","namespace":"Appliance.Control.Bind","method":"SETACK","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/publish","triggerSrc":"CloudControl","timestamp":1721545847,"timestampMs":0,"sign":"28666f3c265c452074c5a4bd2c21e336"},"payload":{}}
/appliance/2401066372070174070148e1e9e92d1d/subscribe {"header":{"messageId":"19441976e37c4caaa63059a7eb6b21c3","namespace":"Appliance.System.Ability","method":"GET","payloadVersion":1,"from":"/appliance/meross_lan/publish","timestamp":1721545847,"timestampMs":0,"sign":"479ed97aec5452e9ce30c07275989254"},"payload":{"ability":{}}}
/appliance/meross_lan/publish {"header":{"messageId":"19441976e37c4caaa63059a7eb6b21c3","namespace":"Appliance.System.Ability","method":"GETACK","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/publish","uuid":"2401066372070174070148e1e9e92d1d","timestamp":1721545846,"timestampMs":400,"sign":"4aa92852c5194199113c1372a04813bc"},"payload":{"payloadVersion":1,"ability":{"Appliance.Config.Key":{},"Appliance.Config.WifiList":{},"Appliance.Config.Wifi":{},"Appliance.Config.WifiX":{},"Appliance.Config.Trace":{},"Appliance.Config.Info":{},"Appliance.System.All":{},"Appliance.System.Hardware":{},"Appliance.System.Firmware":{},"Appliance.System.Debug":{},"Appliance.System.Online":{},"Appliance.System.Time":{},"Appliance.System.Clock":{},"Appliance.System.Ability":{},"Appliance.System.Runtime":{},"Appliance.System.Report":{},"Appliance.System.Position":{},"Appliance.Control.Multiple":{"maxCmdNum":3},"Appliance.Control.Bind":{},"Appliance.Control.Unbind":{},"Appliance.Control.Upgrade":{},"Appliance.Control.ConsumptionH":{},"Appliance.Control.ElectricityX":{},"Appliance.Control.CircuitFactor":{},"Appliance.Control.AlertConfig":{},"Appliance.Control.AlertReport":{},"Appliance.Control.Sensor.History":{}}}}
/appliance/2401066372070174070148e1e9e92d1d/subscribe {"header":{"messageId":"35dccd8783194427aa69603494c5f5f9","namespace":"Appliance.System.All","method":"GET","payloadVersion":1,"from":"/appliance/meross_lan/publish","timestamp":1721545847,"timestampMs":0,"sign":"ff4ab4c8313bbd3dd0d5e649fc7c6e3e"},"payload":{"all":{}}}
/appliance/meross_lan/publish {"header":{"messageId":"35dccd8783194427aa69603494c5f5f9","namespace":"Appliance.System.All","method":"GETACK","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/publish","uuid":"2401066372070174070148e1e9e92d1d","timestamp":1721545846,"timestampMs":467,"sign":"e51325069980776e64bac3ac1981020c"},"payload":{"all":{"system":{"hardware":{"type":"em06","subType":"eu","version":"2.0.0","chipType":"esp32-c3","uuid":"2401066372070174070148e1e9e92d1d","macAddress":"48:e1:e9:e9:2d:1d"},"firmware":{"version":"2.3.8","compileTime":"May 14 2024 -- 15:59:34","encrypt":1,"wifiMac":"48:e1:e9:e9:2d:1d","innerIp":"192.168.2.174","server":"mqtt.my.local","port":8884,"userId":0},"time":{"timestamp":1721545846,"timezone":"Europe/Berlin","timeRule":[[1729990800,3600,0],[1743296400,7200,1],[1761440400,3600,0],[1774746000,7200,1],[1792890000,3600,0],[1806195600,7200,1],[1824944400,3600,0],[1837645200,7200,1],[1856394000,3600,0],[1869094800,7200,1],[1887843600,3600,0],[1901149200,7200,1],[1919293200,3600,0],[1932598800,7200,1],[1950742800,3600,0],[1964048400,7200,1],[1982797200,3600,0],[1995498000,7200,1],[2014246800,3600,0],[2026947600,7200,1]]},"online":{"status":1,"bindId":"0sImwZY8DdGukrQh","who":1}},"digest":{}}}}
/appliance/2401066372070174070148e1e9e92d1d/publish {"header":{"messageId":"3a769bcad0bd0ef3525f3ffaee380da4","namespace":"Appliance.System.Report","method":"PUSH","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/publish","uuid":"2401066372070174070148e1e9e92d1d","timestamp":1721545849,"timestampMs":261,"sign":"0e3f0ddbba1b6abfdc58a5929e3f03fc"},"payload":{"report":[{"type":"1","value":"1","timestamp":1721545849}]}}

The device is added to Home Assistant with two entities. A disabled one for the sensor protocol and one for the signal strenght which reports 100%.

This is it. After that I don't get any more information in Home Assistant.

While testing this I also got the following messages but I was not able to reproduce them. They are not sent frequently. Maybe the are only sent after a certain power usage but I don't know that.

/appliance/2401066372070174070148e1e9e92d1d/publish {"header":{"messageId":"6ec51bc49cedfced2fcc4c923bc30f4d","namespace":"Appliance.Control.Sensor.History","triggerSrc":"energy task","method":"PUSH","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/publish","uuid":"2401066372070174070148e1e9e92d1d","timestamp":1721547256,"timestampMs":93,"sign":"23d1775268e1045f0a0f0b387657b1cd"},"payload":{"history":[{"channel":1,"capacity":16,"value":[{"timestamp":1721545532,"power":0},{"timestamp":1721545590,"power":0},{"timestamp":1721545647,"power":0},{"timestamp":1721545704,"power":0},{"timestamp":1721545762,"power":0},{"timestamp":1721545820,"power":0},{"timestamp":1721545878,"power":0},{"timestamp":1721545938,"power":0},{"timestamp":1721545998,"power":0},{"timestamp":1721546058,"power":0},{"timestamp":1721546118,"power":0},{"timestamp":1721546178,"power":0},{"timestamp":1721546238,"power":0},{"timestamp":1721546297,"power":0},{"timestamp":1721546357,"power":0},{"timestamp":1721546417,"power":0},{"timestamp":1721546477,"power":0},{"timestamp":1721546537,"power":0},{"timestamp":1721546597,"power":0},{"timestamp":1721546657,"power":0},{"timestamp":1721546717,"power":0},{"timestamp":1721546777,"power":0},{"timestamp":1721546836,"power":0},{"timestamp":1721546896,"power":0},{"timestamp":1721546956,"power":0},{"timestamp":1721547016,"power":0},{"timestamp":1721547076,"power":0},{"timestamp":1721547136,"power":0},{"timestamp":1721547196,"power":0},{"timestamp":1721547255,"power":0}]},{"channel":2,"capacity":16,"value":[{"timestamp":1721545532,"power":3365},{"timestamp":1721545590,"power":4616},{"timestamp":1721545647,"power":0},{"timestamp":1721545704,"power":81946},{"timestamp":1721545762,"power":125777},{"timestamp":1721545820,"power":125795},{"timestamp":1721545878,"power":126069},{"timestamp":1721545938,"power":126385},{"timestamp":1721545998,"power":126401},{"timestamp":1721546058,"power":126568},{"timestamp":1721546118,"power":126500},{"timestamp":1721546178,"power":126094},{"timestamp":1721546238,"power":125653},{"timestamp":1721546297,"power":125380},{"timestamp":1721546357,"power":125057},{"timestamp":1721546417,"power":124911},{"timestamp":1721546477,"power":124677},{"timestamp":1721546537,"power":124380},{"timestamp":1721546597,"power":124207},{"timestamp":1721546657,"power":124025},{"timestamp":1721546717,"power":123784},{"timestamp":1721546777,"power":123628},{"timestamp":1721546836,"power":123340},{"timestamp":1721546896,"power":123313},{"timestamp":1721546956,"power":123143},{"timestamp":1721547016,"power":122988},{"timestamp":1721547076,"power":122788},{"timestamp":1721547136,"power":122736},{"timestamp":1721547196,"power":122936},{"timestamp":1721547255,"power":122888}]},{"channel":3,"capacity":16,"value":[{"timestamp":1721545532,"power":0},{"timestamp":1721545590,"power":0},{"timestamp":1721545647,"power":0},{"timestamp":1721545704,"power":0},{"timestamp":1721545762,"power":0},{"timestamp":1721545820,"power":0},{"timestamp":1721545878,"power":0},{"timestamp":1721545938,"power":0},{"timestamp":1721545998,"power":0},{"timestamp":1721546058,"power":0},{"timestamp":1721546118,"power":0},{"timestamp":1721546178,"power":0},{"timestamp":1721546238,"power":0},{"timestamp":1721546297,"power":0},{"timestamp":1721546357,"power":0},{"timestamp":1721546417,"power":0},{"timestamp":1721546477,"power":0},{"timestamp":1721546537,"power":0},{"timestamp":1721546597,"power":0},{"timestamp":1721546657,"power":0},{"timestamp":1721546717,"power":0},{"timestamp":1721546777,"power":0},{"timestamp":1721546836,"power":0},{"timestamp":1721546896,"power":0},{"timestamp":1721546956,"power":0},{"timestamp":1721547016,"power":0},{"timestamp":1721547076,"power":0},{"timestamp":1721547136,"power":0},{"timestamp":1721547196,"power":0},{"timestamp":1721547255,"power":0}]}]}}
/appliance/2401066372070174070148e1e9e92d1d/publish {"header":{"messageId":"566f8d482bada1f8323337d1919833d1","namespace":"Appliance.Control.Sensor.History","triggerSrc":"energy task","method":"PUSH","payloadVersion":1,"from":"/appliance/2401066372070174070148e1e9e92d1d/publish","uuid":"2401066372070174070148e1e9e92d1d","timestamp":1721547260,"timestampMs":829,"sign":"4222a56ad388c4b33effbd3fa02c7762"},"payload":{"history":[{"channel":4,"capacity":16,"value":[{"timestamp":1721545532,"power":-108},{"timestamp":1721545590,"power":-115},{"timestamp":1721545647,"power":-107},{"timestamp":1721545704,"power":-95},{"timestamp":1721545762,"power":-112},{"timestamp":1721545820,"power":-102},{"timestamp":1721545878,"power":-105},{"timestamp":1721545938,"power":-98},{"timestamp":1721545998,"power":-100},{"timestamp":1721546058,"power":-110},{"timestamp":1721546118,"power":-105},{"timestamp":1721546178,"power":-110},{"timestamp":1721546238,"power":-106},{"timestamp":1721546297,"power":-105},{"timestamp":1721546357,"power":-91},{"timestamp":1721546417,"power":-82},{"timestamp":1721546477,"power":-119},{"timestamp":1721546537,"power":-109},{"timestamp":1721546597,"power":-102},{"timestamp":1721546657,"power":-97},{"timestamp":1721546717,"power":-105},{"timestamp":1721546777,"power":-119},{"timestamp":1721546836,"power":-108},{"timestamp":1721546896,"power":-105},{"timestamp":1721546956,"power":-99},{"timestamp":1721547016,"power":-113},{"timestamp":1721547076,"power":-117},{"timestamp":1721547136,"power":-107},{"timestamp":1721547196,"power":-105},{"timestamp":1721547255,"power":-81}]},{"channel":5,"capacity":16,"value":[{"timestamp":1721545532,"power":0},{"timestamp":1721545590,"power":0},{"timestamp":1721545647,"power":0},{"timestamp":1721545704,"power":0},{"timestamp":1721545762,"power":0},{"timestamp":1721545820,"power":0},{"timestamp":1721545878,"power":0},{"timestamp":1721545938,"power":0},{"timestamp":1721545998,"power":0},{"timestamp":1721546058,"power":0},{"timestamp":1721546118,"power":0},{"timestamp":1721546178,"power":0},{"timestamp":1721546238,"power":0},{"timestamp":1721546297,"power":0},{"timestamp":1721546357,"power":0},{"timestamp":1721546417,"power":0},{"timestamp":1721546477,"power":0},{"timestamp":1721546537,"power":0},{"timestamp":1721546597,"power":0},{"timestamp":1721546657,"power":0},{"timestamp":1721546717,"power":0},{"timestamp":1721546777,"power":0},{"timestamp":1721546836,"power":0},{"timestamp":1721546896,"power":0},{"timestamp":1721546956,"power":0},{"timestamp":1721547016,"power":0},{"timestamp":1721547076,"power":0},{"timestamp":1721547136,"power":0},{"timestamp":1721547196,"power":0},{"timestamp":1721547255,"power":0}]},{"channel":6,"capacity":16,"value":[{"timestamp":1721545532,"power":-426},{"timestamp":1721545590,"power":-438},{"timestamp":1721545647,"power":-443},{"timestamp":1721545704,"power":-502},{"timestamp":1721545762,"power":-497},{"timestamp":1721545820,"power":-493},{"timestamp":1721545878,"power":-498},{"timestamp":1721545938,"power":-498},{"timestamp":1721545998,"power":-490},{"timestamp":1721546058,"power":-506},{"timestamp":1721546118,"power":-506},{"timestamp":1721546178,"power":-501},{"timestamp":1721546238,"power":-492},{"timestamp":1721546297,"power":-490},{"timestamp":1721546357,"power":-503},{"timestamp":1721546417,"power":-497},{"timestamp":1721546477,"power":-514},{"timestamp":1721546537,"power":-513},{"timestamp":1721546597,"power":-495},{"timestamp":1721546657,"power":-506},{"timestamp":1721546717,"power":-509},{"timestamp":1721546777,"power":-497},{"timestamp":1721546836,"power":-498},{"timestamp":1721546896,"power":-505},{"timestamp":1721546956,"power":-500},{"timestamp":1721547016,"power":-499},{"timestamp":1721547076,"power":-506},{"timestamp":1721547136,"power":-502},{"timestamp":1721547196,"power":-494},{"timestamp":1721547255,"power":-517}]}]}}

It seems that these contain the correct power consumtions of the devices in mW. The negative numbers seem to be related to the accuracy of the sensor.

I don't know if this is the correct integration or if I should try something different. If this seems interessting for this integration I am more then happy to provied more information or test a new version. I am sufficentily fluent in python to also write a PR if youre are able to give me a hint where to start.

with kind regards Clemens

krahabb commented 1 month ago

Hello @cbergmann, We're almost there I guess... The problem is this device exposes features (through its 'Abilities') which are not currently supported/understood by meross_lan so we'd need some reverse engineering of these features in order to implement these.

As a starter (and hopefully finisher) we'd need at least a 'Download diagnostics' from the device so that these features (called 'namespaces' in Meross jargon) are euristically queried and dumped. After that, if everything works as usual, we should be able to infer the meaning of the different data and add decoding/encoding capabilities to meross_lan in order to map these to useful entities.

Another option (in parallel), would be to enable the 'Create diagnostic entities' in meross_lan -> Configuration -> Diagnostic. This would try to decode the data exposed by these namespaces and map every single value to a plain sensor entity so that you 'd just have some sensors reporting the raw device state.

The final (more powerful) option would be to use the 'Start diagnostic trace' (always in meross_lan-> Conf -> Diagnostics) to grab the full protocol exchange over a longer period of time so that you could physically operate the device (if anything is 'operable' from that) and meross_lan would dump every single message occuring.

I don't know the device but it looks like being just an energy/power sensor with maybe some configuration options. I think we could get it to work at least in a basic scenario, maybe more...

cbergmann commented 1 month ago

Hi, Attach is the downloaded diagnostics information. I also activated diagnostic entities and now have around 20 entities with more or less usefull information. For each of the 6 sensors I get a total consumption (in Wh it seems), A capacity which is 16 (A I think) in all cases and "circuitfactor_factor" of 1. The one missing information that I can see in the original app that is not here up until now is the current usage in W. I also started the diagnostic trace. How can I download that trace information?

config_entry-meross_lan-98f787ca444fe061299056061a3fddc6.json

krahabb commented 1 month ago

Awesome! The trace is saved under <HA config>custom_components/meross_lan/traces

krahabb commented 1 month ago

The History message looks like reporting the (instant?) power every 60 seconds over the last few minutes. At least, the timestamps of the reported values are spaced by 60 seconds ...

The ConsumptionH reminds me of the similar ConsumptionX usually reported by Meross metering plugs and is likely reporting the energy over last few hours together with total energy x channel (that should be easily implemented).

I think the last missing value (i.e. the actual power reading) is being reported by ElectricityX message which isn't correctly queried by standard euristics..I'll better check what else might be done to correctly query this. At any rate, it could also be PUSHED (asynchronously) by the device itself from time to time so, if we're in luck, the trace could report the layout of the message.

cbergmann commented 1 month ago

Hi, attached are traces from yesterday and today. Unfortunately they stop after a few minutes. Is there something I am doing wrong? traces.tar.gz

krahabb commented 1 month ago

I've released a preview with some (small) improvements on entities mapping for EM06. It is still unknown how to decode current power/current readings but I have patched a bit of the 'discovery/diagnostics' code hoping this new preview could better trace these messages (the candidate for this feature is Appliance.Control.ElectricityX namespace)

You could try install the preview and see if anything improves (the consumption readings should be there though). We'd then need to grab the 'Download diagnostics' again and see if we can extract better info with this new release.

cbergmann commented 1 month ago

Hi, I installed the preview version and got a new diagnostics file. At the first glance this has not helped but maybe you can see something in the file. When this does not help, maybe looking a little deeper into the vendors app might give some insight. config_entry-meross_lan-98f787ca444fe061299056061a3fddc6.json

krahabb commented 1 month ago

Hello, I've searched a bit the internet and found that Refoss has its own integration for HA. Nevertheless, it doesn't look very 'complete' and I don't really know how and if it works so I'm reverse engineering their python code to implement support for EM06 in meross_lan. I have 1 question though, if you're up to help with a bit of 'manual querying' before proceeding: In HA -> Developer Tools -> Services look for service named meross_lan.request and setup the parameters as follows:

Now switch to YAML mode and correct the Namespace field to Appliance.Control.ElectricityX (this is needed because this namespace is not provided by the UI list...)

Now to the (verbose) details: I've found in the original Refoss code that the query payload should be {"electricity": {"channel":65535}} but this is actually 'hard' to implement in meross_lan because of a lot of euristics which (actually) don't expect this query format so, in order to implement this, I would need to introduce new euristics/rules but I don't want to do that if they're not necessary. So, if the {"electricity": {}} payload works then we're all happy and I'd quickly proceed to a 'simple' coding to add the namespace feature, else, if we really need to send {"electricity": {"channel":65535}} no problem, it will just take some more time (and add complexity to the code which I always try to avoid)

Having said that, you should try and see if any of these queries work and then post the service reply (I'm confident at least 1 should work) so that I have the correct payload response to inspect.

Thank you! (by the way, just curious: did you try to use the Refoss integration? if so did it work or not?)

cbergmann commented 1 month ago

Hi this is the result for "{\"electricity\": {}}":

request:
  header:
    messageId: fca51ded841b4746aeaa38435b74f56f
    namespace: Appliance.Control.ElectricityX
    method: GET
    payloadVersion: 1
    from: /appliance/meross_lan/publish
    timestamp: 1721722213
    timestampMs: 0
    sign: 726fb500e6620bb07acf8c4f4ca50e6a
  payload:
    electricity: {}
response:
  header:
    messageId: fca51ded841b4746aeaa38435b74f56f
    namespace: Appliance.Control.ElectricityX
    method: GETACK
    payloadVersion: 1
    from: /appliance/2401066372070174070148e1e9e92d1d/publish
    uuid: 2401066372070174070148e1e9e92d1d
    timestamp: 1721722213
    timestampMs: 384
    sign: 726fb500e6620bb07acf8c4f4ca50e6a
  payload:
    electricity:
      - channel: 1
        current: 0
        voltage: 233546
        power: 0
        mConsume: 1967
        factor: 0
      - channel: 2
        current: 576
        voltage: 232037
        power: 115523
        mConsume: 4878
        factor: 0.8636192083358765
      - channel: 3
        current: 0
        voltage: 232124
        power: 0
        mConsume: 59
        factor: 0
      - channel: 4
        current: 308
        voltage: 233777
        power: 770
        mConsume: 0
        factor: 0.010665059089660645
      - channel: 5
        current: 0
        voltage: 232290
        power: 0
        mConsume: 0
        factor: 0
      - channel: 6
        current: 335
        voltage: 232328
        power: -750
        mConsume: 0
        factor: -0.009644150733947754

and this is the result for "{\"electricity\": {\"channel\":65535}}"

request:
  header:
    messageId: f235da9ebe8d4a52bc8288465e5cdd31
    namespace: Appliance.Control.ElectricityX
    method: GET
    payloadVersion: 1
    from: /appliance/meross_lan/publish
    timestamp: 1721722301
    timestampMs: 0
    sign: 535cd26ec302c66ec9c27faaf190e5ad
  payload:
    electricity:
      channel: 65535
response:
  header:
    messageId: f235da9ebe8d4a52bc8288465e5cdd31
    namespace: Appliance.Control.ElectricityX
    method: GETACK
    payloadVersion: 1
    from: /appliance/2401066372070174070148e1e9e92d1d/publish
    uuid: 2401066372070174070148e1e9e92d1d
    timestamp: 1721722301
    timestampMs: 87
    sign: 535cd26ec302c66ec9c27faaf190e5ad
  payload:
    electricity:
      - channel: 1
        current: 0
        voltage: 233680
        power: 0
        mConsume: 1967
        factor: 0
      - channel: 2
        current: 574
        voltage: 233184
        power: 115185
        mConsume: 4881
        factor: 0.8602570295333862
      - channel: 3
        current: 0
        voltage: 232021
        power: 0
        mConsume: 59
        factor: 0
      - channel: 4
        current: 311
        voltage: 233748
        power: 324
        mConsume: 0
        factor: 0.004454255104064941
      - channel: 5
        current: 0
        voltage: 233313
        power: 0
        mConsume: 0
        factor: 0
      - channel: 6
        current: 339
        voltage: 232127
        power: -10
        mConsume: 0
        factor: -0.0001285076141357422

That looks good.

On your question about the Refoss integration: I looked into this but at that time i did not figure out a way to join the device to the WIFI without using the official app and therefore also using the refoss cloud. That was not an option for me. Now that You mentioned that I tried adding it and after fiddeling a little bit with my firewall I got it working. The only problem was that my HA and the device are on differnt networks. The Integration has no option to specify the IP and relies on sending a broadcast to port 9988. That would not reach the device and therefore setup would fail. When configuring iptables to rewrite the broadcast adress to the device adress before sending the udp datagram out the setup completes and everything works. It seems that the Refoss integration communicates via UDP directly. It queries for the information every 15 seconds and gets a reply with the information. This seems to remove the need for a local MQTT Server. If you are interested in implementing this protocol I have captured a pacap file containing the initial setup and the first minutes of updates. As this contains internal IP adresses and potentialy other sensetive information I am more than happy to send it to you via a secure channel.

krahabb commented 1 month ago

I didn't dig too much in the refoss-ha library supporting this but the traffic over UDP could just be the discovery part since I've found parts of code using 'standard' http traffic to query the device. At any rate, I would not go too far (at the moment) since the http stack should work correctly without many issues.

But the trace could become useful in the future...I've sent you a PM

cbergmann commented 1 month ago

I looked into the pcap files and it seems that your are right. The UDP datagrams are always the same and seem to only contain basic information. The main data seems to be transmitted via http.

krahabb commented 1 month ago

Updated pre-release. Beside the naive naming of some sensors it should work. I've removed the decoding of consumption hourly history since the statistics should be better available through HA.

The only 'open question', provided everything else works as expected, is the interpretation of the energy reading. HA has different configurations in order to manage the possible scenarios and these are more or less based off when (and if) the accumulated values resets from time to time (daily-monthly-yearly....) but I don't have a clear understanding of when the EM06 eventually resets these energy readings. The refoss-ha library uses the label/name "Monthly energy" so I would expect these energy values to reset every month but, even if this is a correct assumption, the reset could actually happen anytime: likely the start of the month but not surely.

We have to see what will happen on these.

At the moment I've just used state_class == total for the entity since the energy might flow in any direction if I'm correct and the meter (and HA statistics too) could then track energy import/export. Just, the code never tells HA if the value has been reset (start of new cycle)

cbergmann commented 1 month ago

Hi thanks again for the fast reply. I will monitor this information and give feedback what happens. Unfortunately I will be on vacation next week and might not have time to see if the reading changes.

Regarding energy flow it might be interesting that the original integration also adds a sensorsensor.em06_xx_this_month_energy_returned entity. I would think that this one would count if energy is flowing back to the provider but I can not see this because I currently don't have that in my setup.

If it helps I here is a screenshot of all the entities for one of the sensors of the original integration. This is the same for all 6 sensors (a1,b1,c1,a2,b2,c2). sensors

cbergmann commented 1 month ago

Also I would suggest naming the entities with a1,b1,c1,a2,b2,c2 because the sensors on the device are labeld like this. This would make it more easy to identify which reading is from wich sensor.