krahabb / meross_lan

Home Assistant integration for Meross devices
MIT License
436 stars 47 forks source link

MTS150 support? #112

Closed override80 closed 2 years ago

override80 commented 2 years ago

Hi, I just found that a famous online shop is selling MTS150 thermostatic valves. Is there any support for these?

Thankyou!

krahabb commented 2 years ago

Hello @override80

I'm aware this device is in the market even though I've never seen any.

I can only guess it could be detected by meross_lan but very unlikely managed.

meross_lan should be 'generic' enough to manage basic commands for the device but I guess there are a lot of newer/different protocol messages leading me to think it will hardly work.

If it happens you manage to buy some, you could contribute some traces so I could try reverse engineer these messages: it shouldnt be impossible to implement at least basic support.

Thank you for signaling this!

override80 commented 2 years ago

Unfortunately I got a defective hub and I had to return it. I will keep you updated if I get a new one!

Thank you!

override80 commented 2 years ago

@krahabb i received a new (working) MTS150H kit. I was able to add the MSH300HK hub (FW version 4, I applied all updates) with local custom mqtt thanks to @bytespider tool on my local mqtts (standalone mosquitto, not home assistant one). Looks like the hub is working, i can see messages in MQTT being published. The MTS150 itself is not seen, if you want i can provide all needed traces to you.

At the moment I just see this in MQTT:

/appliance/2108317285157726281348e1e974da5f/publish {"header":{"messageId":"b3da8f20e9f79010560ab71d41a1a150","namespace":"Appliance.Control.Bind","method":"SET","payloadVersion":1,"from":"/appliance/2108317285157726281348e1e974da5f/subscribe","timestamp":1637527376,"timestampMs":262,"sign":"6fe57838435c20f64b47a3918f77abf5"},"payload":{"bind":{"bindTime":1637527376,"time":{"timestamp":1637527376,"timezone":"","timeRule":[]},"hardware":{"type":"msh300hk","subType":"un","version":"4.0.0","chipType":"MT7686","uuid":"2108317285157726281348e1e974da5f","macAddress":"48:e1:e9:74:da:5f"},"firmware":{"version":"4.1.27","homekitVersion":"4.1","compileTime":"2021/09/22 11:52:30 GMT +08:00","encrypt":1,"wifiMac":"46:d9:e7:d7:b7:08","innerIp":"10.11.12.172","server":"10.11.12.98","port":8883,"userId":0}}}}

/appliance/2108317285157726281348e1e974da5f/subscribe {"header": {"messageId": "c3329c66b5064b81ac9c8905c2040e52", "namespace": "Appliance.Hub.Sensor.All", "method": "GET", "payloadVersion": 1, "from": "/appliance/2108317285157726281348e1e974da5f/publish", "timestamp": 1637527375, "timestampMs": 0, "sign": "2d88767309fe3053e84de771db36fb68"}, "payload": {"all": []}}
/appliance/2108317285157726281348e1e974da5f/subscribe {"header": {"messageId": "a8b0212860cc4864bd55a8123c3747c8", "namespace": "Appliance.Hub.Mts100.All", "method": "GET", "payloadVersion": 1, "from": "/appliance/2108317285157726281348e1e974da5f/publish", "timestamp": 1637527375, "timestampMs": 0, "sign": "b70d980eb464b5b9f10d4df63d5d929b"}, "payload": {"all": []}}
/appliance/2108317285157726281348e1e974da5f/subscribe {"header": {"messageId": "68a0c19038084697beccfcd9d8df4cfb", "namespace": "Appliance.Hub.Battery", "method": "GET", "payloadVersion": 1, "from": "/appliance/2108317285157726281348e1e974da5f/publish", "timestamp": 1637527375, "timestampMs": 0, "sign": "3e1d2ae250c375cd3fe1078caecfa93d"}, "payload": {"battery": []}}

/appliance/2108317285157726281348e1e974da5f/publish {"header":{"messageId":"b2f40d9f2ab607c7897a8dd04619a661","namespace":"Appliance.System.Report","method":"PUSH","payloadVersion":1,"from":"/appliance/2108317285157726281348e1e974da5f/publish","timestamp":1637527384,"timestampMs":981,"sign":"57e26064a1014e7c6de591b5adb13ca5"},"payload":{"report":[{"type":"1","value":"1","timestamp":1637527384}]}}

which leads me to think that this is not working as expected (empty payload from mts100). I have internet connection enabled as well, I see a red light on the HUB, but to mee the hub itself looks good (it was detected from your intergration in Home Assistant as well).

If you want to tell me how can I contribute, I would be very glad to help!

Thanks again for your patience

krahabb commented 2 years ago

Hello @override80 ! I guess we can start grabbing a trace from the hub. You can enable tracing in the Hub configuration entry in HA integrations page.

override80 commented 2 years ago

Here you go! https://filebin.net/7nyng0ot4kjbk675

krahabb commented 2 years ago

Awesome! It looks very strange though..tomorrow I'll get to work on it! Thank you

override80 commented 2 years ago

Thanks! Goodnight!

krahabb commented 2 years ago

Well, by the look of it the MTS150 uses the same payloads as the MTS100 series but your trace was overwhelmed by strange 'bind' messages likely pushed by the hub itself. I'm a bit scared of this behaviour because every device I've seen to date just pushes a bit of something when initially connecting to the mqtt broker but then it stops and just plays nicely with your requests (beside the obvious state pushes)

At the moment I've just instructed my code to recognize the mts150 and use it like an mts100 hoping for the best...we'll see

Maybe the hub was in pairing mode at the time and it was just broadcasting these 'bind(s)' as part of the procedure. You can check if it's still spamming your mqtt broker. This spamming could be an issue since I've seen a lot of message requests (from meross_lan) not being responded neither acknowledged nor rejected

I hope to release soon (check the pre-releases/beta) so you'll have a chance to directly try it

Thank you!

override80 commented 2 years ago

I don't know if this makes any sense to you, but the hub has a red light which blinks green every minute or so... maybe there is something wrong in the hub? I can try resetting it and pairing it again

krahabb commented 2 years ago

Does it have internet connectivity? mine (though it's a plain msh300 on older firmware) doesn't work if it's not able to reach out to the net. I don't know why but I guess it is something related to NTP connectivity.

It looks by the log the device has a somewhat correct timestamp though..

override80 commented 2 years ago

It does, via DHCP i am providing my fastweb router as a gateway (without firewall). I tried routing this through my pfsense firewall and dumping packets, I see it's connecting successfully to some NTP servers

override80 commented 2 years ago

I reset the msh300hk and re-setup it. I created another trace, maybe this is somehow better: https://filebin.net/581tucfg694igv3t

krahabb commented 2 years ago

Hello @override80 , The trace looks really like the previous one so I guess 'we have a problem' with this device (maybe, we'll never know for sure until next release) These are my thoughts about the traces:

And these are the next steps:

haroldgrondel commented 2 years ago

Hello, I came across this thread as I am trying to figure out how to automate my smart valves (MT150) thanks to HA. Thanks to you I can see them in HA but appart from an unknown battery status I do not see any sensor data. I'm no developper but as I have a set up of 25 valves with two hubs I am pleased to test / help up to what my skill level allows.

krahabb commented 2 years ago

Hello @haroldgrondel , welcome aboard ;) The mts150, as you can see from this thread, is not 'officially' supported yet but I'm confident we can reach to make it work. The last pre-release (published today) could have some good news (I hope). If you can, just install it and give it a try and then report back. I don't expect flawless functioning, nevertheless I hope the actual beta implementation could give us more insights on how to completely manage that.

In order to speed up things, once you have the pre-release working and able to see your mts150 you can grab a trace and submit it here (follow this to start tracing - more info about tracing in https://github.com/krahabb/meross_lan#troubleshooting.

haroldgrondel commented 2 years ago

I checked the show beta version check box in hacs / add meross lan but max version is 2.3.3. Am I doing something wrong?

krahabb commented 2 years ago

Maybe you'd need to 'refresh' HACS info for the repo. it usually scans github on a rather long polling interval. My HA is actaully showing the available beta

haroldgrondel commented 2 years ago

Worked like a charm, thanks. Clear improvement as now I can see : Sensors

Configuration

Still no values, but it's a step. I will keep you posted if anything changes. If I can be of any help, shoot. Br, Harold

krahabb commented 2 years ago

Nice one...it is just one step in the right direction I was expecting having the entities populated and not 'unavailable' (at least some)

if you can post a trace I'll further inspect what's going on, I'm scared you're falling in the same situation as @override80 where the hub itself is not really cooperating with meross_lan :(

follow https://github.com/krahabb/meross_lan/issues/112#issuecomment-975161170 to start tracing!

override80 commented 2 years ago

Hi @krahabb , same for me! I can even see battery level and temperature, these are the only one populated and not unavailable!

haroldgrondel commented 2 years ago

hi, I activated tracing for 10 min on both hubs (as I have two). How do I retrieve it? (FYI : meross app is acting weird (has problems with the hubs/valves and not with the 'directly connected switched outlets)

krahabb commented 2 years ago

Hello..sorry for the delay @haroldgrondel,

the traces are under 'custom_components/meross_lan/traces': they're .csv files named with the device type (msh300) and the epoch timestamp of their beginning

override80 commented 2 years ago

Hi @krahabb , here is my latest trace https://filebin.net/ckxilwa6fsiui2uy

krahabb commented 2 years ago

Hello @override80, the last trace you posted shows some improvements: -the infamous 'Bind' message is gone so it seems my dumb acknowledge shuts it and that's good. If @haroldgrondel could post it's own traces made with 2.4.0 this could be confirmed for sure -I see the mts150 messages appear in some responses while some others are void. Also it looks like the mts150 is disconnected from the hub most of the time (you can check this by inspecting the 'state' field being set to '2' which means offline while the online state is marked by '1'. The messages related to the mts150 are those which contain the field 'Id' with value '0300FD1B'

What is the actual behaviour of your mts150 entities in HA UI? do they appear as always disconnected or do they sometime show online with some values?

override80 commented 2 years ago

Well @krahabb , basically almost everything is Unavailable except battery and the thermostat itself. Let me attach some screenshots.

I would like to help you with some more details, I wonder if I can sniff traffic sent to meross Mqtt ofgicial servers? Screenshot_20211127-141659 Screenshot_20211127-141644 Screenshot_20211127-141619

krahabb commented 2 years ago

Hello @override80 , I'm out of ideas ot ath moment but still trying to figure out 'what's next'

Regarding sniffing mqtt traffic I guess it would be a little cumbersome since the mqtt implementation needs encrypted (mqtts) traffic and so you cannot easily (unless youre working for a big Agency;) sniff and decrypt that. I remember @albertogeniola did a 'mitm-proxy' in the past to accomplish this but I don't know how hard it could be and if it could still work at this time. Maybe an easier mqtt-relay setup could allow you to track and log the mqtt session but this is just an idea to investigate. Still a decent effort to accomplish if it works since you should (I guess):

To be honest I really don't know how hard it could be: this is just an intuitive idea. As I said @albertogeniola was able in the past to do the mqtt protocol inspection maybe he's up for some hints

krahabb commented 2 years ago

Hello @override80 , I've just found a possible issue in mqtt message signing when the device key is empty (which is likely the vast majority of user cases!) so my code is likely bugged especially when dealing with the hub where the message flow could introduce this issue (I've found this on mine and the suspect came when looking at your trace where a lot of mqtt requests are not replied....(some are some are not but they align so the important messages get mostly discarded by the device because of the incorrect key handling) So, if your hub is on mqtt with an empty key we'll have to fix this before.

I've just patched the code but I don't think I'm able to release it soon. I could just push the commit to the dev branch online but the fix is so easy you'd better off by applying it directly to your actual code base (if you're not scared:)

You should edit the file 'custom_components/meross_lan/meross_device.py' around line 409 in function 'mqtt_request' substitute this code:

self.key or self.replykey,

with this:

self.key,

and restart HA to fix the fix.

After restarting take another trace for the hub but if we're lucky enough the mts150 could pop up working by itself.

EDIT: The other option, without the need to modify source code and restart HA would be to force HTTP protocol in hub integration configuration. This way the mqtt flaw would be bypassed

override80 commented 2 years ago

Ok, I now see temperature and battery :)

Schermata 2021-12-01 alle 15 24 41

Even with the official configuration, I saw that temperature readings are really poor and there is no way to have a decent reading even with calibration. It gets heavily influenced by the radiator itself :/

Traces: https://file.io/LC7EqbztM30M

krahabb commented 2 years ago

well..that's something but still the trace looks weird... It looks (by the trace) that after a while the hub stops responding to my queries for status updates. I would also try to force HTTP in integration config in order to check if it works better or not.

There are a lot of 'drops' in the MQTT flow which is weird. I thought these drops were related to incorrect signing by meross_lan but in my traces with the previous (supposedly) bugged code and with the patch I wrote you these kind of 'missing replies' are not so frequent (not at all with the edit I've posted before)

Also, the battery level in the picture says 96% while the relative message in the trace carries a 63% ..this is hard to explain ;)

There's another clue in the trace though: the mts150 comes online a bit and then drops offline so there's maybe a connection issue between the valve and the hub

override80 commented 2 years ago

Hi @krahabb , this wouldn't surprise me, it's the second defective hub i get from meross. Also with the original firmware, it takes 5 minutes to have temperature calibration applied - I will be returning this again today.

Thanks for devoting so much time to this issue, at this point I will stick with the Meross thinkgs I have and suggest everyone not to buy new Meross products. I also wrote them that they should open source protocol specifications - obviously I got no reply -_-

krahabb commented 2 years ago

Yeah..it's hard for me to say the hub is defective since I never know if meross_lan is acting fine or bad on the device. Nevertheless, if it doesn't work with the official app then it is likely defective

I'm sorry too..very strange you got 2 defective hub in a row. I think Meross are good products overall but yes, there are a lot of faulty devices coming into the market. It happened to me with a bulb a while ago

Thank you anyway for close support while trying to investigate this behaviour: I had the channce to fix a bunch of small issues here and there while checking this :)

haroldgrondel commented 2 years ago

Hello guys, thank you so much for your hard work. I'm sorry I can't be of more help but I'm still trying to figure out whet is the password to access the files on my VM (vdi downloaded for HA)... Sorry to suck that much and not be of any help.

krahabb commented 2 years ago

Don't worry! any contribution is welcomed and makes feeling this community alive and growing: that will make everyone more and more supporting for the benefit of all. I really like it ;)

haroldgrondel commented 2 years ago

I will switch to a raspberry instal and start collecting some hopefully useful data.

krahabb commented 2 years ago

closing since stale and mts150 seems to work from other sources/issues