search

rytilahti / python-miio

Python library & console tool for controlling Xiaomi smart appliances
https://python-miio.readthedocs.io
GNU General Public License v3.0
3.52k stars 543 forks source link

Some devices not responding across subnets. #422

Closed cnrd closed 5 years ago

cnrd commented 5 years ago

Hi

I'm trying to change around a couple of things in my HASS setup, which seems to have surfaced a bug in the communication with chuangmi_ir devices.

I can ping the device just fine:

ping 10.0.3.18
PING 10.0.3.18 (10.0.3.18) 56(84) bytes of data.
64 bytes from 10.0.3.18: icmp_seq=1 ttl=254 time=5.14 ms
64 bytes from 10.0.3.18: icmp_seq=2 ttl=254 time=6.86 ms
64 bytes from 10.0.3.18: icmp_seq=3 ttl=254 time=3.88 ms
64 bytes from 10.0.3.18: icmp_seq=4 ttl=254 time=2.10 ms
--- 10.0.3.18 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3005ms
rtt min/avg/max/mdev = 2.107/4.502/6.866/1.742 ms

but if i try to get info:

mirobo --ip 10.0.3.18 --token [TOKEN] info
ERROR:miio.device:Unable to discover a device at address 10.0.3.18
Error: Unable to discover the device 10.0.3.18

doing the same thing with my Roborock Vacuum works fine:

mirobo --ip 10.0.3.17 --token [TOKEN]
State: Charging
Battery: 100 %
Fanspeed: 100 %
Cleaning since: 0:02:34
Cleaned area: 2.765 m

It does work fine if I have an IP in the same subnet:

mirobo --ip 10.0.3.18 --token [TOKEN] info
chuangmi.ir.v2 v1.2.4_38 ([MAC]) @ 10.0.3.18 - token: [TOKEN]

I have tested on 2 different physical machines in different subnets, so it shouldn't be a problem with my network configuration.

ivancwc commented 5 years ago

yes this is definitely a bug. i bought two chuangmi_ir devices and encountered same issue

cnrd commented 5 years ago

@syssi any chance you know what is going on here? :-)

syssi commented 5 years ago

I own the device and will try to reproduce the issue. I don't know what's going on here. The code is untouched for month. I've no clue why it's broken now.

cnrd commented 5 years ago

I just don't think we have catched this bug before, is it only manifests when trying to communicate across subnets (in my case routed VLANs)

In my case the IR device is in 10.0.3.0/24 If I try to access it from another device in 10.0.3.0/24 then everything is fine. If I try to access it from eg. 10.0.0.0/24 or 10.0.1.0/24 (both of which have access through the firewall) to the device, then it does not work.

One idea I had is just that the device does not like to communicate with IPs that are not in the same subnets as itself, in which case I don't think there is anything we can do.

ivancwc commented 5 years ago

I just don't think we have catched this bug before, is it only manifests when trying to communicate across subnets (in my case routed VLANs)

In my case the IR device is in 10.0.3.0/24 If I try to access it from another device in 10.0.3.0/24 then everything is fine. If I try to access it from eg. 10.0.0.0/24 or 10.0.1.0/24 (both of which have access through the firewall) to the device, then it does not work.

One idea I had is just that the device does not like to communicate with IPs that are not in the same subnets as itself, in which case I don't think there is anything we can do.

for my case, my hassbian and the remote is in the same subnet, sadly i still get the issue

domibarton commented 5 years ago

I've the same issue with my humidifiers!

My test script:

#!/usr/bin/env python

from miio.airhumidifier import AirHumidifier

x = AirHumidifier(ip='10.9.10.23', token='…')
print(x.status())

My error:

Unable to discover a device at address 10.9.10.23
Traceback (most recent call last):
  File "./test.py", line 6, in <module>
    print(x.status())
  File "/opt/homeassistant/homeassistant/lib/python3.5/site-packages/miio/airhumidifier.py", line 280, in status
    self.device_info = self.info()
  File "/opt/homeassistant/homeassistant/lib/python3.5/site-packages/miio/device.py", line 319, in info
    return DeviceInfo(self.send("miIO.info"))
  File "/opt/homeassistant/homeassistant/lib/python3.5/site-packages/miio/device.py", line 224, in send
    self.do_discover()
  File "/opt/homeassistant/homeassistant/lib/python3.5/site-packages/miio/device.py", line 162, in do_discover
    raise DeviceException("Unable to discover the device %s" % self.ip)
miio.exceptions.DeviceException: Unable to discover the device 10.9.10.23

I've the same issue with the v1 and ca1 humidifiers, but not the RoboRock. My HomeAssistant (i.e. where the script run) is in 10.1.10.0/24, while all devices (humidifiers & vacuum) are in 10.9.10.0/24.

I'm sure the tokens and IP address are correct, as I can ping the IP address and the tokens are fetched from the latest iOS backup. The devices work flawlessly in the Mi Home app.

syssi commented 5 years ago

@domibarton Could you capture some network traffic? There must be a difference between the MiHome app (android <-> device) and the python-miio traffic (python-miio <-> device).

domibarton commented 5 years ago

@syssi I didn't capture anything, but I can try if you want. The communication isn't encrypted?

I'm using the iOS client. When starting the humidifier via, isn't the traffic then routed via Xiaomi's cloud service/server?

domibarton commented 5 years ago

So my setup looks like this:

                    +-------------+ 
                    |HomeAssistant| 
                    +------+------+ 
  Trusted VLAN             | 10.1.10.2
  10.1.10.0/24             |              10.1.10.1 +-----------+
  -------------------------+------------------------|           |
                                                    |  Gateway  +--> Internet Of Shit
  -------------------------+------------------------|           |
  Untrusted VLAN           |              10.9.10.1 +-----------+
  10.9.10.0/24             | 10.9.10.23   
                      +----+-----+
                      |Humidifier|
                      +----------+

The firewall is configured as following:

I also tried to ACCEPT everything, to make sure the firewall isn't the issue here. No success either!

I'm now capturing traffic the following ways:

There's no reply to the UDP packet, thus the timeout before the Unable to discover the device… error.

syssi commented 5 years ago

The MiHome app talks the same protocol as python-miio as long the device is available directly. If your are not at home the instructions are delivered (TCP?) via the xiaomi cloud. Your capture proved: The MiHome app is unable to communicate directly, too.

cnrd commented 5 years ago

My guess is still that it is a "security" feature where the device will not respond to messages not coming from the same subnet. @domibarton any chance you are able to replay that first UDP package but change the IP header such that it looks to be coming from the same subnet?

domibarton commented 5 years ago

@syssi OC the app isn't communicating directly, as I'm connected to a different subnet. I assume the app wouldn't "guess" that the device is probably in another routed subnet and just give it a try. I think the app will check the direct connection only when the device IP is in the same subnet / SSID or alike?

@cnrd Mhm good point, I can try to SNAT that UDP package and see if I get a response from the device. Or I could configure the Untrusted VLAN tag on the RPi, so that I've a direct connection.

Nvmd of it, the RoboRock and the humidifiers apparently work differently. The RoboRock is in the same subnet and responds to the UDP packages, while the humidifers stay silent. I'll test the SNAT-thingy this evening (Switzerland UTC+1 ^^) and let you know!

domibarton commented 5 years ago

@cnrd Sorry for the delay, but I was quite busy the last three days. But I've good news:

YAY IT WORKS! I've masqueraded the traffic on the outbound interface of the VLAN/subnet 10.9.10.0/24 on the gateway. Now the device answers properly :)

cnrd commented 5 years ago

Thank you for confirming my suspicion :-) @syssi is there anything we can do about this or do we just have to accept that it only respond if the devices are in the same subnet? (Also I'm not sure if it happens for other devices, but we should probably add something to the documentation, if it can't be fixed).

domibarton commented 5 years ago

Problem

As @cnrd described above, discovering and querying devices across subnets doesn't work. @cnrd experienced this issue with chuangmi_ir devices, while I discovered the same issue with my humidifers!

Please read the comments above for more insights.
Long story, short: I don't know if this is a bug or a feature, but I guess it's a "security feature". In the end it doesn't matter, as this is hardcoded on the Xiaomi devices. Communication happens over UDP and if a packet source doesn't match the subnet of the Xiaomi device, there's simply no response.

Workarounds

Move into the same subnet

The most obvious solution: Move yourself into the same subnet as the Xiaomi devices or vice-versa.

Dual-homed node

Instead of moving your whole HomeAssistant (or whatever hub) into a new subnet, you can also dual-home your node (e.g. server, Raspberry Pi). You can do this either physically by connecting two ethernet cables (requires 2 ethernet ports on your server), or virtually by using VLAN's.

I'd probably prefer VLAN's in this case ;)
Just remember one thing: If your server was connected to a "trusted subnet" before and you're going to dual-home it, think about security. It was probably OK to have no local firewall when your server was connected to the trusted subnet, as you've a central firewall (and no port forwardings) in place. However, now with 2 legs in 2 subnets - one of them probably in a more "insecure" net - you might want to think about configuring a local firewall to prohibit access to private services.

Masquerading

If you've multiple VLAN's/subnets and you're in control over the router in between, then I'd setup masquerading for the outgoing routing interface of the VLAN/subnet where the Xiaomi devices reside. This basically means changing the source address in the UDP packet headers to the IP address of routing interface. If you want to know more about this, just inform yourself about packet masquerading and/or SNAT.

domibarton commented 5 years ago

I don't think we can do anything about that in the miio Python lib. That's basically a "networking issue" and you've to be in control of the router in between to "fake" the source.

At least if we're staying at this communication protocol & payload. There's probably another way to communicate with the Xiaomi devices, but I didn't look into that. Xiaomi's cloud servers also communicate with the devices. However, the devices are initiating the connection there, so that's most likely a dead path too ;)

Nevermind of it, I'd recommend changing the title of this issue to something more generic, as it doesn't only affect IR devices. What I can say right now is that it's also affecting both Xiaomi humidifer types, but not the RoboRock S50. Mentioning this issue in the docs would also be helpful for new users :)

domibarton commented 5 years ago

Btw thanks for your awesome work guys ;)

rytilahti commented 5 years ago

@domibarton considering you have been exploring this issue for a while, it would be really awesome if you could create a PR to add your discoveries to the documentation. A new troubleshooting section would be very helpful (there are also other well-known issues, e.g., the token encryption, that are currently only documented in various github issues which are really hard to locate aftwards).

domibarton commented 5 years ago

@rytilahti I'll have a look into it, as I love Sphinx as a Python dev… ;)

domibarton commented 5 years ago

@cnrd What was your device? I'm currently writing the docs, and I'd like to mention that in it!

domibarton commented 5 years ago

@cnrd Are you OK with the solution, can we close the issue? I think we can't do anything about that, except for the workarounds above.

cnrd commented 5 years ago

👍

ekkljs commented 5 years ago

Can someone help me to set up the masquerading for the outgoing routing interface of the subnet where the Xiaomi devices reside? My HA is on office and it is connected to home network via IPSec.

I can change the config on routers in home network and office network but I am not sure how above can be done. Any help would be appreciated.

domibarton commented 5 years ago

@ekkljs what kind of router do you have?

ekkljs commented 5 years ago

Edgerouter in home network and Pfsense in office network.

ekkljs commented 5 years ago

@domibarton You don't need to tell me exact configuration on any of my router. It would be good starting point if you could give me some general configuration on the router. For example, the HA server is on 10.0.0.x network and Xiaomi devices are on 192.168.1.x network. Many thanks!

domibarton commented 5 years ago

@ekkljs sorry for the delay. Did you read the official troubleshooting guide?

If you’re in control of the router in between, then you have one more chance to get the communication up & running. You can configure IP masquearding on the outgoing routing interface for the subnet where the MI device resides. IP masquerading (NAT) basically changes the source address in the UDP packet to the IP address of the outbound routing interface.

I'm not sure how masquerading works with IPsec, as it's a bit more complicated. However, what you're basically need to do is to setup IP masquerading FROM the 10.0.0.0/24 subnet INTO the 192.168.1.0/24 subnet. As your HA resides in the 10.0.0.0/24 subnet, it will by default send packages with its source IP (i.e. 10.0.0.x), but the Xiaomi device won't respond to it, as the IP isn't part of its subnet (i.e. 192.168.1.0/24). You need to make sure the source IP of the package arriving on the Xiaomi device matches its subnet, thus SNAT/masquerading.

However, as you're using an IPsec tunnel, a simple masquerading might not work. Most likely your IPsec interface might be your WAN interface, thus masquerading might use the WAN IP address instead of the local/left IPsec interface address. However, I found this on the interweb:

After all - this feature was presented in pFsense 2.1 - to make BINAT before IPSEC, this will allow to masquerade all traffic under specific IP and afterthat send it to tunnel.

I'd probably check that out :)

accelle17 commented 4 years ago

How to do this via Unifi USG and switch? I've got my HA on 192.168.1.0/24 subnet while I have moved Xiaomi MIIO devices to 192.168.20.0/24 IOT subnets. I tested adding the specific HA IP as source and the iot subnet as the destination and eth1 (LAN) as outbound-interface. It will be hard if I need to move the MIIOs back to the same HA subnet again like what I did with the gateway..

accelle17 commented 4 years ago

nvm, i changed the outbound-interface from eth1 to eth1.20 (iot vlan) and it worked. i had igmp snooping and igmp enhance enabled though when I was trying to fix the xiaomi gateway problem.

k1ng440 commented 4 years ago

any idea how I do the nat on pfsense?

the HA server is on 192.168.0.x network and Xiaomi devices are on 192.168.50.x network.

Many thanks!

roflcoopter commented 4 years ago

any idea how I do the nat on pfsense?

the HA server is on 192.168.0.x network and Xiaomi devices are on 192.168.50.x network.

Many thanks!

If you're still having issues, this is how i solved 1:1 NAT in pfSense: image

Spent way too many hours on this...

k1ng440 commented 4 years ago

@roflcoopter https://puu.sh/EZ9Q2/9adde81450.png

zuberspace commented 4 years ago

nvm, i changed the outbound-interface from eth1 to eth1.20 (iot vlan) and it worked. i had igmp snooping and igmp enhance enabled though when I was trying to fix the xiaomi gateway problem.

hey, could't you tell me where I have to configure it? I have a Unifi USG and APs but I don't see any options named outbound-interface.

accelle17 commented 4 years ago

this is configured via config.gateway.json, you can look it on google on where to put it and how to configure. this is what I used (192.168.20.x is the iot vlan):

"service": { "nat": { "rule": { "5010": { "description": "hass", "destination": { "address": "192.168.20.0/24" }, "log": "disable", "outbound-interface": "eth1.20", "protocol": "all", "source": { "address": "192.168.1.0/24" }, "type": "masquerade" } } } }

zuberspace commented 4 years ago

this is configured via config.gateway.json, you can look it on google on where to put it and how to configure. this is what I used (192.168.20.x is the iot vlan):

"service": { "nat": { "rule": { "5010": { "description": "hass", "destination": { "address": "192.168.20.0/24" }, "log": "disable", "outbound-interface": "eth1.20", "protocol": "all", "source": { "address": "192.168.1.0/24" }, "type": "masquerade" } } } }

thank you very much. Got it to work. At first I thought that "eth1.20" refers to the ip range 192.168.20.x ... but after checking my network interfaces I figured it is the vlan number.

ipatalas commented 3 years ago

I think I am experiencing the same issue but.. in my case it's the same network.

I have bought Xiaomi Fan recently, integrated it with HA and it was working fine for about a day. Then it magically stopped working. I've run miiocli directly then to see if that's related to HA, the integration itself or something else. Got the same error: image

I double checked the token. Ran the same command for Xiaomi Vacuum and it works just fine. I tried following the path with different subnets because my HA runs in docker so I thought it might be from a different network which would then perfectly make sense with what's covered in that issue.

Well.. I ran a packet sniffer on my router then to see this: image

Those are 3 attempts (6 packets each) from different sources. All of them originate from the same subnet though (to be honest I have only one subnet in my LAN). I'm out of ideas :( Anyone had something similar?

filikun commented 3 years ago

@midijunk @accelle17 Have you found an easier way of configuring this in the USG? I don't feel confident enough to try configuring this config.gateway.json.

zuberspace commented 3 years ago

@filikun Sorry, I configured this once and it works. If you need any help with that we surely can figure something out.

fuomag9 commented 3 years ago

I can confirm that setting up masquerading between vlans on dest port 54321 did indeed work and all miio-based libraries started working for me, so big thanks to @domibarton!

Klapperman commented 3 years ago

I can confirm that setting up masquerading between vlans on dest port 54321 did indeed work and all miio-based libraries started working for me, so big thanks to @domibarton!

@fuomag9 can you give an example of the rule you implemented to do MASQUERADE with a specific port? Currently I have:

iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 192.168.20.0/24 -j MASQUERADE

is adding --dport 54321 enough before the -j?

fuomag9 commented 3 years ago

I can confirm that setting up masquerading between vlans on dest port 54321 did indeed work and all miio-based libraries started working for me, so big thanks to @domibarton!

@fuomag9 can you give an example of the rule you implemented to do MASQUERADE with a specific port? Currently I have:

iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 192.168.20.0/24 -j MASQUERADE

is adding --dport 54321 enough before the -j?

Unfortunately I'm not too familiar with iptables syntax since I'm not using it, but it looks right (I haven't tried it tho)

filikun commented 3 years ago

@filikun Sorry, I configured this once and it works. If you need any help with that we surely can figure something out.

Sorry for the extreme late reply but is this an ok setup? "service": { "nat": { "rule": { "5010": { "description": "hass", "destination": { "address": "XIAOMI_IP/24" }, "log": "disable", "outbound-interface": "eth1.20", "protocol": "all", "source": { "address": "HOME_ASSISTANT_IP/24" }, "type": "masquerade" } } } }

And is "eth1.20" the VLAN of the IoT or the HA network? Both my HA and IoT are on different VLAN where 40 is my IoT and 20 is my HA.

dave-castle commented 2 years ago

Could someone help me out doing this on UDM Pro, please?

hundehausen commented 2 years ago

I need help with my UDM Pro as well.

gespo89 commented 2 years ago

@dave-castle @hundehausen

So, for the UDM pro folks if you never figured this out, just ran into this myself earlier today trying to get my Roborock S7 set up. Unfortunately, the UDM pro doesn't really support this out of the box through the GUI, so you need to do it by hand. Furthermore, if you set the rule up manually, it will wipe it on reboots and firmware upgrades. Here is what you need to do:

  1. SSH into the UDM pro. You may need turn on SSH if it isn't already. Go to setup.ui.com and click the settings at the very bottom of the page. Turn on ssh and add a password. Then you should be able to.

  2. Add your rule as shown above: iptables -t nat -A POSTROUTING -s YOUR_HOMEASSISTANT_IP/24 -d YOUR_DEVICE_IP/24 -j MASQUERADE

  3. Verify the rule is working and your device is functional. Mine still showed as disconnected until I rebooted homeassistant, and then it picked it up. However, you still aren't done, as this will get wiped out the moment the UDM Pro reboots.

  4. Install the UDM pro boot script. Follow the instructions here: https://github.com/boostchicken/udm-utilities/blob/master/on-boot-script/README.md#steps

  5. Make a script in /mnt/data/on_boot.d with contents like this:

#!/bin/sh
iptables -t nat -A POSTROUTING -s YOUR_HOMEASSISTANT_IP/24 -d YOUR_DEVICE_IP/24 -j MASQUERADE
  1. Make it executable: chmod +x your-script.sh

That should do it.

Ming-A commented 2 years ago

so im having similar error '[homeassistant.components.xiaomi_miio.remote] Device unavailable or token incorrect: Unable to discover the device 10.69.20.19'

image

I did this in opnsense and seems like it still doesn't work. i double checked the IP and token and its correct

mac-lucky commented 2 years ago

I can confirm that setting up masquerading between vlans on dest port 54321 did indeed work and all miio-based libraries started working for me, so big thanks to @domibarton!

@fuomag9 can you give an example of the rule you implemented to do MASQUERADE with a specific port? Currently I have:

iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 192.168.20.0/24 -j MASQUERADE

is adding --dport 54321 enough before the -j?

I tried that, but it didn't work. How to pass the port 54321?

jcconnell commented 2 years ago

Thanks to the info in this thread I was able to resolve my vacuum issues after a month or so of ignoring it. Thank you!

Here's a working configuration for EdgeRouter users:

Device LAN Interface
Home Assistant 10.0.10.9/24 eth3
Roborock E25 10.0.1.168/24 switch0
image

NOTE: The protocol could probably be limited to UDP and the destination port to 54321.

hoppel118 commented 2 years ago

I can confirm that setting up masquerading between vlans on dest port 54321 did indeed work and all miio-based libraries started working for me, so big thanks to @domibarton!

@fuomag9 can you give an example of the rule you implemented to do MASQUERADE with a specific port? Currently I have: iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 192.168.20.0/24 -j MASQUERADE is adding --dport 54321 enough before the -j?

I tried that, but it didn't work. How to pass the port 54321?

Hi,

the complete iptables command (for the udm-pro) is the following:

iptables -t nat -A POSTROUTING -s YOUR_HOMEASSISTANT_IP/24 -d YOUR_DEVICE_IP/24 -p UDP -j MASQUERADE --to-ports 54321

This works here for two Xiaomi Smart Fan models:

A long standing „issue“ is solved for me. Thank you so much, guys. :D

Regards Hoppel

syssi commented 2 years ago

If this command is added to the docs please remove the /24 subnet. /32 would be correct for a single host and can be omitted in this case.