search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.11k stars 29.79k forks source link

Xiaomi Gateway not discovered and port 9898 4321 closed #16407

Closed hoedha12 closed 5 years ago

hoedha12 commented 6 years ago

Home Assistant release with the issue:

0.77.2

Last working Home Assistant release (if known): Not known.

Operating environment (Hass.io/Docker/Windows/etc.):

Hass.io running on Ubuntu Desktop 18.04.1 LTS in Lenovo m93p machine (i5 CPU)

Component/platform:

xiaomi_aqara

Description of problem: Cannot get hassio to discover my gateway automatically. When I specified the gateway IP, hassio returned error of cannot communicating with my gateway. I've followed all the steps in the docs page, and also the troubleshooting steps there to no avail. In addition, I have :

  1. Tried 3 different routers : Tenda AC6 wireless router (which I confirm with their customer service as multicast-capable); Huawei HG8245A modem-wireless router from my ISP; and D-LINK L7-N-R2000 wireless router.
  2. Tried in hass.io and homeasssistant in docker
  3. Tried in VMware installation under Win10
  4. Tried switching from wi-fi to ethernet cable

The gateway and sensors all are working fine in Mi Home app. Mi Home app is in latest version. Once I got the gateway, I've updated the gateway to the latest firmware of 1.4.1_157.0143 before adding sensors.

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

xiaomi_aqara:
  gateways:
    - mac: 7811dcb37d__
      key: hh9wckv6rwyd3b__
      host: 192.168.0.108

logger:
  default: warn
  logs:
    homeassistant.components.xiaomi_aqara: debug
    homeassistant.components.sensor.xiaomi_aqara: debug
    homeassistant.components.switch.xiaomi_aqara: debug
    homeassistant.components.binary_sensor: debug
    xiaomi_gateway: debug

I've tried empyting the mac, gave it semicolon, remove 'host',etc, still get the same error. Here is the debug log : 

2018-09-04 05:19:50 DEBUG (SyncWorker_12) [homeassistant.components.xiaomi_aqara] Expecting 1 gateways
2018-09-04 05:19:50 INFO (SyncWorker_12) [homeassistant.components.xiaomi_aqara] Discovering Xiaomi Gateways (Try 1)
2018-09-04 05:19:50 INFO (SyncWorker_12) [xiaomi_gateway] Xiaomi Gateway 7811dcb37d98 configured at IP 192.168.0.108:9898
2018-09-04 05:19:50 DEBUG (SyncWorker_12) [xiaomi_gateway] _send_cmd >> b'{"cmd":"read","sid":"7811dcb37d98"}'
2018-09-04 05:20:00 WARNING (MainThread) [homeassistant.setup] Setup of xiaomi_aqara is taking over 10 seconds.
2018-09-04 05:20:00 ERROR (SyncWorker_12) [xiaomi_gateway] Cannot connect to Gateway
2018-09-04 05:20:00 ERROR (SyncWorker_12) [xiaomi_gateway] No data in response from hub None
2018-09-04 05:20:00 INFO (SyncWorker_12) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-04 05:20:00 DEBUG (SyncWorker_12) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-04 05:20:10 ERROR (SyncWorker_12) [xiaomi_gateway] Cannot connect to Gateway
2018-09-04 05:20:10 INFO (SyncWorker_12) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-04 05:20:10 DEBUG (SyncWorker_12) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-04 05:20:20 ERROR (SyncWorker_12) [xiaomi_gateway] Cannot connect to Gateway
2018-09-04 05:20:20 INFO (SyncWorker_12) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-04 05:20:20 DEBUG (SyncWorker_12) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-04 05:20:30 ERROR (SyncWorker_12) [xiaomi_gateway] Cannot connect to Gateway
2018-09-04 05:20:30 INFO (SyncWorker_12) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-04 05:20:30 DEBUG (SyncWorker_12) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-04 05:20:40 ERROR (SyncWorker_12) [xiaomi_gateway] Cannot connect to Gateway
2018-09-04 05:20:40 INFO (SyncWorker_12) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-04 05:20:40 DEBUG (SyncWorker_12) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-04 05:20:50 ERROR (SyncWorker_12) [xiaomi_gateway] Cannot connect to Gateway
2018-09-04 05:20:55 INFO (SyncWorker_12) [xiaomi_gateway] Gateway discovery finished in 5 seconds
2018-09-04 05:20:55 INFO (SyncWorker_12) [xiaomi_gateway] Creating Multicast Socket
2018-09-04 05:20:55 DEBUG (SyncWorker_12) [homeassistant.components.xiaomi_aqara] Gateways discovered. Listening for broadcasts
2018-09-04 05:20:55 INFO (MainThread) [homeassistant.components.binary_sensor] Setting up binary_sensor.xiaomi_aqara

Here is the nmap scan of port 9898 and 4321 of gateway IP : 

root@hoedha-m93p:/# nmap -sU 192.168.0.108 -p 9898,4321

Starting Nmap 7.60 ( https://nmap.org ) at 2018-09-04 12:26 WIB
Nmap scan report for 192.168.0.108
Host is up (0.12s latency).

PORT     STATE  SERVICE
4321/udp closed rwhois
9898/udp closed monkeycom
MAC Address: 78:11:DC:B3:7D:98 (Xiaomi Electronics,co.)

Nmap done: 1 IP address (1 host up) scanned in 13.77 seconds
root@hoedha-m93p:/# nmap -sU 192.168.0.108 -p 9898,4321

Starting Nmap 7.60 ( https://nmap.org ) at 2018-09-04 12:27 WIB
Nmap scan report for 192.168.0.108
Host is up (0.16s latency).

PORT     STATE  SERVICE
4321/udp closed rwhois
9898/udp closed monkeycom
MAC Address: 78:11:DC:B3:7D:98 (Xiaomi Electronics,co.)

Nmap done: 1 IP address (1 host up) scanned in 13.96 seconds
root@hoedha-m93p:/# nmap -sU 192.168.0.108 -p 9898,4321

Starting Nmap 7.60 ( https://nmap.org ) at 2018-09-04 12:29 WIB
Nmap scan report for 192.168.0.108
Host is up (0.0024s latency).

PORT     STATE  SERVICE
4321/udp closed rwhois
9898/udp closed monkeycom
MAC Address: 78:11:DC:B3:7D:98 (Xiaomi Electronics,co.)

Nmap done: 1 IP address (1 host up) scanned in 13.56 seconds

And here is the tcpdump at the restart of hassio : 

root@hoedha-m93p:/# tcpdump -A -i eno1  udp port 4321 or udp port 9898
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eno1, link-type EN10MB (Ethernet), capture size 262144 bytes
12:33:26.302007 IP hoedha-m93p.60529 > 192.168.0.108.9898: UDP, length 35
E..?q.@.@.Gw...e...l.q&..+.^{"cmd":"read","sid":"7811dcb37d98"}
12:33:36.317824 IP hoedha-m93p.60529 > 192.168.0.108.9898: UDP, length 23
E..3s.@.@.D....e...l.q&....R{"cmd" : "get_id_list"}
12:33:46.330963 IP hoedha-m93p.60529 > 192.168.0.108.9898: UDP, length 23
E..3v.@.@.B....e...l.q&....R{"cmd" : "get_id_list"}
12:33:56.344100 IP hoedha-m93p.60529 > 192.168.0.108.9898: UDP, length 23
E..3y.@.@.>....e...l.q&....R{"cmd" : "get_id_list"}
12:34:06.357121 IP hoedha-m93p.60529 > 192.168.0.108.9898: UDP, length 23
E..3. @.@.6x...e...l.q&....R{"cmd" : "get_id_list"}
12:34:16.370379 IP hoedha-m93p.60529 > 192.168.0.108.9898: UDP, length 23
E..3..@.@.1....e...l.q&....R{"cmd" : "get_id_list"}
12:34:26.382218 IP hoedha-m93p.53878 > 224.0.0.50.4321: UDP, length 15
E..+..@...>....e...2.v.....h{"cmd":"whois"}
^C
7 packets captured
7 packets received by filter
0 packets dropped by kernel

Traceback (if applicable):

Additional information:

dshokouhi commented 6 years ago

I had similar issues and I ended up adding the following to the configuration:

discovery_retry: 5

this ended up making the gateway always discoverable

hoedha12 commented 6 years ago

thanks for the suggestion, but it didn't work.. @dshokouhi I've added your line as follow:

xiaomi_aqara:
  discovery_retry: 5
  gateways:
    - mac: 7811dcb37d__
      key: uxdvhrdsx0j1mz__
      host: 192.168.0.108

Hassio still can't find and communicate with the gateway on restart. Here is the debug log : 

2018-09-05 01:06:55 DEBUG (SyncWorker_8) [homeassistant.components.xiaomi_aqara] Expecting 1 gateways
2018-09-05 01:06:55 INFO (SyncWorker_8) [homeassistant.components.xiaomi_aqara] Discovering Xiaomi Gateways (Try 1)
2018-09-05 01:06:55 INFO (SyncWorker_8) [xiaomi_gateway] Xiaomi Gateway 7811dcb37d98 configured at IP 192.168.0.108:9898
2018-09-05 01:06:55 DEBUG (SyncWorker_8) [xiaomi_gateway] _send_cmd >> b'{"cmd":"read","sid":"7811dcb37d98"}'
2018-09-05 01:07:05 WARNING (MainThread) [homeassistant.setup] Setup of xiaomi_aqara is taking over 10 seconds.
2018-09-05 01:07:05 ERROR (SyncWorker_8) [xiaomi_gateway] Cannot connect to Gateway
2018-09-05 01:07:05 ERROR (SyncWorker_8) [xiaomi_gateway] No data in response from hub None
2018-09-05 01:07:05 INFO (SyncWorker_8) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-05 01:07:05 DEBUG (SyncWorker_8) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-05 01:07:15 ERROR (SyncWorker_8) [xiaomi_gateway] Cannot connect to Gateway
2018-09-05 01:07:15 INFO (SyncWorker_8) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-05 01:07:15 DEBUG (SyncWorker_8) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-05 01:07:25 ERROR (SyncWorker_8) [xiaomi_gateway] Cannot connect to Gateway
2018-09-05 01:07:25 INFO (SyncWorker_8) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-05 01:07:25 DEBUG (SyncWorker_8) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-05 01:07:35 ERROR (SyncWorker_8) [xiaomi_gateway] Cannot connect to Gateway
2018-09-05 01:07:35 INFO (SyncWorker_8) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-05 01:07:35 DEBUG (SyncWorker_8) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-05 01:07:45 ERROR (SyncWorker_8) [xiaomi_gateway] Cannot connect to Gateway
2018-09-05 01:07:45 INFO (SyncWorker_8) [xiaomi_gateway] Discovering Xiaomi Devices
2018-09-05 01:07:45 DEBUG (SyncWorker_8) [xiaomi_gateway] _send_cmd >> b'{"cmd" : "get_id_list"}'
2018-09-05 01:07:55 ERROR (SyncWorker_8) [xiaomi_gateway] Cannot connect to Gateway
2018-09-05 01:08:00 INFO (SyncWorker_8) [xiaomi_gateway] Gateway discovery finished in 5 seconds
2018-09-05 01:08:00 INFO (SyncWorker_8) [xiaomi_gateway] Creating Multicast Socket
2018-09-05 01:08:00 DEBUG (SyncWorker_8) [homeassistant.components.xiaomi_aqara] Gateways discovered. Listening for broadcasts
2018-09-05 01:08:00 INFO (MainThread) [homeassistant.components.binary_sensor] Setting up binary_sensor.xiaomi_aqara

Here is the tcpdump and nmap of the gateway IP : 

root@hoedha-m93p:~# tcpdump -A -i eno1  udp port 4321 or udp port 9898
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eno1, link-type EN10MB (Ethernet), capture size 262144 bytes
08:12:12.046828 IP hoedha-m93p.47612 > 192.168.0.108.9898: UDP, length 35
E..?qu@.@.G....e...l..&..+.^{"cmd":"read","sid":"7811dcb37d98"}
08:12:22.062081 IP hoedha-m93p.47612 > 192.168.0.108.9898: UDP, length 23
E..3r4@.@.Fd...e...l..&....R{"cmd" : "get_id_list"}
08:12:32.075041 IP hoedha-m93p.47612 > 192.168.0.108.9898: UDP, length 23
E..3x1@.@.@g...e...l..&....R{"cmd" : "get_id_list"}
08:12:42.088028 IP hoedha-m93p.47612 > 192.168.0.108.9898: UDP, length 23
E..3.t@.@.8$...e...l..&....R{"cmd" : "get_id_list"}
08:12:52.100997 IP hoedha-m93p.47612 > 192.168.0.108.9898: UDP, length 23
E..3..@.@.4....e...l..&....R{"cmd" : "get_id_list"}
08:13:02.114075 IP hoedha-m93p.47612 > 192.168.0.108.9898: UDP, length 23
E..3..@.@.3....e...l..&....R{"cmd" : "get_id_list"}
08:13:12.125507 IP hoedha-m93p.56449 > 224.0.0.50.4321: UDP, length 15
E..+.
@....x...e...2.......h{"cmd":"whois"}
root@hoedha-m93p:~# nmap -sU 192.168.0.108 -p 9898,4321

Starting Nmap 7.60 ( https://nmap.org ) at 2018-09-05 08:14 WIB
Nmap scan report for 192.168.0.108
Host is up (0.19s latency).

PORT     STATE  SERVICE
4321/udp closed rwhois
9898/udp closed monkeycom
MAC Address: 78:11:DC:B3:7D:98 (Xiaomi Electronics,co.)

Nmap done: 1 IP address (1 host up) scanned in 14.17 seconds

Seems like the gateway doesn't want to communicate with outside. I've disabled-enable the LAN access from dev mode in the mi-home app numerous times. Sometimes I get open|filtered state on the nmap, but most of the time (99%) I get closed state. Both condition result in same outcome, no response from the gateway to 'whois' from the hassio.

Could it be :

  1. I got a defective gateway? But it's working well with mi-home app
  2. The latest firmware of the gateway is blocking outside connection?
  3. The latest mi-home app has a bug in enabling LAN access?
dshokouhi commented 6 years ago

I have the latest gateway firmware and the latest app and my devices work well.

Did you make sure to follow all troubleshooting steps outlined in the docs?

https://www.home-assistant.io/components/xiaomi_aqara/#connection-problem

The only other thing I can think of is the issue may be with your router.

hoedha12 commented 6 years ago

Yep, I've make sure I have followed all the troubleshooting steps :

  1. Make sure you have enabled LAN access. --> done that image

  2. Turn off the firewall on the system where Home Assistant is running.

    root@hoedha-m93p:~# ufw status
Status: inactive

  3. Ensure your router supports multicast as this is a requirement of the Xiaomi Gateway. I've contacted Tenda customer service and they told me that the router support multicast. Morever I've tried using iperf to test multicast capability of the network as to this guide (https://serverfault.com/questions/211482/tools-to-test-multicast-routing). In server side listening and receiving multicast packet(192.168.0.108, hassio installation):

    root@hoedha-m93p:~# iperf -s -u -B 224.0.0.50 -i 1 -p 9898
------------------------------------------------------------
Server listening on UDP port 9898
Binding to local address 224.0.0.50
Joining multicast group  224.0.0.50
Receiving 1470 byte datagrams
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 224.0.0.50 port 9898 connected with 192.168.0.143 port 38458
[ ID] Interval       Transfer     Bandwidth        Jitter   Lost/Total Datagrams
[  3]  0.0- 1.0 sec   125 KBytes  1.02 Mbits/sec  14.600 ms    0/   87 (0%)
[  3]  1.0- 2.0 sec   122 KBytes  1000 Kbits/sec  16.387 ms    0/   85 (0%)
[  3]  2.0- 3.0 sec   134 KBytes  1.09 Mbits/sec  14.239 ms    0/   93 (0%)
[  3]  0.0- 3.0 sec   386 KBytes  1.04 Mbits/sec  13.136 ms    0/  269 (0%)
^Croot@hoedha-m93p:~#

Sending from another machine in the network : 

root@homeassistant:~# iperf -c 224.0.0.50 -u -T 32 -t 3 -i 1 -p9898
------------------------------------------------------------
Client connecting to 224.0.0.50, UDP port 9898
Sending 1470 byte datagrams, IPG target: 11215.21 us (kalman adjust)
Setting multicast TTL to 32
UDP buffer size:  208 KByte (default)
------------------------------------------------------------
[  3] local 192.168.0.143 port 38458 connected with 224.0.0.50 port 9898
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   131 KBytes  1.07 Mbits/sec
[  3]  1.0- 2.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  2.0- 3.0 sec   128 KBytes  1.05 Mbits/sec
[  3]  0.0- 3.0 sec   386 KBytes  1.05 Mbits/sec
[  3] Sent 269 datagrams

So I guess my router really support multicast. I've also tried Huawei HG8245A modem-wireless router from my ISP; and D-LINK L7-N-R2000 wireless router, all with same result.

  1. Try to leave the MAC address mac: blank. --> done that, same result

5.Try to set discovery_retry: 10. --> done that, tried 5, 10, 20 and same result

  1. Try to disable and then enable LAN access. --> yep, numerous times

  2. Hard reset the gateway: Press the button of the gateway 30 seconds and start again from scratch. --> yep, I've lost count of how many times I have reset this gateway to solve this problem

  3. If you are using Home Assistant in Docker, make sure to use --net=host. --> I'm using hass.io which I think will be automatically running in host mode in docker

  4. If you receive an {"error":"Invalid key"} in your log while trying to control the gateway light, you should generate the key again using an Android Phone or alternatively an emulator such as bluestacks. In some instances there is an issue with keys being generated using the iOS application. --> never get this far

  5. If the required library “PyXiaomiGateway” cannot be installed you will need to install some missing system dependencies python3-dev, libssl-dev, libffi-dev manually (e.g., $ sudo apt-get install python3-dev libssl-dev libffi-dev). --> yep, I've run this command to make sure

    root@hoedha-m93p:~# apt install python3-dev libssl-dev libffi-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
libffi-dev is already the newest version (3.2.1-8).
libssl-dev is already the newest version (1.1.0g-2ubuntu4.1).
python3-dev is already the newest version (3.6.5-3ubuntu1).
0 upgraded, 0 newly installed, 0 to remove and 14 not upgraded.

Seems like the communication process stop at this point. Hassio opens for listening multicast connection, but the gateway never send anything...

  1. Is there anything I can do to get more info/detail on this problem?
  2. Is there any way I could downgrade the gateway firmware?
  3. @dshokouhi could you please perform nmap on the gateway to see whether the port is open or closed?
hoedha12 commented 5 years ago

Issue is solved, the gateway is defective in a way and can't (never) open its comm port at 9898 and 4321. I've bought second gateway and it works out of the box.

Haraguroicha commented 5 years ago

I think the issue still exists, because there have the new model of mi-gateway1 which doesn't open 9898 and 4321 port for receive UDP packets, and only the device mac address match following prefix are okay to communicate2 with home-assistant through 9898 and 4321 port

  1. 28:6c:07
  2. 34:CE:00
  3. 28:6C:07

and other wise mac address out of this list will not got any response

References:

  1. How to check the model
  2. How to check the mac address is match?
LounisBou commented 5 years ago

@Haraguroicha is right ! It's a device version problem. But it's NOT an hardware problem, because device can manage UDP multicast. So it's a firmware problem.

But in the post @Haraguroicha links above, people with old gateway version manage to open port 9898 but only in some firmware version.

As far as I know about this issue, it look like some firmware version on some hardware version didn't manage to open port 9898 when device is set on wireless communication via the dev mode of the MiHome app.

Hope someone will find a solution to this problem. (Best source of information about this issue is the post on openhab forum : https://community.openhab.org/t/openhab2-xiaomi-mi-gateway-does-not-respond/52963/)

Quynh-Nguyen commented 5 years ago

You should try connect via UDP protocol:

sudo nmap -sU xx.xx.xx.xx -p 9898,4321                                                                
Password:
Starting Nmap 7.70 ( https://nmap.org ) at 2019-01-16 15:09 +07
Nmap scan report for xx.xx.xx.xx
Host is up (0.088s latency).

PORT     STATE         SERVICE
4321/udp open|filtered rwhois
9898/udp open          monkeycom
MAC Address: 7C:xx:xx..... (Unknown)

Nmap done: 1 IP address (1 host up) scanned in 1.95 seconds
assada commented 5 years ago

Two days ago my gateway worked fine. But now, his return some error when i am try to turning light on. But when i am set brightness manualy its work fine. In dev-states my gateway allways in state off. Recreating key on android doesnt help. Full reset gateway too.

Got error element in data {"error":"Invalid key"}
Quynh-Nguyen commented 5 years ago

@Assada Yes I'm facing same problem https://github.com/home-assistant/home-assistant/issues/20153

invtechservices commented 5 years ago

I am also now seeing this problem. Updated to both 0.85.1 and firmware 1.4.1_164. I did not have this issue previously with 0.80.2 and the previous firmware. I did not previously get the invalid key errors. I am able to control the gateway with HA. Turn on/off light... but the first action - ie. turn on/off light or movement on one of the sensors returns the key error and then basically all stop. I can no longer activate switches attached to the gateway or does HA get any updates from any sensors.

Everything works fine in the XIaomi app.

[xiaomi_gateway] Got error element in data {"error":"Invalid key"}

syssi commented 5 years ago

@assada @Quynh-Nguyen @invtechservices Please don't hijack this issue. If you receive always a "invalid key" error create a new issue. If you receive this error sometimes please follow this one: https://github.com/home-assistant/home-assistant/issues/20153

syssi commented 5 years ago

The solution for this issue is: https://github.com/home-assistant/home-assistant/issues/16407#issuecomment-450794763

xbmcnut commented 5 years ago

FYI, my device with MAC 34:CE:00 did not have ports open after a factory reset. Had to solder three wires and send it a serial command using the documentation here https://community.openhab.org/t/solved-openhab2-xiaomi-mi-gateway-does-not-respond/52963/126. Works a treat now after 72hrs with no sleep!

MazaHaka7sk commented 5 years ago

Открытие портов 9898 Xiaomi Gateway https://www.youtube.com/watch?v=AhkPdIujzM0&t=1s