"Keenetic NDMS2 Router" integration doesnt work with remote sites

[banditto9]

The problem

Hello! I'm happy with the HA and how the project is being developed - many thanks for your time and continuous development - as a simple user I appreciate this a lot!

I'm using 3 sites in different locations interconnected with Keenetic routers (under VPN I use WireGuard as the fastest and simplest protocols in setup). Everything is fine with HA including remote cameras and automations with them.

But there is one big issue I can't solve - this is the deivce-tracker/presence detection on remote sites. I use "Keenetic NDMS2 Router" HA built-in integration which works perfectly only in the same LAN segment where HA is running. And I can't setup the rest of 2 remote locations in this integration. Something is inside the core or Telnet protocol is limiting from connection. Integration 'sometimes' 'patially' appears (saying "Initializing" and nothing goes further that message) but in most cases I just get an error in HA. Telnet accounts are correct - I can connect via Telnet on remote LAN to both of Keenetics so this is not a credentials issue for sure.

I have found that another 3rd party integration (https://github.com/notabene00/HA-Keenetic-device_tracker) works fine but that integration doesn't allow to assign zones to the discovered objects. But I can say that HTTP seems to be a bit better solution for the remote VPN spokes in comparison with the Telnet which seems to be sensitive to the network lags and response time (I can be wrong, sorry for that - this is just my assumption).

In regard to the above-said I would be very obliged if you can consider adding an HTTP connection option (in addition to the existing Telnet one) or by making changes to the existing Telnet-based "Keenetic NDMS2 Router" integration so it can work with the remotely connected VPN spokes over Telnet.

Many thanks for your time and attention to this!

What version of Home Assistant Core has the issue?

2023.1.6

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Keenetic NDMS2 Router

Link to integration documentation on our website

https://www.home-assistant.io/integrations/keenetic_ndms2/

Diagnostics information

Setup: HW: Intel NUC (x86) Home Assistant 2023.1.6 Supervisor 2022.12.1 Operating System 9.4 Frontend 20230110.0 - latest

In the log extract below HA tries to connect to one of the remote VPN spokes (WireGuard) which is a Keenetic Viva (KN-1910 model) - the error I get is "Error connecting to telnet server: No expected response from server".

The 3rd spoke (which is Keenetic Peak KN-2710 also connected via WireGuard) doesn't show any error at all.

manufacturer: Keenetic Ltd. vendor: Keenetic series: KN model: Viva (KN-1910) hw_version: 10198000 hw_type: router hw_id: KN-1910 device: Viva region: RU description: Viva (KN-1911) 9ka

2023-01-13 19:08:46.477 DEBUG (SyncWorker_5) [ndms2_client.client] release: 3.09.C.2.0-0 2023-01-13 19:08:46.477 DEBUG (SyncWorker_5) [ndms2_client.client] sandbox: stable 2023-01-13 19:08:46.477 DEBUG (SyncWorker_5) [ndms2_client.client] title: 3.9.2 2023-01-13 19:08:46.477 DEBUG (SyncWorker_5) [ndms2_client.client] arch: mips 2023-01-13 19:08:46.478 DEBUG (SyncWorker_5) [ndms2_client.client] ndm: 2023-01-13 19:08:46.478 DEBUG (SyncWorker_5) [ndms2_client.client] exact: 0-9356adc 2023-01-13 19:08:46.478 DEBUG (SyncWorker_5) [ndms2_client.client] cdate: 22 Dec 2022 2023-01-13 19:08:46.478 DEBUG (SyncWorker_5) [ndms2_client.client] bsp: 2023-01-13 19:08:46.478 DEBUG (SyncWorker_5) [ndms2_client.client] exact: 0-41b8845ff9 2023-01-13 19:08:46.479 DEBUG (SyncWorker_5) [ndms2_client.client] cdate: 22 Dec 2022 2023-01-13 19:08:46.479 DEBUG (SyncWorker_5) [ndms2_client.client] ndw: 2023-01-13 19:08:46.479 DEBUG (SyncWorker_5) [ndms2_client.client] version: 3.9.220 2023-01-13 19:08:46.479 DEBUG (SyncWorker_5) [ndms2_client.client] features: wifi_button,wifi5ghz,led_control,vht2ghz,mimo5ghz,atf2ghz,atf5ghz,dual_image,wifi_ft,wpa3,hwnat 2023-01-13 19:08:46.480 DEBUG (SyncWorker_5) [ndms2_client.client] components: acl,afp,base,cloudcontrol,config-ap,config-client,config-repeater,corewireless,ddns,dhcpd,dlna,dns-filter,dns-https,dns-tls,dot1x,eoip,exfat,ext,fat,ftp,gre,hfsplus,igmp,ike-client,ip6,ipip,ipsec,l2tp,lang-en,mdns,miniupnpd,mws,nathelper-esp,nathelper-ftp,nathelper-h323,nathelper-pptp,nathelper-rtsp,nathelper-sip,ndmp,ndns,ntce,ntfs,openvpn,opkg,opkg-kmod-fs,pingcheck,ppe,pppoe,pptp,sftp,ssh,sstp,sstp-server,storage,trafficcontrol,tsmb,udpxy,usb,usblte,usbmodem,usbnet,usbnet-extra,usbqmi,vpnserver,vpnserver-l2tp,webdav,wireguard 2023-01-13 19:08:46.480 DEBUG (SyncWorker_5) [ndms2_client.client] manufacturer: Keenetic Ltd. 2023-01-13 19:08:46.480 DEBUG (SyncWorker_5) [ndms2_client.client] vendor: Keenetic 2023-01-13 19:08:46.505 DEBUG (SyncWorker_5) [ndms2_client.client] series: KN 2023-01-13 19:08:46.505 DEBUG (SyncWorker_5) [ndms2_client.client] model: Viva (KN-1910) 2023-01-13 19:08:46.505 DEBUG (SyncWorker_5) [ndms2_client.client] hw_version: 10198000 2023-01-13 19:08:46.505 DEBUG (SyncWorker_5) [ndms2_client.client] hw_type: router 2023-01-13 19:08:46.505 DEBUG (SyncWorker_5) [ndms2_client.client] hw_id: KN-1910 2023-01-13 19:08:46.505 DEBUG (SyncWorker_5) [ndms2_client.client] device: Viva 2023-01-13 19:08:46.505 DEBUG (SyncWorker_5) [ndms2_client.client] region: RU 2023-01-13 19:08:46.506 DEBUG (SyncWorker_5) [ndms2_client.client] description: Viva (KN-1911) 9ka 2023-01-13 19:08:46.506 DEBUG (SyncWorker_5) [ndms2_client.client] Raw router info: {'release': '3.09.C.2.0-0', 'sandbox': 'stable', 'title': '3.9.2', 'arch': 'mips', 'ndm': {'exact': '0-9356adc', 'cdate': '22 Dec 2022'}, 'bsp': {'exact': '0-41b8845ff9', 'cdate': '22 Dec 2022'}, 'ndw': {'version': '3.9.220', 'features': 'wifi_button,wifi5ghz,led_control,vht2ghz,mimo5ghz,atf2ghz,atf5ghz,dual_image,wifi_ft,wpa3,hwnat', 'components': 'acl,afp,base,cloudcontrol,config-ap,config-client,config-repeater,corewireless,ddns,dhcpd,dlna,dns-filter,dns-https,dns-tls,dot1x,eoip,exfat,ext,fat,ftp,gre,hfsplus,igmp,ike-client,ip6,ipip,ipsec,l2tp,lang-en,mdns,miniupnpd,mws,nathelper-esp,nathelper-ftp,nathelper-h323,nathelper-pptp,nathelper-rtsp,nathelper-sip,ndmp,ndns,ntce,ntfs,openvpn,opkg,opkg-kmod-fs,pingcheck,ppe,pppoe,pptp,sftp,ssh,sstp,sstp-server,storage,trafficcontrol,tsmb,udpxy,usb,usblte,usbmodem,usbnet,usbnet-extra,usbqmi,vpnserver,vpnserver-l2tp,webdav,wireguard'}, 'manufacturer': 'Keenetic Ltd.', 'vendor': 'Keenetic', 'series': 'KN', 'model': 'Viva (KN-1910)', 'hw_version': '10198000', 'hw_type': 'router', 'hw_id': 'KN-1910', 'device': 'Viva', 'region': 'RU', 'description': 'Viva (KN-1911) 9ka'} 2023-01-13 19:09:16.758 ERROR (SyncWorker_9) [ndms2_client.connection] Error connecting to telnet server: No expected response from server

One the "half-success" remote Keenetics' shows that Telnet connection is established and the drops again. (172.16.82.2 is the IP of the WireGuard tunnel on one of the VPN ends).

I] Jan 14 21:29:52 telnetd: a new connection from ::ffff:172.16.82.2 accepted. [I] Jan 14 21:29:53 ndm: Core::Server: started Session /var/run/ndm.core.socket. [I] Jan 14 21:29:54 ndm: Core::Authenticator: user "ha_monitor" authenticated, realm "Keenetic Viva", tag "cli". [I] Jan 14 21:29:55 telnetd: ::ffff:172.16.82.2 client disconnected. [I] Jan 14 21:29:55 ndm: Core::Session: client disconnected. [I] Jan 14 21:29:55 telnetd: a new connection from ::ffff:172.16.82.2 accepted. [I] Jan 14 21:29:55 ndm: Core::Server: started Session /var/run/ndm.core.socket.

At the same time another integration (Reolink IP cameras) works just fine over the same VPN. The difference that Reolink connects only to the FTP and not Telnet (if this information helps somehow..)

[I] Jan 14 21:33:51 pure-ftpd: (secam@172.16.82.1) [NOTICE] /231 elevators_01_20230114213333.mp4 uploaded (17043683 bytes, 621.25KB/sec). [I] Jan 14 21:33:51 pure-ftpd: (secam@172.16.82.1) [INFO] Logout. [I] Jan 14 21:33:59 pure-ftpd: (secam@172.16.82.2) [NOTICE] /Street_01_20230114213336.mp4 uploaded (6941148 bytes, 223.13KB/sec). [I] Jan 14 21:33:59 pure-ftpd: (secam@172.16.82.2) [INFO] Logout.

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

[home-assistant[bot]]

Hey there @foxel, mind taking a look at this issue as it has been labeled with an integration (keenetic_ndms2) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `keenetic_ndms2` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign keenetic_ndms2` Removes the current integration label and assignees on the issue, add the integration domain after the command.

_{^{(message by CodeOwnersMention)}}

---

keenetic_ndms2 documentation keenetic_ndms2 source _{^{(message by IssueLinks)}}

[foxel]

Found the issue is reproducible with high-latency connections. https://pypi.org/project/ndms2-client/0.1.2/ fixes this for the client library.

Can you try replacing the dependency on your HA instance?

[banditto9]

Hello Andrey and thank you very much for your prompt response and the fix!

I'm an ordinary user and have HAOS installed on x86 - I would appreciate a lot if you can give me a short hint how to replace the dependency..

I tried to install it to the "/custom_components/" folder and activate via configuration.yaml but built-in checker says "Platform error device_tracker.ndms2_client - Integration 'ndms2_client' not found." Dev docs says it should be "homeassistant/components/" path but I have no access to that folder..

/custom_components/ I tried to install it via HACS but it doesn't seem to be a HACS compatible integration.. Tried to get a backup and replace the files inside it but the backup does only a copy of my accessible data folders like blueprints, ssl, custom_components etc.. Sorry for not being helpful - I wish to make a test from my side but due to the lack of knowledge I'm stuck a bit..

[banditto9]

Sorry for bothering, forgot to mention that very close (if not the same) issue has been raised under the following thread: https://github.com/foxel/python_ndms2_client/issues/7

[banditto9]

update: I managed to install the "Keenetic NDMS2 Router" integration as a 'custom component'. I also just added 0.1.2 to be sure it's loaded as a custom and not as a built-in integration.

Sorry, in my previous attempt one of the files were from the old installation and everything messed out. Very sorry for that and thank you for your time - my knowledge of HA has increased today a lot :))

IT WORKS! 🍻 this is magic, everything works just fine and this is fantastic! 3 of my locations with HA(OS) being 1500 km work like they all are in the same LAN. Thank you very much and God bless you!🙏-

[foxel]

@banditto9 thanks for testing, now I'm pretty confident the solution is ok. I still need to make a PR to HA to include the newer dependency. Once the update makes its way to the main branch we can close the issue.

[foxel]

@home-assistant reopen

[banditto9]

@foxel thank you, sure, I already informed one fellow from 4PDA who has raised the issue around 2 years ago(!) - I think he will find today's Monday as a not bad beginning of the week ))

I've found that HA (by design) allows only one "home" zone and multiple "areas", but all 'areas' are assigned to a Home zone by default, they can't be assigned to different zones. This is a big design failure as now there are many people and their families who live atomised within one city or country and there should be several 'home zones' and with areas which can be assigned to them. This is not your issue, this is just a problem of HA 'space architecture' - I will try to raise a feature request to HA team. This is obvious that today (and with the development of ZigBee-LAN coordinators, one HA instance can control multiple sites and it's not resource consuming for a host in terms of hardware - and now this is not possible. I can track desktop computers now with the fixed integration but for mobile phones this is useless as tracking will show that they are all "home" if they are connected at least to one router from the integration. Yes, GPS tracking is still in place but zone entry/leave is not smoothly tracked by a phone app sometimes, so the additional zone tracking could be helpful but not now as there is only one "home zone"..

Many thanks again and have a good week! Appreciate your help a lot!