search

hoobs-org / HOOBS

Build your Smart Home with HOOBS. Connect over 2,000 Accessories to your favorite Ecosystem.
https://hoobs.org
GNU General Public License v3.0
551 stars 51 forks source link

Accessories work locally, "not responding" remotely, Home Hub is not talking to HOOBS, maybe mDNS issue #759

Closed jlg89 closed 3 years ago

jlg89 commented 4 years ago

Description The HomeKit Home Hubs (Apple TV 4K primary, iPad secondary) do not appear to be able to talk to the HOOBS service, and it has something to do with the mDNS records coming from HOOBS.

Setup: HOOBS server is assigned a static IP via DHCP. Connected via Ethernet, WiFi is not enabled. Home Hub Apple TV is on Ethernet, Home Hub iPad is on WiFi. Everything is in the same broadcast domain, same subnet, no odd routing at all. HomeKit accessories include ~10 "native" accessories, and ~15 bridged via HOOBS, including several cameras.

All accessories work perfectly in the Home app, on macOS and iOS, when on the local network. When using the Home app remotely (via mobile or foreign WiFi), HOOBS accessories are usually (but not always) "no response" in the Home app. In these instances, the HOOBS log does not show that the Home Hub is even attempting to connect to the bridged accessories. HomeKit native accessories work reliably, even remotely.

I duplicated the HOOBS mDNS service record by creating a custom avahi service file on the HOOBS server. Once I did this, remote access to the HOOBS accessories began working reliably.

So, it would appear that there is something about the mDNS record being generated by HOOBS itself that is not passing the correct information to the Home Hub.

Is this a config issue, local DNS problem, or...?

Version HOOBS 3.2.6 Node Version 12.16.2 RPi BCM2835 - Pi 3 Model B+ "buster" kernel 4.19.118-v7+

Did you upgrade Please let us know if you upgraded from a previous version.

List your plugins Please include a list of the plugins you are using. We need to be able to find it on NPM. If the plugin is no published, please include a link to the repository.

homebridge-camera-ffmpeg homebridge-dafang-mqtt-republish homebridge-harmony homebridge-infinitude homebridge-myq2 homebridge-neato homebridge-smtp-motion homebridge-tplink-smarthome

Post your config

{
    "server": {
        "port": 80,
        "origin": "*",
        "autostart": 0,
        "home_setup_id": "X-HM://0000000000000",
        "polling_seconds": 5
    },
    "client": {
        "default_route": "status",
        "inactive_logoff": 30,
        "theme": "hoobs-light",
        "locale": "en",
        "temp_units": "fahrenheit",
        "country_code": "US",
        "postal_code": "00000",
        "latitude": "00.0000000",
        "longitude": "-00.0000000"
    },
    "bridge": {
        "name": "HOOBS",
        "port": 51826,
        "pin": "000-00-000",
        "username": "01:23:45:67:89:AB"
    },
    "description": "Homebridge server",
    "ports": {},
    "accessories": [],
    "platforms": [
        {
            "platform": "Camera-ffmpeg",
            "plugin_map": {
                "plugin_name": "homebridge-camera-ffmpeg"
            },
            "cameras": [
                {
                    "name": "Camera 1",
                    "manufacturer": "Wyze",
                    "model": "Wyze Cam Pan",
                    "videoConfig": {
                        "source": "-i rtsp://192.168.77.5/live",
                        "stillImageSource": "-i rtsp://192.168.77.5/live -vframes 1 -r 1",
                        "maxStreams": 2,
                        "maxWidth": 960,
                        "maxHeight": 540,
                        "maxFPS": 15,
                        "audio": true,
                        "debug": false
                    }
                },
                {
                    "name": "Camera 2",
                    "manufacturer": "Wyze",
                    "model": "Wyze Cam V2 + Dafang",
                    "motion": true,
                    "motionTimeout": 5,
                    "videoConfig": {
                        "source": "-i rtsp://192.168.77.13:8554/unicast",
                        "stillImageSource": "-i https://192.168.77.13/cgi-bin/currentpic.cgi",
                        "maxStreams": 2,
                        "maxWidth": 768,
                        "maxHeight": 432,
                        "maxFPS": 15,
                        "audio": true,
                        "debug": false
                    }
                },
                {
                    "name": "Camera 3",
                    "manufacturer": "August Home",
                    "model": "Doorbell Cam Pro",
                    "serialNumber": "0000000000",
                    "videoConfig": {
                        "source": "-i rtsp://192.168.77.7/live/stream",
                        "stillImageSource": "-i rtsp://192.168.77.7/live/stream -vframes 1 -r 1",
                        "maxStreams": 2,
                        "maxWidth": 480,
                        "maxHeight": 640,
                        "maxFPS": 15,
                        "audio": true,
                        "debug": false
                    }
                }
            ],
            "name": "Camera FFMPEG",
            "mqtt": "127.0.0.1",
            "portmqtt": 1883,
            "usermqtt": "mqtt-user",
            "passmqtt": "mqtt-password",
            "topic": "hoobs-alerts",
            "porthttp": 8089,
            "interfaceName": "eth0"
        },
        {
            "platform": "InfinitudePlatform",
            "plugin_map": {
                "plugin_name": "homebridge-infinitude"
            },
            "url": "http://127.0.0.1:3000"
        },
        {
            "platform": "HarmonyHubWebSocket",
            "plugin_map": {
                "plugin_name": "homebridge-harmony"
            },
            "name": "Harmony Hub",
            "hubIP": "192.168.77.9",
            "hubName": "Harmony Hub",
            "publishAllTVAsExternalAccessory": true,
            "TVAccessory": false,
            "mainActivity": "Watch Apple TV",
            "switchAccessories": true,
            "publishGeneralMuteSwitch": true,
            "publishGeneralVolumeSlider": true,
            "linkVolumeControlToTV": true,
            "publishSwitchActivitiesAsIndividualAccessories": true,
            "publishDevicesAsIndividualAccessories": true,
            "publishSequencesAsIndividualAccessories": true,
            "publishHomeControlsAsIndividualAccessories": true,
            "showCommandsAtStartup": false,
            "addAllActivitiesToSkippedIfSameStateActivitiesList": false,
            "playPauseBehavior": false,
            "cleanCache": false
        },
        {
            "platform": "NeatoVacuumRobot",
            "plugin_map": {
                "plugin_name": "homebridge-neato"
            },
            "email": "email",
            "password": "password"
        },
        {
            "platform": "TplinkSmarthome",
            "plugin_map": {
                "plugin_name": "homebridge-tplink-smarthome"
            }
        },
        {
            "platform": "dafangMqtt",
            "plugin_map": {
                "plugin_name": "homebridge-dafang-mqtt-republish"
            },
            "name": "Dafang MQTT",
            "cameras": [
                {
                    "name": "Camera 2",
                    "dafang_topic": "hoobs/cam2"
                }
            ],
            "server": "127.0.0.1",
            "port": 1883,
            "homebridge_topic": "hoobs-alerts"
        },
        {
            "platform": "smtpMotion",
            "plugin_map": {
                "plugin_name": "homebridge-smtp-motion"
            },
            "name": "SMTP Motion",
            "smtp_port": 2525,
            "http_port": 8089
        },
        {
            "platform": "myQ",
            "plugin_map": {
                "plugin_name": "homebridge-myq2"
            },
            "email": "email",
            "password": "password"
        }
    ]
}

Contents of /etc/avahi/services/hoobs.service:

<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
  <name>HOOBS 1293</name>
  <service>
    <type>_hap._tcp</type>
    <port>51826</port>
    <host-name>hoobs.local</host-name>
    <txt-record>md=HOOBS Server</txt-record>
    <txt-record>pv=1.1</txt-record>
    <txt-record>id=01:23:45:67:89:AB</txt-record>
    <txt-record>c#=110</txt-record>
    <txt-record>s#=1</txt-record>
    <txt-record>ff=0</txt-record>
    <txt-record>sf=0</txt-record>
    <txt-record>ci=2</txt-record>
    <txt-record>sh=XXXXXX==</txt-record>
  </service>
</service-group>
BobbySlope commented 4 years ago

how did u setup your second mdns service, please provide the content of you own avahi service

jlg89 commented 4 years ago

No separate avahi service, it's the one that already runs on the HOOBS server. I just added the /etc/avahi/services/hoobs.service file, and the avahi service picked it up automatically.

I know that HOOBS is already advertising itself, because this shows up in syslog:

Aug 15 09:26:07 hoobs avahi-daemon[310]: Loading service file /services/hoobs.service. Aug 15 09:26:07 hoobs avahi-daemon[310]: Service name conflict for "HOOBS 1293" (/services/hoobs.service), retrying with "HOOBS 1293 #2". Aug 15 09:26:09 hoobs avahi-daemon[310]: Service "HOOBS 1293 #2" (/services/hoobs.service) successfully established.

What I'm trying to figure out is, why does the Home Hub work when this secondary service is loaded, but not with the "standard" HOOBS service (on the same box!)? Seems to me they both point to the same place, so both should work the same, but they obviously don't.

jlg89 commented 4 years ago

The only difference I can see between what HOOBS is publishing and my avahi service file is that HOOBS is broadcasting its hostname as "01_23_45_67_89_AB.local" whereas my avahi service broadcasts the hostname as "hoobs.local" which matches local DNS. Both hostnames ping successfully from my macOS workstation, but there's no way for me to verify what the Apple TV (Home Hub) is doing, except by watching Wireshark mDNS traffic.

All I do know is that, without the additional avahi service file in place, the Home Hub isn't talking to the HOOBS bridge at least 90% of the time--doesn't even attempt to pull any data--so remote Home clients get no response from any HOOBS accessories. Every once in awhile it works fine, though, which is why this is such a frustrating problem. I've been through all the standard troubleshooting steps, reboot everything, enable/disable Apple TV and an iPad as Home Hubs, delete & re-add the HOOBS bridge, delete the home from HomeKit altogether and recreate it from scratch, etc. but nothing has helped. Best as I can tell, this is some sort of mDNS weirdness going on between HOOBS and the Home Hub, and adding that avahi service file fixes it somehow.

jlg89 commented 4 years ago

I thought maybe it could be related to this: https://github.com/homebridge/homebridge/issues/2089

But "avahi-browse --verbose _hap._tcp" shows both IPv4 and IPv6 entries for the HOOBS service (two for each protocol, the one HOOBS is providing and the one from my avahi service).

So, the IPv4 vs IPv6 issue doesn't seem to be the problem here.

jlg89 commented 4 years ago

how did u setup your second mdns service, please provide the content of you own avahi service

Contents of /etc/avahi/avahi-daemon.conf (unchanged from stock HOOBS install):

# This file is part of avahi.
#
# avahi is free software; you can redistribute it and/or modify it
# under the terms of the GNU Lesser General Public License as
# published by the Free Software Foundation; either version 2 of the
# License, or (at your option) any later version.
#
# avahi is distributed in the hope that it will be useful, but WITHOUT
# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public
# License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with avahi; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307
# USA.

# See avahi-daemon.conf(5) for more information on this configuration
# file!

[server]
#host-name=foo
#domain-name=local
#browse-domains=0pointer.de, zeroconf.org
use-ipv4=yes
use-ipv6=yes
#allow-interfaces=eth0
#deny-interfaces=eth1
#check-response-ttl=no
#use-iff-running=no
#enable-dbus=yes
#disallow-other-stacks=no
#allow-point-to-point=no
#cache-entries-max=4096
#clients-max=4096
#objects-per-client-max=1024
#entries-per-entry-group-max=32
ratelimit-interval-usec=1000000
ratelimit-burst=1000

[wide-area]
enable-wide-area=yes

[publish]
#disable-publishing=no
#disable-user-service-publishing=no
#add-service-cookie=no
#publish-addresses=yes
publish-hinfo=no
publish-workstation=no
#publish-domain=yes
#publish-dns-servers=192.168.50.1, 192.168.50.2
#publish-resolv-conf-dns-servers=yes
#publish-aaaa-on-ipv4=yes
#publish-a-on-ipv6=no

[reflector]
#enable-reflector=no
#reflect-ipv=no

[rlimits]
#rlimit-as=
#rlimit-core=0
#rlimit-data=8388608
#rlimit-fsize=0
#rlimit-nofile=768
#rlimit-stack=8388608
#rlimit-nproc=3
askovi commented 4 years ago

Have you tried disabling ipv6? Both in /boot/cmdline.txt > ipv6.disable=1 and in avahi settings: use-ipv6=no Then Reset Connection from the Help menu and reconnect the bridge.

This is most likely a router issue, which router are you using? If you can, I would recommend disabling ipv6 in your router settings as well. Then reboot the pi.

jlg89 commented 4 years ago

The router is IPFire, which doesn't do ipv6 at all. Home app remote access works reliably with the "HOOBS duplicate" avahi service in place on the HOOBS RPi. If it were an ipv6 issue, I wouldn't think this would help. I'll give it a shot, though.

jlg89 commented 4 years ago

Yesterday, after updating my iOS devices to 13.6.1, everything disappeared from my home. All accessories, everything, from every device. It was like someone at Apple hit the reset button on it. So I took the opportunity to delete the home, ensuring that it was cleared out on all my devices, reset the HOOBS connection, disabled ipv6, removed the redundant avahi service file (which is now incorrect anyway), rebooted everything. Left everything overnight, to be sure the mystery iCloud propagation had time to do whatever it does and give mDNS caches time to timeout. This morning, I created a new home in the Home app, and tried to add HOOBS. The Home app says that it couldn't add the device. I try again, and the Home app says that the device has already been added. I manually enter the 8-digit code, but HOOBS doesn't appear in the "nearby accessories" browser. Other devices appear. Running avahi-browse on the HOOBS server shows that HOOBS is advertising itself via ipv4, and shows the other accessories.

Oh, and when I have HOOBS either "Reset Connection" or "Generate New Username," about 50% of the time it deletes the entire configuration and I have to restore everything in config.json after the server/client/bridge sections.

jlg89 commented 4 years ago

So now I'm in an endless cycle. Try to add the HOOBS server as an accessory in the Home app. It says, "Couldn't add HOOBS. Home couldn't connect to this accessory," and the only choice is to Cancel. Try again, and it says, "Accessory already added. To add this accessory to your home, you'll need to reset it first." Generate a new username in HOOBS, restart everything, rinse, repeat.

It isn't just HOOBS, a similar thing happens with my gen 4 August lock, it just can't be found. Other devices, including older August locks, are reconnecting properly.

I'm basically giving up on HomeKit until iOS 14 comes out. What a monumental waste of time.

jlg89 commented 3 years ago

Is there a way to enable a debug log that would show what's going on from HOOBS' perspective when I try to use the Home app to add HOOBS to HomeKit? I think it would be helpful if I can figure out whether or not HomeKit is actually trying to talk to HOOBS, and/or whether or not HOOBS is responding. HomeKit obviously doesn't give me any assistance in this regard.

jlg89 commented 3 years ago

I finally just wiped my HOOBS SD card, burned a fresh install image, and started over. HomeKit added it as an accessory on the first try. Now to reconfigure everything, and see if I can break it again.

For the record, things really headed south when disabled ipv6. I think that advice might have been a workaround in the past, but from what I can tell it is no longer a good idea. Once I disabled ipv6, even after I re-enabled it, nothing worked quite right.

jlg89 commented 3 years ago

After wiping out everything (wiped/reinstalled HOOBS, deleted all HomeKit configurations) and rebuilding from scratch, I ended up right back at the beginning of this thread, with the same workaround--adding a "hoobs.service" file to /etc/avahi/services makes the HOOBS accessories work remotely. I don't know what it is, but there's something to it.

I can say that the "stock" HOOBS does not seem to advertise itself via ipv6, i.e. "avahi-browse --verbose _hap._tcp" only shows an ipv4 service for HOOBS. Once I add the custom service file, there are two ipv4 and one ipv6 services listed. And things work. Even local Home app access is faster--accessory status pops up immediately, camera video syncs immediately, whereas with "stock" HOOBS it might take 10-15 seconds for things to update.

stale[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

jlg89 commented 3 years ago

I am now on HOOBS 3.3.5, just updated from 3.3.3. HOOBS still does not advertise itself in Bonjour via IPv6.

Running "avahi-browse -a" from the command line, and looking at all entries for the HOOBS server, I see these (the HOOBS box is also running Samba):

You'll notice that there's no IPv6 record for _hap._tcp. This omission somehow breaks remote HomeKit access. My educated guess is that the HomeKit hubs insist upon using IPv6 to communicate with the HOOBS bridge on the local network.

Once I create an avahi service file (as mentioned above) that duplicates the existing _hap._tcp parameters, the avahi daemon creates both IPv4 and IPv6 entries, and HomeKit remote access works.

Notice that there are now two IPv4 _hap._tcp entries, one from HOOBS and one from my duplicate service, but only one IPv6 entry, from my duplicate service.

mkellsy commented 3 years ago

On our V3 images, we don't mess with Avahi. It is setup exactly how Raspbian wanted it.

What's next is all beta. We are configuring Avahi in our V4 images, and the new UI allows users to change the broadcasted host. I will make sure the new image broadcasts IPv6 too.

jlg89 commented 3 years ago

I had to add an Avahi service to duplicate the mDNS broadcast of a device that lives at the other end of an Ethernet-over-coax bridge that doesn't pass broadcast traffic, so that's what got me thinking about whether or not HOOBS was actually broadcasting both IPv4 and IPv6 entries in the first place. I didn't have to change the main Avahi config at all (as it shipped on the HOOBS SD card), just had to use a Bonjour sniffer to give me all the details of the existing broadcast so I could duplicate it with an Avahi service file.