search

bwssytems / ha-bridge

Home automation bridge that emulates a Philips Hue light system and can control other systems such as a Vera, Harmony Hub, Nest, MiLight bulbs or any other system that has an http/https/tcp/udp interface. This is a compact impl to run on small format computers. This is impl started from this project https://github.com/armzilla/amazon-echo-ha-bridge.
Apache License 2.0
1.45k stars 198 forks source link

Alexa Discovery Issues - Need to understand discovery process a bit more #1161

Open craigski opened 4 years ago

craigski commented 4 years ago

HABridge been fine for almost a year, since fixing the port 80 issue:

https://github.com/bwssytems/ha-bridge/issues/1022#issuecomment-447318634

I recently tried to add a new device, but Alexa wont discover. Deleted all devices, upgraded to 5.3.0, renumber, still no discovery.

I can see the SSDP broadcasts from the HAbridge on the network, but Alexa Gen2, Gen3 or Alexa iOS APP will not discover the HAbridge. There is no request from the Alexa devices to the HAbridge. I am wondering if Amazon has changed how they are now discovering the Hue?

I have been reading this:

https://developers.meethue.com/develop/application-design-guidance/hue-bridge-discovery/

"All our bridges support UPnP, but it is advised to move to using mDNS and N-UPnP as that will be faster, mDNS is supported by more devices, and we only use a subset of the UPnP protocol (SSDP). "

bwssytems commented 4 years ago

So, mDNS is in the ha-bridge, but I believe Alexa Echo does not support that.

Please turn on traceupnp in the Bridge Control Settings and then do a discover after that. Please post the log. Include the IPs of the alexa and the ha-bridge.

craigski commented 4 years ago

Did you see the (2) in the discovery guide above, where the url https://discovery.meethue.com is used to get bridge info? Also all the examples are using https. No idea which of these Amazon is using, but possibly (2) and not getting bridge ID?

Here is my log, I'm afraid there is nothing useful, there is no local request from any Amazon device. I have tried gen 2, gen 3 and iOS App for discovery. This log is starting discovery from a gen2

Bridge: 10.134.10.124 Alexa (gen2): 10.134.10.44

12-10-2019 21:19:38.309 INFO HA Bridge startup sequence... com.bwssystems.HABridge.HABridge
12-10-2019 21:19:38.988 INFO HA Bridge (v5.3.0) initializing.... com.bwssystems.HABridge.HABridge
12-10-2019 21:19:38.999 INFO reading from config file: /home/pi/ha-bridge/data/habridge.config com.bwssystems.HABridge.BridgeSettings
12-10-2019 21:19:41.785 INFO StaticResourceHandler configured with folder = /public spark.staticfiles.StaticFilesConfiguration
12-10-2019 21:19:41.866 INFO System control service started.... com.bwssystems.HABridge.SystemControl
12-10-2019 21:19:42.152 INFO Initializing UDP response Socket... com.bwssystems.HABridge.util.UDPDatagramSender
12-10-2019 21:19:42.186 INFO UDP response Socket initialized to: 50000 com.bwssystems.HABridge.util.UDPDatagramSender
12-10-2019 21:19:42.210 INFO HTTP Home created. com.bwssystems.HABridge.plugins.http.HTTPHome
12-10-2019 21:19:42.243 INFO Harmony Home created. No Harmony devices configured. com.bwssystems.HABridge.plugins.harmony.HarmonyHome
12-10-2019 21:19:42.255 INFO Nest Home created. No Nest configured. com.bwssystems.HABridge.plugins.NestBridge.NestHome
12-10-2019 21:19:42.263 INFO Hue passthru Home created. No Hue passtrhu systems configured. com.bwssystems.HABridge.plugins.hue.HueHome
12-10-2019 21:19:42.271 INFO HAL Home created. No HAL devices configured. com.bwssystems.HABridge.plugins.hal.HalHome
12-10-2019 21:19:42.280 INFO MQTT Home created. No MQTT Clients configured. com.bwssystems.HABridge.plugins.mqtt.MQTTHome
12-10-2019 21:19:42.288 INFO HomeAssistant Home created. No HomeAssistants configured. com.bwssystems.HABridge.plugins.hass.HassHome
12-10-2019 21:19:42.294 INFO HomeWizard Home created. No HomeWizard gateways configured. com.bwssystems.HABridge.plugins.homewizard.HomeWizardHome
12-10-2019 21:19:42.300 INFO Command Home for system program execution created. com.bwssystems.HABridge.plugins.exec.CommandHome
12-10-2019 21:19:42.311 INFO TCP Home created. com.bwssystems.HABridge.plugins.tcp.TCPHome
12-10-2019 21:19:42.322 INFO UDP Home created. com.bwssystems.HABridge.plugins.udp.UDPHome
12-10-2019 21:19:42.327 INFO Vera Home created. No Veras configured. com.bwssystems.HABridge.plugins.vera.VeraHome
12-10-2019 21:19:42.333 INFO Fibaro Home created. No Fibaros configured. com.bwssystems.HABridge.plugins.fibaro.FibaroHome
12-10-2019 21:19:42.341 INFO Domoticz Home created. com.bwssystems.HABridge.plugins.domoticz.DomoticzHome
12-10-2019 21:19:42.352 INFO Somfy Home created. No Somfys configured. com.bwssystems.HABridge.plugins.somfy.SomfyHome
12-10-2019 21:19:42.367 INFO LifxDevice Home created. No LifxDevices configured. com.bwssystems.HABridge.plugins.lifx.LifxHome
12-10-2019 21:19:42.376 INFO OpenHAB Home created. No OpenHABs configured. com.bwssystems.HABridge.plugins.openhab.OpenHABHome
12-10-2019 21:19:42.386 INFO FHEM Home created. No FHEMs configured. com.bwssystems.HABridge.plugins.fhem.FHEMHome
12-10-2019 21:19:42.406 INFO Broadlink Home created. No Broadlinks configured. com.bwssystems.HABridge.plugins.broadlink.BroadlinkHome
12-10-2019 21:19:42.420 INFO Mozilla IOT Home created. No Mozilla IOTs configured. com.bwssystems.HABridge.plugins.moziot.MozIotHome
12-10-2019 21:19:42.428 INFO HomeGenie Home created. No HomeGenies configured. com.bwssystems.HABridge.plugins.homegenie.HomeGenieHome
12-10-2019 21:19:42.530 INFO == Spark has ignited ... spark.embeddedserver.jetty.EmbeddedJettyServer
12-10-2019 21:19:42.531 INFO >> Listening on 0.0.0.0:80 spark.embeddedserver.jetty.EmbeddedJettyServer
12-10-2019 21:19:42.562 INFO HABridge device management service started.... com.bwssystems.HABridge.devicemanagmeent.DeviceResource
12-10-2019 21:19:42.729 INFO Hue emulator service started.... com.bwssystems.HABridge.hue.HueMulator
12-10-2019 21:19:43.002 INFO Traceupnp: upnp config address: 10.134.10.124 -useIface: false on web server: 0.0.0.0:80 com.bwssystems.HABridge.HABridge
12-10-2019 21:19:43.008 INFO Description xml service started.... com.bwssystems.HABridge.upnp.UpnpSettingsResource
12-10-2019 21:19:43.036 INFO UPNP Discovery Listener starting.... com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:19:43.038 INFO Traceupnp: Interface: eth0 valid usable IP address: /10.134.10.124 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:19:43.040 INFO Traceupnp: Adding eth0 to our upnp join interface set. com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:19:43.041 INFO Traceupnp: Interface: lo valid usable IP address: /127.0.0.1 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:19:43.042 INFO Traceupnp: Adding lo to our upnp join interface set. com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:19:43.042 INFO Create and run mDNS service. com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:19:43.217 INFO UPNP Discovery Listener running and ready.... com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:20:56.851 INFO Link button pressed.... com.bwssystems.HABridge.SystemControl
12-10-2019 21:20:59.160 INFO Traceupnp: sendUpnpNotify notifyTemplate1 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:20:59.414 INFO Traceupnp: sendUpnpNotify notifyTemplate2 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:20:59.668 INFO Traceupnp: sendUpnpNotify notifyTemplate3 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:21:19.932 INFO Traceupnp: sendUpnpNotify notifyTemplate1 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:21:20.185 INFO Traceupnp: sendUpnpNotify notifyTemplate2 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:21:20.438 INFO Traceupnp: sendUpnpNotify notifyTemplate3 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:21:26.855 INFO Link button time ended.... com.bwssystems.HABridge.LinkButtonPressed
12-10-2019 21:21:50.192 INFO Traceupnp: sendUpnpNotify notifyTemplate1 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:21:50.446 INFO Traceupnp: sendUpnpNotify notifyTemplate2 com.bwssystems.HABridge.upnp.UpnpListener
12-10-2019 21:21:50.699 INFO Traceupnp: sendUpnpNotify notifyTemplate3 com.bwssystems.HABridge.upnp.UpnpListener
bwssytems commented 4 years ago

So, I assume you did an Alexa discovery through this log. Since there is no log of receiving any UPNP SSDP Search messages from the Alexa or any devices (which is wierd) there may be an issue with your network settings. Did those change recently? Since you are looking at network packets, can you see the SSDP messages from the Alexa?

P.S. The search for https://discovery.meethue.com is the N-UPNP request which is proprietary to the Philips Hue Hyb and the ha-bridge is not a Philips product.

craigski commented 4 years ago

OK, so looking at above log, I can see no traffic from Echo to bridge. Digging a little deeper, I confirmed this with tcpdump in the Pi runng habridge on the LAN

sudo tcpdump -v host [ip address of Echo]

I have done some network traces and the Alexa is doing a SSDP multicast on the WLAN (as per developer doc above option (1)), but this is not arriving on LAN. It seems SSDP using address 239.255.255.250 is not traversing LAN <--> WLAN for some reason. I'm still looking into what is blocking the SSDP.

I will get a USB wifi Pi adapter and attach to WLAN to confirm discovery of habridge is still working.

bwssytems commented 4 years ago

Did you have any updates to your router firmware? Were there any updates to the firewall on your Pi?

witt312 commented 4 years ago

One thing I’ve noticed while tracing upnp traffic during discovery is that on 5.3, the bridge responds with 0.0.0.0:80 as the address. When rolling back to 5.1 the configured address is used in response. Maybe this is just a logging issue, but 5.1 consistently allows device discovery while 5.3 never does. Also, 5.3 never logs receiving any Alexa discovery requests.

EnGamma commented 4 years ago

I too just found out that Alexa is no longer discovering new devices. Existing devices are showing as offline, but they are still working.

I've not tested new device discovery in 5 months, until now. Log attached. ha-bridge.20191212.txt

witt312 commented 4 years ago

I was reading through some of the Home Assistant forum and they’ve had the exact same issue. I skimmed through the thread but it sounds like they implemented a solution recently for the discovery and the “doesn’t support requested value” issue. Thanks Amazon.

craigski commented 4 years ago

Maybe this is just a logging issue, but 5.1 consistently allows device discovery while 5.3 never does. Also, 5.3 never logs receiving any Alexa discovery requests.

I 'discovered' same :) however in my case the reason 5.3 didn't log any requests was because it didn't receive any requests.

What I have learnt so far:

Initiating a discovery from Alexa will cause all the Alexa devices to send a SSDP multicast using destination ip address 239.255.255.250. Using tcpdump on my Pi connected to Ethernet, I did not see any of these requests from the Alexas connected to wifi. For some reason, that I don't fully understand (yet), these multicasts were not traversing from my wifi network to lan. I understand this is due to something called IGMP snooping:

https://en.wikipedia.org/wiki/Internet_Group_Management_Protocol

Some network devices (switches, routers, accesspoints?) implement this to create a multicast group to restrict multicast traffic to only those devices on the network that are interested in it. If this network is not working correctly, the SSDP multicasts will not be received on the habridge.

There is a very easy way to test this if your habridge is running on a Pi on lan, use tcpdump with the multicast address from your Pi

sudo tcpdump -v host 239.255.255.250

You should see 3 SSDP requests from each Alexa on your network.

If you dont see these, there is something wrong with your network, and you will need to figure out what, before looking at habridge logs.

Restarting Pi, switches, access points, routers, Alexa's may help. Another solution is a wifi usb adapter for your Pi, so the Pi/habridge and Alexa(s) are all on wifi.

If you see these SSDP requests in tcpdump, then you can dive a little deeper in habridge logs.

HTH

craigski commented 4 years ago

I too just found out that Alexa is no longer discovering new devices. Existing devices are showing as offline, but they are still working.

I've not tested new device discovery in 5 months, until now. Log attached. ha-bridge.20191212.txt

Just had a quick scan through your log, as you can see, you are getting the 3 SSDP requests from each of your Alexa devices, some IPs not listed in your summary at top? I would suggest disconnecting all bar one, and then initiating a discovery.

Huibuh1111 commented 4 years ago

Just a hint: you could try to have a look on the code from release 4.x.x and what has changed to the current 5.x.x release from habridge. I don't know excatly the difference, but i had the same issues that Alexa didn't find any more my devices.

I tried all the suggested solutions: changed the Port from 81 to 80, renumbered the devices and tried to discover the devices with and without the link-button several times - no luck.

The (simple) solution was: i made a downgrade from the current 5.3.0-release to the latest 4.5.6-release (habridge running on port 80), rebooted the pi, started again the discovery by Alexa and voilà: all devices discovered and manageable by Alexa without problems.

So, at the moment i'll keep the (old) 4.5.6-release running... :-)

fakenius commented 4 years ago

@Huibuh1111: you are completely right. For month I also did everything what had been discussed and suggested here without any success. Now after downgrading to 4.5.6 it works like a charm again. Thanks a lot :-)

soxfan1966 commented 4 years ago

Got a new Echo for Christmas and added to my network (I previously just had 3 Echo Dots). When I went to try my lights on the new Echo they did not work. I checked the setup in the Alexa App and all my devices from HA Bridge are missing. I went to the HA Bridge on my Pi and removed all the ones I had and then added them again and tried to do the discovery. But Alexa is not finding the devices now. I had an issue before Christmas when I tried to add a single new x10 device - it was added OK to HA Bridge but Alexa did not find it. Then today I noticed all of them were missing.

The devices are in HA Bridge and they work (I am using HA Bridge with HomeGenie on a Pi X10 Hub). I can turn them on via HA Bridge - just cannot get Alexa to discover the devices again.

Any suggestions ?

Mitchelll010 commented 4 years ago

Same here, has to be a problem in habridge.

The only way to recognise new devices is to remove all devices from alexa manually, renumber the device id's in habridge en discover again. This is a pain in the ass because you will lose all the routines you made for alexa..

Has to be in habridge, i think i will be going to do the same thing as MasterRay2.. downgrade to 4.5.6.

Unless there will be a fix soon offcourse.

bwssytems commented 4 years ago

For all: The new Release Candidate is out for testing.... https://github.com/bwssytems/ha-bridge/releases/tag/v5.3.1RC1

Please post your issues or successes at #1192

dougdalton commented 3 years ago

I have updated my previously working config to 5.3.1 and it still cannot discover devices, I have noticed this in all 5.x updates. I have 2 fire tvs v2 and 1 echo dot original and none of their discoveries work