bwssytems / ha-bridge

Home automation bridge that emulates a Philips Hue light system and can control other systems such as a Vera, Harmony Hub, Nest, MiLight bulbs or any other system that has an http/https/tcp/udp interface. This is a compact impl to run on small format computers. This is impl started from this project https://github.com/armzilla/amazon-echo-ha-bridge.
Apache License 2.0
1.45k stars 197 forks source link

Alexa cannot discover devices which used to work. #1293

Open shawn-lindberg opened 3 years ago

shawn-lindberg commented 3 years ago

I apologize if this should have been a reply to another issue, but I have been reading quite a few threads on discovery issues and none of them seem to explain what is happening.

I had been using HA Bridge for quite a few years, most recently version 5.3, with a couple of devices on a Raspberry Pi 3 B. The day before yesterday, Alexa on my Echo Dot started reporting that the devices were not responding. I started digging into this, and confirmed that the devices still responded through the HA Bridge web interface. So, I deleted the devices from the Alexa app and tried to rediscover them. Initially, it looked like this wasn't working because because of a warning about the UPNP IP address about matching a known address on the host, but I was able to fix that by removing a Wi-Fi dongle from the Pi that I didn't need anyway, and resetting the hostname. I also upgraded the bridge to version 5.4. However, the discovery will still not work.

I believe I have tried everything in the troubleshooting page at least once. In particular, I have verified that my Netgear Nighthawk X6 router has UPNP enabled, and the bridge is using a static IP. The logs are interesting. In this case, the Pi running the bridge is IP address 192.168.1.110, the Echo is .14, and my Synology NAS is .12. You can see that the Echo does request a list of the lights, but I don't see a response. habridgelog.txt

EDIT: One other little tidbit that I forgot to add. My friend was over trying to help me with this issue yesterday and he tried to see if we could discover the devices with the Anymote app on his cell phone connected to my network. He was able to, but I wasn't able to do that later.

Any help would be most appreciated.

shawn-lindberg commented 3 years ago

I just tried the Anymote app again, and it was able to discover devices and turn them on and off. Here is the relevant portion of the log: habridgeanymotelog.txt

bwssytems commented 3 years ago

So, if the Alexa asked for the list of lights, this will be an issue of IDs cached by amazon. Did you renumber your devices with a different seed?

shawn-lindberg commented 3 years ago

Yes, I re-numbered the devices using seeds of 200, 300, 400, and 500 (originally the seed was 100). I also went into my Alexa device history and deleted the device history.

Shadow27374 commented 3 years ago

I probably have the same problem.

A few days ago I had to install HA-Bridge 5.4.0 and reallocate the IDs (100,200,etc.). After that I still had duplicate and triplicate entries. I had to remove each device individually in Alexa. Removing them all at once does not delete them from Alexa and they would be found again.

Finally, again assign new IDs and Alexa had all my devices once.

About two days later, Alexa independently searched for new devices at night. I saw that because I had seen in the morning a notification under Android the found devices. But since I was asleep, this is not possible...

In the AlexaApp were then again all possible devices double and triple.

From now on I can do what I want it will only Hue lamps that are integrated via the HA bridge found. All other devices via HA-Bridge are no longer found even with new IDs.

mjrgh commented 3 years ago

Same thing happened to me a few days ago. I didn't make any changes in my local setup at all, so my guess is that Amazon pushed an Alexa update that broke compatibility. In the morning this happened, I had a big pile of notifications from the Alexa App on my iPad saying that it had discovered many of my devices again, and as others experienced, I had duplicates of most devices in my device list in the app. The last time this happened a few months ago, the solution was to forget all the devices on the Alexa side, renumber them in ha-bridge, and go through Alexa discovery again. No luck with that approach this time. Alexa is discovering a random subset of the devices on each attempt (and it's really irritating because it keeps adding duplicates each time it does), but none of them work. In some cases, Alexa just won't find them, and in other cases, it'll find them, but it can't reach them with commands: on commands, it'll spin for about 30 seconds and finally time out with "X is not responding, please check its network connection and power supply". Nothing I've tried so far makes any difference.

Shadow27374 commented 3 years ago

Alexa is discovering a random subset of the devices on each attempt (and it's really irritating because it keeps adding duplicates each time it does), but none of them work.

Unfortunately, you have to delete the devices in Alexa individually. The button to delete all does not work properly, then they come back again and again.

And yes, something must have happened at Amazon last night. Now all my scripts don't work anymore. -.-

Shadow27374 commented 3 years ago

One more small note. If I include real Hue lamps from Philips via the HA-Bridge and do not use the Hue Skill, they are still found by Alexa.

The current problem seems to affect only simulated Hues.

shawn-lindberg commented 3 years ago

Interesting that other people have experienced this problem or something like it as well. It does make me wonder if it is related to an update Amazon pushed out. But all of my other real physical devices to work, only to fake devices seem to be having problems.

mjrgh commented 3 years ago

It does make me wonder if it is related to an update Amazon pushed out.

There's nothing to wonder about - that's all but certain, based on my setup. Everything else I run is local and manually configured, and I haven't changed anything in a long time. (In fact, the last time I changed anything was the last time this happened, which was also caused by an Alexa update that Amazon pushed out.) The only thing in my system that can be screwed up by remote control is Alexa. The problem spontaneously appeared one morning, coinciding with a bunch of Alexa App notifications about newly discovering all of my devices. The most plausible explanation by far is an Alexa update.

shawn-lindberg commented 3 years ago

Yeah, I am agreeing with you. The fact that real physical devices are still working maybe exposes a weakness in the emulation, though?

beandi commented 3 years ago

I faced the same problem yesterday - after some searching i found another thread, where a solution was described: The last digits of the unique-IDs have to be the same, so if your device has an unique id of 00:27:18:5E:E3:0D-7E, you have to change it to 00:27:18:5E:E3:0D-0D. This change has to be done directly in the device.db-file. After that, every device was found without any problem.

Shadow27374 commented 3 years ago

I faced the same problem yesterday - after some searching i found another thread, where a solution was described: The last digits of the unique-IDs have to be the same, so if your device has an unique id of 00:27:18:5E:E3:0D-7E, you have to change it to 00:27:18:5E:E3:0D-0D. This change has to be done directly in the device.db-file. After that, every device was found without any problem.

I just tried that now and it does not work.

All My devices now have these 2 values identical after the MAC address.

beandi commented 3 years ago

I just tried that now and it does not work.

Strange - I also removed the "copy"-text from mapid AND enabled the link-button, but the one change which definitely led to success was the edit of the unique IDs. I stopped the service during the edit, have you re-initialized the bridge?

Shadow27374 commented 3 years ago

Yes, I have reinitialized. I turned off the HA bridge with systemctl, edited the devices.db via nano and restarted the HA bridge.

I still have a defective Fibaro switch where the relay is defective and only the ZWave part works. Thus, I can find and address my defective switch from Alexa via the official Fibaro Skill.

The switch only triggers a LUA script which sends an HTTP request to switch my fake device (air conditioner).

This is total overkill (50€ each) but at least I can now switch myA/C on and off. :D

Epik77 commented 3 years ago

I can confirm that the same problem happened to me last week. Echo dot3 scanned the devices notifying me of new devices creating duplicates. I removed everything and since then it hasn't detected any devices on my habridge anymore. Surely there must have been a firmware update on the device.

kristofvanwoensel commented 3 years ago

Same problem here.... I removed all cache from my echo device, reinitiated the echo, reinstalled ha-bridge, renumbered and renamed my devices...nothing seems to make Alexa discover my devices.

castellorafa commented 3 years ago

Same here, one 'curious' thing is that along with the missed hue , Alexa has detected my HP printer as a new device, the printer has been there for years and Alexa never said nothing about it (also tried deleting on alexa devices, renumbering on ha-bridge and other things that did work on my last discovery issue, but not working this time...)

Shadow27374 commented 3 years ago

I hope bwssystems finds out what Amazon has done. Would be a pity for the actually good software and especially my devices.

At the moment I use my fingers like stone age people.

Quiphainyr commented 3 years ago

Same issue here. I guess it's due to an update of the Echo devices. I tried several solutions like deleting everything and adding them one at a time, downgrade ha-bridge to 3.x, 4.x and 5.x, change the unique-ID's, renumber devices with different seeds, but none of the solutions worked for me. Eventually I ended up with using Controlicz as a workaround. I really would like to have ha-bridge working again, but I guess for now it's waiting on a solution that works.

dbrucep commented 3 years ago

For me as well. Started last weekend when notifications on my phone says it discovered new devices where were old instances. Worked to remove and disconnected all echo's. Started with just one but it won't discover devices anymore even by renumber or editing the device ID's. Trace shows the handshake between my Echo and the RPi but it the Echo doesn't seem to do anything with the responce. Like someone said, it did now find my HP Printer (no big deal to me). Could be there deploying/updating software for the new Echo Gen 4 that includes a HUB. Hopefully not the end of the line for us. I suspect that there is $$ motivating this maybe since those vendors that provide Skill's might pay a fee and Phillips took some action.

Hope I'm wrong and this is just a short lived breakage.

mjrgh commented 3 years ago

I suspect that there is $$ motivating this maybe since those vendors that provide Skill's might pay a fee and Phillips took some action.

That seems unlikely given that everyone so far has reported that the real Hue devices are still working with Alexa. If Amazon were intentionally trying to disable Hue devices to extract a payment from Philips, it would have been pretty incompetent to leave the real Philips devices working and only break an emulator that they've probably never heard of and probably don't test against. My guess is that it's just that they don't test their changes against HA-Bridge, and they made some change that works with the original Philips hubs but exposes some new compatibility defect in the emulator.

SelmaUrban commented 3 years ago

Hi, can someone check this out:

00:17:88:XX:XX:XX-XX = this is what my uids from ha-bridge look like 00:17:88:XX:XX:XX:XX:XX-XX = this is the hue id style you see hue ids are longer.

so i stop the bridge and extend some uid in the device.id and after restart my echo found the bridge-device

by the way, the last digits are not the same in my test!

coincidence or can you do that too?

Shadow27374 commented 3 years ago

It just worked for me on one device! I'm just deleting all skill devices again and test it now again completely via the HA-Bridge.

Let's see.

the4jaggz2 commented 3 years ago

Just to add some further confusion/insights, my experience has been as exactly described by mjrgh et al with one exception. I moved my bridge from a RPI running HA Bridge 5.1 (connected to my Vera controllers) to HA Bridge 5.4 on Synology (running a virtual LAN to implement port 80 versus 8080 which Alexa can't find). I needed to move to 5.4 to incorporate Home Assistant which 5.1 does not support. Alexa discovered all the applicable devices but then started doubling up (on my Vera ones only, not my HA devices) and of course, the rest of the story has already been documented in the above posts. In desperation though, I shut down the service on my Synology and turned the PI back on, ran the discovery and everything came back with no issues and has been running very well for the last 3 days. Initially, I was going to suggest it was 5.4 related or UID or seed number etc, but it seems everyone has test those solutions and nothing has worked. So net, net, I have no idea why my original service work (has the shorter uids) but hopefully SelmaUrban is onto something.

Shadow27374 commented 3 years ago

coincidence or can you do that too?

Thank you it works with all my devices!

mjrgh commented 3 years ago

Hi, can someone check this out:

00:17:88:XX:XX:XX-XX = this is what my uids from ha-bridge look like 00:17:88:XX:XX:XX:XX:XX-XX = this is the hue id style you see hue ids are longer.

so i stop the bridge and extend some uid in the device.id and after restart my echo found the bridge-device

by the way, the last digits are not the same in my test!

I'll give that a try and see what happens. Interestingly, check out this old post on issue #1243 from bwssystems, Dec 1 2020, https://github.com/bwssytems/ha-bridge/issues/1243#issuecomment-736940051:

RC3 is now out in the releases section. Updated upnp handling and device unique ids to be more specific. Also it seems the unique ids have been too long since inception and that may be the root cause. Delete all your devices in alexa and use renumber before you discover.

So it was too LONG, and now it's too SHORT... Makes me wonder if Amazon is making us run around in circles here by repeatedly changing the UID format Alexa recognizes.

SelmaUrban commented 3 years ago

I'm runing a Conbee II stick on a rpi and all ids (Ikea,...) are like the hue ids. All devices found by my echos.

kristofvanwoensel commented 3 years ago

It's not working here. I changed "f8:99:13:9d:f5:e1:05-93" to "f8:99:13:9d:f5:e1:a2:e4:05-93"

Is there anything else I need to change?

SelmaUrban commented 3 years ago

Klick the 'Link' button in ha bridge! Without i get also no disvover.

i had some old bridge devices with :X-X after changing to :XX-XX is works too

Shadow27374 commented 3 years ago

"f8:99:13:9d:f5:e1:a2:e4:05-93" One too long.

I have added only one pair 00 at the beginning 00:XX:XX:XX.....XX-XX.

SelmaUrban commented 3 years ago

yes, it's too long use a editor like notepad with fixed with font!

f8:99:13:9d:f5:e1:a2:e4:05-93
00:17:88:XX:XX:XX:XX:XX-XX
Epik77 commented 3 years ago

I confirm the correct functioning on dot3 I add 2 digits to the "uniqueid" value i also left the old id We hope they fix this problem in the next version, it would be a shame to edit the device.db every time

Quiphainyr commented 3 years ago

I just tested this workaround and it indeed seems to work! Only adding "00:" to the beginning of every unique ID seems to be enough and the Link-button has to be pressed while discovering.

Full list of steps the way it works for me:

Thanks @SelmaUrban for this workaround!

Shadow27374 commented 3 years ago

For me it worked without the link button, just as a hint.

Quiphainyr commented 3 years ago

I tried with and without the Link-button, but without pressing the Link-button didn't work for me.

castellorafa commented 3 years ago

Working too on a Echo Plus 2nd gen (with link button) , thanks @SelmaUrban !!

Shadow27374 commented 3 years ago

Interesting that some only work with. But that is no longer a real hurdle.

kristofvanwoensel commented 3 years ago

yes, it's too long use a editor like notepad with fixed with font!

f8:99:13:9d:f5:e1:a2:e4:05-93
00:17:88:XX:XX:XX:XX:XX-XX

Since it's working for most of you, there must be some other issue with my Echo. I cleared my Smart Device cache, added 00 to the number HA-Bridge generates in device.db, restarted the service, pushed the link button but nothing appears in my Alexa app.

Quiphainyr commented 3 years ago

I just removed all devices from my Alexa app with the "Remove all" button. Then I disconnected/stopped my bridges (HA-bridge and Controlicz). Did a discovery with Alexa and some old devices showed up. I removed those devices with the "Remove" option one by one (not with the "Remove all" button). Then I did another discovery and no devices showed up like it supposed to. After that I restarted my HA-bridge and now Alexa discovers my devices without pressing the Link-button.

Perhaps the above helps for you as well @kristofvanwoensel

It works for me on both Echo Gen2 and Gen3 and I'm working with HA-bridge 5.4.0.

dbrucep commented 3 years ago

Well, I edited the device file unique ID's in the following form: 00:9c:c1:38:f8:dc:04:cb-cb

Restarted Bridge with new configuation running 5.4.0.

With a single Echo 2 Dot (US FW 663647720) and no HA-Bridge devices to forget. Started Discorvery on Both Web and Asking Dot and Clicking On Link button.....All combinations. Still not finding devices.

FYI - I had total operation success with the Bridge for several years and have 66 devices. It broke last Friday when Amazon pushed an update maybe? I been following the various attemps by everyone to no avail. I was last using Bridge version 5.2.0 sucessfully before I move to 5.4.0 which had no apparent affect.

jayjupdhig commented 3 years ago

I tried it with Quiphainyr's hint ("I just tested this workaround ... "), but it does not work so it seems: https://github.com/bwssytems/ha-bridge/issues/1295

Question: Does somebody have an Echo Dot Gen 2?

If yes, what's the firmware version you have? When i remember right, there are DIFFERENT hardware version (or revisions?), and the most current update has not the same version number of the "same" device. ("same" with quotes mean -> outside the same, not inside ;-))

jayjupdhig commented 3 years ago

I also disabled the non-default parameter "UPNP Original (simple version) = true/activated" - but it still does not work.

The link button does not make any difference!

Shadow27374 commented 3 years ago

Question: Does somebody have an Echo Dot Gen 2?

I have here first version as well as the 2nd and the 3rd slightly thicker.

Quiphainyr commented 3 years ago

@jayjupdhig and @dbrucep

Do you guys see any requests from the Echo's in the syslog while discovering and "Trace UPNP calls" enabled under "Bridge Control"?

Example: [main] INFO com.bwssystems.HABridge.upnp.UpnpListener - Traceupnp: SSDP M-SEARCH packet from x.x.x.x:50000 [main] INFO com.bwssystems.HABridge.upnp.UpnpListener - Traceupnp: send upnp discovery template 1 with response address: x.x.x.x:80 to address: x.x.x.x:50000 [qtp8628262-25] INFO com.bwssystems.HABridge.hue.HueMulator - Traceupnp: hue lights list requested by user: 1f59............ from address: x.x.x.x [qtp8628262-16] INFO com.bwssystems.HABridge.hue.HueMulator - Traceupnp: hue api user create requested: {"devicetype": "Echo"} from address: x.x.x.x

I do have an Echo Dot Gen 2, but it's currently in my car and haven't tested with that one. I tested it with an Echo Gen 2 and an Echo Dot Gen 3.

I also had a working situation before last week with around 50 devices and suddenly it broke last week. So it's probably indeed some kind of update at Amazon's end.

jayjupdhig commented 3 years ago

Do you guys see any requests from the Echo's in the syslog while discovering and "Trace UPNP calls" enabled under "Bridge Control"?

Yes, i see some processes running while i do a smarthome device search...

Quiphainyr commented 3 years ago

@jayjupdhig

Sorry, I didn't see the screenshots you sent. I just checked them. I noticed that your HA-bridge is listening on:

spark.embeddedserver.jetty.EmbeddedJettyServer - >> Listening on 192.168.178.5:80

And within your HA-bridge it's also set to 192.168.178.5. But the requests from the Echo seems to go to 192.168.178.4. Now I'm not sure if it's possible to have it working this way, but can you try to disable the eth0:192.168.178.4 or try to disable the 192.168.178.5 and change HA-bridge to 192.168.178.4?

dbrucep commented 3 years ago

Yes, here is one the several trace request: Traceupnp: send upnp discovery template 1 with response address: 192.168.1.147:80 to address: 192.168.1.34:50000

...147 is Bridge ...34 is the Echo

dbrucep commented 3 years ago

Here's an image of a discovery trace.

image

jayjupdhig commented 3 years ago

I'm at home now, tomorrow i will continue my "research", thx...

Quiphainyr commented 3 years ago

@dbrucep

When looking at one of your earlier comments, you mentioned that you changed an ID to 00:9c:c1:38:f8:dc:04:cb-cb. Did you only put the "00:" in front of the ID, or did you also change the last two digits "cb-cb"? I tried changing the last digits to have them the same as it was mentioned as a possible solution, but that didn't work for me. When reinstalling HA-bridge, I only added "00:" to the beginning of the ID and did not change the last two digits.