bwssytems / ha-bridge

Home automation bridge that emulates a Philips Hue light system and can control other systems such as a Vera, Harmony Hub, Nest, MiLight bulbs or any other system that has an http/https/tcp/udp interface. This is a compact impl to run on small format computers. This is impl started from this project https://github.com/armzilla/amazon-echo-ha-bridge.
Apache License 2.0
1.45k stars 198 forks source link

Alexa cannot discover devices which used to work. #1293

Open shawn-lindberg opened 3 years ago

shawn-lindberg commented 3 years ago

I apologize if this should have been a reply to another issue, but I have been reading quite a few threads on discovery issues and none of them seem to explain what is happening.

I had been using HA Bridge for quite a few years, most recently version 5.3, with a couple of devices on a Raspberry Pi 3 B. The day before yesterday, Alexa on my Echo Dot started reporting that the devices were not responding. I started digging into this, and confirmed that the devices still responded through the HA Bridge web interface. So, I deleted the devices from the Alexa app and tried to rediscover them. Initially, it looked like this wasn't working because because of a warning about the UPNP IP address about matching a known address on the host, but I was able to fix that by removing a Wi-Fi dongle from the Pi that I didn't need anyway, and resetting the hostname. I also upgraded the bridge to version 5.4. However, the discovery will still not work.

I believe I have tried everything in the troubleshooting page at least once. In particular, I have verified that my Netgear Nighthawk X6 router has UPNP enabled, and the bridge is using a static IP. The logs are interesting. In this case, the Pi running the bridge is IP address 192.168.1.110, the Echo is .14, and my Synology NAS is .12. You can see that the Echo does request a list of the lights, but I don't see a response. habridgelog.txt

EDIT: One other little tidbit that I forgot to add. My friend was over trying to help me with this issue yesterday and he tried to see if we could discover the devices with the Anymote app on his cell phone connected to my network. He was able to, but I wasn't able to do that later.

Any help would be most appreciated.

dbrucep commented 3 years ago

I will give that a try since I added the front "00:" to the already modified "cb-cb"

jayjupdhig commented 3 years ago

thx, will try the same tomorrow...

Quiphainyr commented 3 years ago

What I did was change the name of my HA-bridge folder to "ha-bridge_old", created a new directory "ha-bridge" and installed HA-bridge 5.4.0 from scratch. No settings and no devices. Entered the IP of my Domoticz and the credentials in the new HA-bridge under Bridge Control. All devices were found in HA-bridge. Built a couple of the learned devices to have them show up in the Bridge Devices tab, stopped the HA-bridge service and then started adding the "00:" to the beginning of the unique ID's in the device.db file.

dbrucep commented 3 years ago

So going back to my orignal unique ids and adding "00:" to the front now has my ids in the following form; 00:9c:c1:38:f8:dc:04:cb-f1 The resutls are the same during discovery with no bridge devices being discovered. Trace shows the following image

dbrucep commented 3 years ago

Unless there is a breakthru from someone I will try to simplify my Bridge similar to Quiphainyr suggestion later today. Thanks everyone for working this. Glad to see sucess is possible at least temporarily. Would be nice to get to root cause so that a permanent solution can be developed for all. Right now the behavior appears to be co dependent on many factors associated with the Unique Ids.

dbrucep commented 3 years ago

OK, I went ahead and renamed my ha-bridge directory and created a new instance. Reinstalled ha-bridge in the new empty directory. Started up the bridge and linked to my VERA ZWave Controller. Built one device which was a simple light switch and nothing else. It still can't be dicovered and I tried modifing the Unique Id by inserting a leading "00:" with still no discovery.

00:f8:99:13:9d:f5:e1:05-93

This was done with both a Gen 2 and Gen 4 Echo Dot separtely.

Trace shows communication between Bridge and Echo, just no discovery. Its like the format being sent by the bridge to the echo is not valid in some way but there are no identifing errors.

Its a simple install......and easy to play with but its a random guess as to what to try if anything else. Hopefully BWS is troubleshooting it

shawn-lindberg commented 3 years ago

I just tested this workaround and it indeed seems to work! Only adding "00:" to the beginning of every unique ID seems to be enough and the Link-button has to be pressed while discovering.

Full list of steps the way it works for me:

  • sudo systemctl stop ha-bridge
  • cd ha-bridge folder
  • cd data
  • sudo nano device.db
  • use CTRL+W and search "unique" to find the unique ID's
  • add "00:" at te beginning of every unique ID
  • save the file
  • sudo systemctl start ha-bridge
  • press "Discover" within the Alexa app/site
  • while Alexa is discovering, press the Link-button under "Bridge Devices" within HA-bridge (make sure "Use Link Button" is checked under Bridge Control -> Update Security Settings)
  • the devices should show up within de Alexa app/site

Thanks @SelmaUrban for this workaround!

Thank you for the nice instruction write up. I did have a chance to try this out tonight, but unfortunately after adding 00: to each device, re-initializing, and trying to discover twice combined with the link button, I was still not able to find any of my old devices.

SelmaUrban commented 3 years ago

You need to stop ha bridge! If you change and re-init. it reset the change.

dbrucep commented 3 years ago

I would first make a backup of the device file. Second, stop the Bridge and make a copy of the device file to edit with a new file name. Edit this file unique Id's accordingly. Finnally restart the Bridge and restore from new device file. I confirm that unique I'd changes were made before starting a discovery. Discovery still does not find devices.

I'm curious as to whether Amazon might have modified things a little differnetly here in the US. Has anyone been successful with this remedy in the United States? I say this not knowing where successful contibutors are from but it seems that the communcation runs late night eary morning compared to my California time for those finding this solution. It could be a phased rollout or deployment by Amazon that is showing the mixed results. Anyway, glad to hear some have solved it for now.

acorreane commented 3 years ago

seed id 400 and added a octet in uid in device.db file worked for me. thanks @SelmaUrban example: 01:02:03:05:06:07:08-08

Quiphainyr commented 3 years ago

@dbrucep

Can you try to do it without restoring the old device.db file? So after the new installation of HA-bridge in a new HA-bridge folder, configure the Domoticz connection within HA-Bridge and build a couple of items that HA-bridge receives from Domoticz. It will create a new device.db file in the Ha-bridge->data folder. Stop HA-bridge, change the unique ID's in this newly created device.db file, start HA-bridge again and then try to discover within Alexa.

Not sure if it will help, but I think it's worth the try.

dbrucep commented 3 years ago

Quiphainyr I did it this way as well. Still the same..... Nothing discoverd. Your hub is Domoticz and mine is Vera but I believe that would be the only difference other than possibly Alexa Software/FW.

Quiphainyr commented 3 years ago

@dbrucep

That's strange indeed. I don't think Domoticz vs. Vera would be the issue as the problem is between HA-bridge and Alexa, but it's weird that it doesn't seem to work for you. I don't know all the details of how these connections work and I'm trying my best to help you get a working solution for now as well, but I'm currently out of any thoughts what it could be in your case.

dbrucep commented 3 years ago

Thanks for the help. I'm at a loss as well other than to wail and see if BWS is working (troubleshooting) it or Amazon makes another update. Another possible difference is my Echo's are US and not the International version. Not sure which is being used by others on this thread but if the're US based and not successful it suggest why results are different.

shawn-lindberg commented 3 years ago

You need to stop ha bridge! If you change and re-init. it reset the change.

Yes, I did stop it, make the change, and then start it back up again. I also verified that the device IDs were updated before trying the discovery. Is that what you mean?

lylerhardy commented 3 years ago

I am in the US and using Echo Dot Gen2. I followed the suggestions multiple different ways (at least two dozen times) as covered in this thread. Nothing worked. Alexa would not discover my single light on a fresh install of 5.4.0. Today I tested again, with the leading 00: AND matching the last two characters (where previously I had only added the leading 00: and had NOT changed the last two characters) to follow the pattern 00:XX:XX:XX:XX:XX:XX:YY-YY and it discovered the light. I next added a second light making the above two changes again and it discovered the 2nd light. Lastly, I added a 3rd light and only added the 00: at the beginning and DID NOT match the last two characters. Surprisingly, it also added the 3rd light.
Not sure if my experience helps. I don't know why it didn't find the 1st light until I matched the last two characters, but found the 3rd light even when I didn't.

mjrgh commented 3 years ago

I don't know why it didn't find the 1st light until I matched the last two characters, but found the 3rd light even when I didn't.

One thing I've noticed in my attempts to get it working is that the discovery process isn't repeatable for me between passes. Each time I do it, making no changes to the setup between passes, it discovers a different subset of the devices (and my bigger problem has been that even the devices that ARE discovered don't actually accept commands). That might explain the inconsistent behavior you're seeing, and why some people are reporting that they get discovery working and some aren't. All of the "this works/this doesn't" might end up being random coincidences. Maybe if everyone who's reported one way or the other runs a few more discovery passes, their results would change each time too. There seems to be something sporadic or stochastic in the behavior on my system, at least.

d3smo commented 3 years ago

Adding the leading 00: to follow the pattern 00:XX:XX:XX:XX:XX:XX:YY-YY also work for me, take care at the numbers of couples of characters i'd got 6 couples before so i had to add 00:00: to match the height couples of charaters before the -

kristofvanwoensel commented 3 years ago

Persistence pays off...

I tried every solution you guys suggested, but these steps worked for my Echo 2th generation International version

  1. Restore backup in Ha-bridge GUI
  2. Click 'Renumber devices'
  3. Go to device.db and add 00 before each uniqueID
  4. Save file
  5. sudo systemctl stop ha-bridge.service and sudo systemctl start ha-bridge.service
  6. Remove all smart-devices from Alexa app
  7. Remove Cache (https://www.amazon.com/hz/mycd/myx#/home/alexaPrivacy/deviceHistory)
  8. Find devices (no LINK button used)
jayjupdhig commented 3 years ago

That "Remove Cache" links redirects me to the page https://www.amazon.com/hz/mycd/myx#/home/content/booksAll/dateDsc/ ... not related with books i.m.o. (?)

jayjupdhig commented 3 years ago

Oh, i was not logged in, now the link https://www.amazon.de/hz/mycd/myx#/home/alexaPrivacy/deviceHistory seems to work ...

jayjupdhig commented 3 years ago

-> How to change "id" and "uniqueid" in the device.db via a HTTP request to the HA-Bridge webserver?

I can't change the device.db file because the HA-Bridge sets back the device.db file back to 0644... and with 0644, i can't change it via PHP exec(...) command.

How to solve that issue? Any hints?

Quiphainyr commented 3 years ago

@jayjupdhig Do you only have an option to use HTTP requests to the HA-bridge server, or do you have the possibility to logon to the HA-bridge server as well? I assume you're running your HA-bridge on a local machine, which would mean you should be able to access that machine, right? That way you can just change the device.db file with a text editor on the machine itself, or download the file to another machine, make the adjustments and upload it back to the HA-bridge server, depending on what you prefer.

jayjupdhig commented 3 years ago

No option for use - your SW needs to be end-user-friendly... so the whole workaround thing (re-numbering, add "00:" to UID) should run via PHP...

FadingDreams commented 3 years ago

Is the leading 00: really required? Tasmota had the same problem and there it seems to be solved by changing the end string to "uniqueid":"xx:xx:xx:xx:xx:xx:00:11-01" (https://github.com/arendst/Tasmota/pull/11415). Unfortunately I have no possibility to try it out this week.

bwssytems commented 3 years ago

Interesting, I just looked at the Hue Developers info for the API and it now shows 9 octets. I had 7 previously, fixed it to 8 and now 9 is in the example. I will put in a switch on bridge control to renumber using 9 octets so that you can try it.

bwssytems commented 3 years ago

Please delete your devices in the echo before you try this or you may end up with dups...

Try this release and set the "Unique ID to use 9 Octets" in the Bridge Control tab, save the setting, then rediscover devices on the echo.

https://github.com/bwssytems/ha-bridge/releases/tag/v5.4.1RC1

ThoWoKi commented 3 years ago

My Echo 2 (EU/DE) just received an Update to Software-Version 663648120. Everything works now again!

ha-bridge 5.4.0 (with previously added "00:" to the beginning of the uniqueid.)

castellorafa commented 3 years ago

Hi bwssytems , no luck with de RC1 and the 9 octet button from Spain with Echo Plus gen2 , 5.4 was working fine till today with the leading 00: fix by hand done some days ago. I supose it is no related, but some days ago my Hp printer 'appeared' as a device in my echo (it have never been there as a device) and today it is gone again... May be a new echo update?.

dbrucep commented 3 years ago

New release 5.4.1RC1 didn't work for me as well with 9 button checked and new Id Seed followed by a Re-Number. The prior unique id adjustment also never worked as well for my installation in US with Echo Dot Gen 2. Handshakes between Echo and Bridge occur during discovery but no devices appear in Alexa App.

Anyway, still not functional for two weeks since Amazon pushed an auto discovery.

bwssytems commented 3 years ago

Well, not sure what can be done. I have no issues with either version on discovery, but mine is a US version. Wondering if the European version is different for the Echo.

castellorafa commented 3 years ago

Just in case it could have utility, i put here the degub trace from one of my failed discovery . debug_trace_ha_bridge_discover_from_Alexa.txt

dbrucep commented 3 years ago

Well, I'm glad to hear you have a US working version. Did you get an auto dicovery that broke it and were able to get it to work with the new 9 Octet? If not maybe it could be a difference in FW for the Echo Gen 2. My version is 663647720. Not sure if that was what change the past week.

jayjupdhig commented 3 years ago

Was is fixed in the new RC1 version? THank you for your feedback.

castellorafa commented 3 years ago

Hi ThoWoKi can you show us a sample working unique-id please ?

ThoWoKi commented 3 years ago

Hi ThoWoKi can you show us a sample working unique-id please ?

"uniqueid": "00:f8:99:13:9d:f5:e1:05-93", "uniqueid": "00:38:b3:ef:f8:ba:f5:66-27", "uniqueid": "00:ec:89:56:63:7a:99:78-7b", "uniqueid": "00:69:74:ce:5a:c6:60:61-0b",

I also attached my (beautified) device.db. device.db.txt

castellorafa commented 3 years ago

Ha ha, this is crazy..... I deleted all devices on ha-brigde, added just one with the 9-octet option enabled and it received the same unique-id of the first on ThoWoki list, rediscovered from Alexa , and then instead of only this device alone, re-appeared all my last time working devices... restored the last working copy of "Bridge Device DB Backup" (all of then with the 8 octet length) , an everything is working again..... and yes, I have done the same things before with no result till today.... But how long will it last?...... Nobody knows ... :-(

dbrucep commented 3 years ago

Still no good here.

I have made just one device with 9-octet option on and renumbered devices starting at 100. Unique Id is also the same 00:f8:99:13:9d:f5:e1:05-93 (interesting)

Anyway, device discovery with Alexa still does not return device.

HalRum commented 3 years ago

Wow! I thought I was losing my mind until I found this thread. I coincidently changed my Internet provider this past week and have been fighting the problems described above for a few days now. It was relatively straight forward to change the IP addresses for HG and HA but then Alexa would not discover any of my switches from HG. I dropped all my echo's multiple times but they get rediscovered and nothing else. The work arounds sound like someone made changes and forgot to drop all the old device ID's when the devices are dropped. I guess they work fine until you go to rediscover them eventually. Once dropped they don't get reactivated to the same device ID's so any changes you make to the device ID makes Alexa "discover" a new device each time. This apparently works OK until you try another rediscovery? Sounds like it's hardly worth trying to keep up with changing device ID's until someone addresses the real issue?

jayjupdhig commented 3 years ago

Sometime it works, sometime not - that s**t seems to be useable as a random generator!! ;-)

Neokamikaze commented 3 years ago

Hello, same problem here. Alexa no longer detected the equipment but with the RC1 and a modification of the unique id it obviously works again. I continue my tests to see if it works.

ThoWoKi commented 3 years ago

Sorry, I forgot to mention... since 2 Days my Echo does not find any devices anymore. The HA-Bridge devices, which worked before are unrecognizable... :(

So I think it is not a primary problem regarding HA-Bridge. It seems more like a problem of the undocumentated API of Alexa...

The Software-Version of the Echo Gen2 (EU/DE) is still 663648120

wube1 commented 3 years ago

Hi Everyone, I'm using QNAP for my home automation (Domoticz) and I have used container station with HABridge installed (5.3.0 version). It was working fine until March 9th. So far i can see that except container I'm using there are also Raspberry Pi 1,2,3 HABridge images which dont work for me (Maybe because my QNAP has x86 architecture). Any advice how to get it to work (Echo dot gen2)?

bwssytems commented 3 years ago

Alright, I have reset my echo to see if I can see the issue. I have found that the echo is not sending out UPNP messages at all. So no discovery of devices will happen at all. Will need to see if there is any news from amazon on local discovery of hue devices.

beandi commented 3 years ago

https://github.com/bwssytems/ha-bridge/issues/1270#issuecomment-812894624 Had some success by editing device.db and replacing the last "-"-separator with a ":"-separator and 9 octets - every device was instantly discovered

svezg commented 3 years ago

After pulling my hair out for days, I'm back online again based on @beandi response above... After I initially updated to 5.4.1RC1 and used 9 Octets I had no success whatever I did; when changing the ID and replaced the last "-" to ":" no success either... As last resort I removed the echo (dot rev2 EU) from my account and waited for about 30 mins; added it again and magic happened!! In earlier test I used the privacy settings on the web page to forget the devices but apparently that is somehow different than removing it from your account.

So... please give this a try and if you succeed join me in my happy dance when you get your house back online.

ThoWoKi commented 3 years ago

On my side it's also working again. Without further changes. Let's see how long it works.

HalRum commented 3 years ago

Alexa is working again with HG and HA entries!
I downloaded RC1, deleted my device.db file, forced the recreation of my habridge.config file by renaming it an making sure no reference to it was in the ha-bridge.service file. I then restarted ha-bridge with the new RC1 jar file, went in and turned on the 9 octet setting, pointed the HomeGenie settings to the proper IP and Port, and then recreated my Bridge entries from the HomeGenie Devices tab. I then went to the Alexa app on my phone and went to devices, selected my Echo Dot's and Echo Studio and Deregistered all of them. Next my Alexa app quickly rediscovered new Echo devices and I registered my Dot 3 first. When registration completed I asked Alexa to discover new devices and it finally found all 16 of them that I had just recreated in HA. Initial testing shows at least some of them are responding normally now to Alexa requests to turn on and off. Thank you BWSSYSTEMS for the new patched release! By the way, since all device.db entries were recreated when I used the HomeGenie Devices tab to recreate them, I did not have to edit the file and change the ID's manually. I also could not get the bulk add option to work for me.

jayjupdhig commented 3 years ago

ok thx

bwssytems commented 3 years ago

So, got my echo to do the upnp discovery and see that 5.4.1RC1 is broke on renumbering. 9 octets with the last one with a dash works when I edit the device.db. Will look into whats wrong with the code.