home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
71.08k stars 29.73k forks source link

MyQ integration not working after upgrade to 2023.10.2, Error: : 403 - Forbidden #101912

Closed NukeThemTillTheyGlow closed 10 months ago

NukeThemTillTheyGlow commented 11 months ago

The problem

This issue existed prior to the update to 10.2, but 10.2 was supposed to fix the problem and appears to have done so for others. I did not apply any of the patches mentioned in the pre-update thread.

What version of Home Assistant Core has the issue?

core-2023.10.2

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

MyQ

Link to integration documentation on our website

https://www.home-assistant.io/integrations/myq/

Diagnostics information

home-assistant_myq_2023-10-12T19-36-26.955Z.log

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

drothenberger commented 11 months ago

Home Assistant 2023.10.3 Supervisor 2023.10.0 Operating System 11.0 Frontend 20231005.0 - latest and I'm using Terminal & SSH Current version: 9.7.1 it won't let me copy the entire session but here's what I get [core-ssh ~]$ docker exec -it homeassistant /bin/bash bash: docker: command not found

You need the "Advanced SSH & Web Terminal" add-on with protection mode disabled to use the docker command.

But, since you don't already have that installed, it seems unlikely that you ran any of the pip commands, since they require the docker command first.

Are you sure you don't have a "pymyq" folder in your config folder? You can check with this command:

ls -d /root/config/pymyq

If the output is "pymyq", you have the folder, and should run rm -fr /root/config/pymyq

drothenberger commented 11 months ago

Home Assistant 2023.10.3 Supervisor 2023.10.0 Operating System 11.0 Frontend 20231005.0 - latest

That command should be run directly from the shell of the system you are running hass on, that is if you are running a docker version of hass ..

They're running Home Assistant OS, not the docker version.

shiftylilbastrd commented 11 months ago

I had Terminal & SSH installed not Advanced SSH & Web Terminal. Switched over and everything worked perfect

NukeThemTillTheyGlow commented 11 months ago

Home Assistant 2023.10.3 Supervisor 2023.10.0 Operating System 11.0 Frontend 20231005.0 - latest and I'm using Terminal & SSH Current version: 9.7.1 it won't let me copy the entire session but here's what I get [core-ssh ~]$ docker exec -it homeassistant /bin/bash bash: docker: command not found

You need the "Advanced SSH & Web Terminal" add-on with protection mode disabled to use the docker command.

But, since you don't already have that installed, it seems unlikely that you ran any of the pip commands, since they require the docker command first.

Are you sure you don't have a "pymyq" folder in your config folder? You can check with this command:

ls -d /root/config/pymyq

If the output is "pymyq", you have the folder, and should run rm -fr /root/config/pymyq

Boom! That did it! I used the regular Terminal option, but confirmed the folder was there and then removed it, rebooted and enabled the integration. Worked immediately. Many thanks!

drothenberger commented 11 months ago

Boom! That did it! I used the regular Terminal option, but confirmed the folder was there and then removed it, rebooted and enabled the integration. Worked immediately. Many thanks!

I'm glad you got it sorted.

I usually just use the Studio Code Server add-on to delete the folder. :-)

jonahgcarpenter commented 11 months ago

docker exec -it homeassistant /bin/bash pip uninstall python-myq rm -rf config/pymyq rm -rf /usr/src/homeassistant/homeassistant/components/myq/ wget https://github.com/home-assistant/core/archive/refs/tags/2023.10.3.tar.gz tar -xvf 2023.10.3.tar.gz cp -fr core-2023.10.3/homeassistant/components/myq /usr/src/homeassistant/homeassistant/components/ rm -rf core-2023.10.3 rm 2023.10.3.tar.gz

I tried these commands with no luck, then i tried these

/root/config
wget https://github.com/justinlindh/Python-MyQ/archive/refs/heads/fix-403-forbidden.zip
unzip -o -j fix-403-forbidden.zip -d pymyq

i unadded the integration way before this and now when i try to re-add it it just sits and spins on the "Please wait, starting configuration wizard for myq" and i have not a clue where to go from here but this all started with the common 403 error found above.

kilrgt commented 10 months ago

Anyone get anywhere with this yet? I use this for a few automations and if it’s not going to work, I’ll have to find another way, not frustrated with people helping with this,just frustrated at the situation.I wish i was smarter lol

BillBlight commented 10 months ago

Anyone get anywhere with this yet? I use this for a few automations and if it’s not going to work, I’ll have to find another way, not frustrated with people helping with this,just frustrated at the situation.I wish i was smarter lol

Seems to be working with stock Home Assistant 2023.10.3, I did remove and re-add the integration but works fine ..

(you may have to disable it for at least a couple hours to get rid of the throttle/timeout)

kilrgt commented 10 months ago

Anyone get anywhere with this yet? I use this for a few automations and if it’s not going to work, I’ll have to find another way, not frustrated with people helping with this,just frustrated at the situation.I wish i was smarter lol

Seems to be working with stock Home Assistant 2023.10.3, I did remove and re-add the integration but works fine ..

(you may have to disable it for at least a couple hours to get rid of the throttle/timeout)

I had it disabled for a day, then retried to enable..nothing then I deleted it , restarted tried to re add but then it wouldn’t log in with know good credentials. Restored to 10.2, to get the integration back in HA, then updated again to 10.3 and still erroring out. It’s defiantly confusing ,doesn't seem too many others are still having issues. Thanks for the reply

BillBlight commented 10 months ago

Anyone get anywhere with this yet? I use this for a few automations and if it’s not going to work, I’ll have to find another way, not frustrated with people helping with this,just frustrated at the situation.I wish i was smarter lol

Seems to be working with stock Home Assistant 2023.10.3, I did remove and re-add the integration but works fine .. (you may have to disable it for at least a couple hours to get rid of the throttle/timeout)

I had it disabled for a day, then retried to enable..nothing then I deleted it , restarted tried to re add but then it wouldn’t log in with know good credentials. Restored to 10.2, to get the integration back in HA, then updated again to 10.3 and still erroring out. It’s defiantly confusing ,doesn't seem too many others are still having issues. Thanks for the reply

Need to make sure any other patches for this you had installed are completely removed ..

AdrianGarside commented 10 months ago

Stopped working again a few days ago. Finally got around to updating my docker image from 9.3 to 10.3. Initially it didn't connect with 429 errors and that triggered the back off but I checked now and it's successfully working again now a few hours later.

kilrgt commented 10 months ago

if I uninstall the MyQ integration, does that achieve the same result of removing the old patch fix?

Didn’t work for me,I did uninstall it , restarted then tried to reinstall but wouldn’t log in with with good credentials. I’ve tried todo what most people say will work and remove any patches i installed last month, and no matter what i try it’s not working.

iFixRobots commented 10 months ago

Same here. Can't connect at all now. CleanShot 2023-10-15 at 20 19 57@2x

sjogreen commented 10 months ago

I'm running home assistant OS, haven't previously taken any manual steps to make this work, and seeing the same problems. I tried uninstalling and reinstalling and get the Failed to Connect message on login that @iFixRobots screenshotted above.

gsxdsm commented 10 months ago

Same removed everything, uninstalled, re-installed. Can't login.

kilrgt commented 10 months ago

Finally had a little time to get this back up and running. Reading all the comments , trying all of the suggestions it wouldn't work. Maybe for those that have the same issue as I, here is what i did to get it back up and running, I did try to use the Advance SSH, nothing..so I took another approach, full disclaimer, try at your own risk

First do a complete back up then uninstall the MY Q add on Restart HA then use the add-on File editor Remove all of the files out of the folder pymyq (delete each one) You can try and delete the main folder (pymyq) but it will error out rename the pymyq folder to what ever you want Restart HA--- do not try and reinstall MYQ yet. Wait 2 hours and reset up, if is errors out disable My Q wait another few hours and retry

Again this is what i did after i tried all of the suggestions, it worked for me but if you corrupt your instance please know this only worked for me, this is why you always back up

iFixRobots commented 10 months ago

Note on @kilrgt's message: I saw a lot of people had been removing or talking about this folder. So I decided to look into it.

It looks like this folder is the culprit after the latest Core updates. Here's what I did to fix it:

  1. Use Terminal or File Editor to login to your HA.
  2. Type pwd to see which folder you're on.
  3. You need to be here: /root/config
  4. Type cd /root/config
  5. Now let's delete that folder we keep talking about sudo rm -r pymyq
  6. Restart HA (This NEEDS to happen). Developer Tools -> Restart -> Restart Home Assistant --wait for it to come back up.
  7. Settings -> Devices & Services
  8. Add Integration
  9. Type myq
  10. Enter username & password
  11. It should be working now.
kilrgt commented 10 months ago

Thanks for your input !

kilrgt commented 10 months ago

Tried what you suggested and instantly get a cannot connect... 🤷🏼

Just wait a couple of hours.... Disable the Integration if its still there. Restart and try install again

lightingman117 commented 10 months ago

I can confirm that mine garage door (Liftmaster via MyQ) stopped working a few weeks ago while running 23.8.3 I updated to 23.10.3 - did not fix. I deleted the integration and waited 2 days. I added it back. The MyQ Gateway is now 'connected' (Both AC & Battery), but the garage door does not show up.

Should I delete everything and start over? Not sure what to do next.

Edit: Weird, I went into the MyQ app and clicked around randomly looking for anything that might be broken. I renamed my garage door. Then I went into HA again and found the cover was connected to the MyQ as a device. Seems to be working again.

Thanks!

NukeThemTillTheyGlow commented 10 months ago

FYI, I was using SmartThings previously and someone build a MyQ integration that uses Docker to connect ST to MyQ. Noticed they are having all of the same issues as Home Assistant MyQ integration. Hope this helps.
[(https://github.com/brbeaird/SmartThings-MyQ-Edge/issues/47)

Last, first time I've created an issue on GH. Do I need to be doing anything with this? I'm up and running again after deleting the old folder.

Lash-L commented 10 months ago

If you are no longer having an issue @NukeThemTillTheyGlow Then you can close the issue

NukeThemTillTheyGlow commented 10 months ago

Will do. thanks.

NukeThemTillTheyGlow commented 10 months ago

Closing as deleting the old MyQ folder was my issue.

chinmayjade commented 10 months ago

I am still getting the Failed to connect error when trying to re-install the integration. I followed the @iFixRobots's steps and even allowed ~15 hours after delete & restart.

Any solutions?

jeeftor commented 10 months ago

Mine worked till mid-day today and now its down again :(

jeeftor commented 10 months ago
image
bobokun commented 10 months ago

I'm not even able to install myq integration through the UI. When searching for MyQ in the add integration button it doesn't provide any popup asking for credentials.

lightingman117 commented 10 months ago

Welp, it broke again.

Gonna order two of these (ratgdo) and be done with cloud/MyQ completely.

smgraver1 commented 10 months ago

Mine won't let me log in

JGrantman commented 10 months ago

Same boat. Had it working in Homebridge and read MyQ Integration was fixed so deleted it from Homebridge and created MyQ integration from scratch in Home Assistant. Installed but would not come online and logs show same error as most others in the recent posts ending up with 429 too many requests.

I uninstalled MyQ and scrubbed looking for anything left over. Couple restarts and went to shell and run HA Core Rebuild and HA Supervisor Repair... then full hardware power down and startup.

Check MyQ app on iOS and Android device and working -- no issues.

Went and tried to install MyQ integration from scratch in HA and would not even prompt. So I went to Home Assistant MyQ integration web page and selected add which did link back to my HA instance and pop up the login UI window but it would not work. Immediately after clicking "Submit" it came back with "Failed to Connect". However, NOTHING in logs. There isn't even a line written to confirm the submit was received and sent to MyQ cloud.

I went to 2 other different computers to use different web browsers (to eliminate any possibility of cache issue) EXACT SAME experience.

It's one thing if MyQ stops working unprompted (as it has done in past but not in year or more). This keeps happening only after an update to Home Assistant and until today I could always get it working in Homebridge when it broke in Home Assistant.

I waited 4 hours and I am still getting refused by MyQ for too many attempts. I think earlier today there were hundreds or maybe thousands of lines in log with Home Assistant pounding MyQ, getting refused for too many attempts, literally writing in log how it was pausing but then continuing to pound MyQ and filling up log.

Admittedly Chamberlain's entire MyQ platform is ... uh... not world class :) And from previous experience in enterprise IT with the company (15+ years ago) I have zero expectation it will get better. But it was working in Home Assistant until a Home Assistant update. Then work arounds that worked are nuked with subsequent updates without fixing root cause... leaving things worse after each update.

It would not be that frustrating if it was a Tuya sketch-ware Amazon fake vendor product. Because it nukes the garage door it is more irritating than when a Tuya light won't change color correctly or falls off network.

webmogul1 commented 10 months ago

Also, "was opening" is not being reported anymore.

jermanoid commented 10 months ago

It's a good time to short MyQ/liftmaster/chamberlain stock. They're doing all this nonsense, and who is going to pay for their doodoo cloud when everyone already uses all these other well established free or self hosted clouds?

rgnet5 commented 10 months ago

MyQ is definitely throttling their API. Getting the same 429 error (too many requests) on HomeBridge

Lash-L commented 10 months ago

It's one thing if MyQ stops working unprompted (as it has done in past but not in year or more). This keeps happening only after an update to Home Assistant and until today I could always get it working in Homebridge when it broke in Home Assistant.

@JGrantman it has nothing to do with a HA update. Users who did not update at all were having issues at all. This is all on myq's side. It is currently broken in homebridge as well for most users.

wrmacj commented 10 months ago

@Lash-L Do you think that it has anything to do with their implementation of Cloudflare. It apears that when I first login the the MyQ app after and IOS reboot and the first time I login the service goes to Cloudflare for a few seconds before it takes me to the login screen that says, "Welcome Back" Every time after this when I launch the MyQ app it goes imediatly to the Login screen with the "Welcome Back" back, as is Cloudflare has validated my phone and app. Each time I repeat the process of loging out and rebooting the phone(I may also have to switch from Wifi to Cell) I get a get forwarded to the CloudFlare site and get a new Ray ID. If I do not logout and reboot, I do not get see the CloudFlare site. image

wrender commented 10 months ago

Mine also just suddenly stopped working again today.

stephenmetts commented 10 months ago

FWIW - I'm experiencing the same issue.

jermanoid commented 10 months ago

Has anyone tried to sniff the traffic coming from the MyQ garage doors to see what those calls looks like? I'm not an expert in this. But perhaps there is a way to emulate their server locally instead of authenticating with MyQ cloud as a client. Then use a local DNS entry to route the myq-cloud.com domains to the homeassistant server.

BillBlight commented 10 months ago

Mine just suddenly started working, just decided to re-enable it for giggles, and it is currently working ..

AdrianGarside commented 10 months ago

Still not working for me. Seems to be trying but failing to pause:

2023-10-20 20:56:19.647 DEBUG (MainThread) [pymyq.api] Performing initial authentication into MyQ 2023-10-20 20:56:19.648 DEBUG (MainThread) [pymyq.api] Scheduling token refresh, last refresh was None 2023-10-20 20:56:19.649 DEBUG (MainThread) [pymyq.api] Initiating OAuth authentication 2023-10-20 20:56:19.649 DEBUG (MainThread) [pymyq.api] Retrieving authentication page 2023-10-20 20:56:19.650 DEBUG (MainThread) [pymyq.request] Sending myq api request https://partner-identity.myq-cloud.com/connect/authorize and headers {'redirect': 'follow'} with connection pooling 2023-10-20 20:56:20.013 DEBUG (MainThread) [pymyq.request] Attempt 1 request failed with exception : 429 - Too Many Requests 2023-10-20 20:56:20.013 DEBUG (MainThread) [pymyq.request] Too many request have been made - putting a temporary pause on sending any requests for 45 minutes 2023-10-20 20:56:20.016 WARNING (MainThread) [homeassistant.config_entries] Config entry 'REDACTED' for myq integration not ready yet: Got 429 error - stopping request until 2023-10-21 04:41:20.014125. there were 1 request; Retrying in background

5 seconds later (not 45 minutes) it tries again:

2023-10-20 20:56:25.129 DEBUG (MainThread) [pymyq.api] Performing initial authentication into MyQ 2023-10-20 20:56:25.129 DEBUG (MainThread) [pymyq.api] Scheduling token refresh, last refresh was None 2023-10-20 20:56:25.130 DEBUG (MainThread) [pymyq.api] Initiating OAuth authentication 2023-10-20 20:56:25.130 DEBUG (MainThread) [pymyq.api] Retrieving authentication page 2023-10-20 20:56:25.130 DEBUG (MainThread) [pymyq.request] Sending myq api request https://partner-identity.myq-cloud.com/connect/authorize and headers {'redirect': 'follow'} with connection pooling 2023-10-20 20:56:25.433 DEBUG (MainThread) [pymyq.request] Attempt 1 request failed with exception : 429 - Too Many Requests 2023-10-20 20:56:25.433 DEBUG (MainThread) [pymyq.request] Too many request have been made - putting a temporary pause on sending any requests for 45 minutes

JGrantman commented 10 months ago

It's one thing if MyQ stops working unprompted (as it has done in past but not in year or more). This keeps happening only after an update to Home Assistant and until today I could always get it working in Homebridge when it broke in Home Assistant.

@JGrantman it has nothing to do with a HA update. Users who did not update at all were having issues at all. This is all on myq's side. It is currently broken in homebridge as well for most users.

You appear to be correct. The update stuff only accelerated what looks to have been inevitable. At some point a valid login is rechecked and that is when the issue hits. Looking more and more like a deliberate move by Chamberlain... not specifically to lock out DIYers but to make changes. In past month or two have been final cut off of the Google Home integration, addition of in-car integration for multiple car manufacturers and final cut off for homekit.

Clearly changes are taking place and several have mentioned addition of Cloudflare which may be final hurdle that can't be easily or reliably jumped.

It's been a good run ... well a bumpy run with periods of stability which seems reasonable for the $19 I paid on a massive deal for the MyQ garbage hub on Amazon maybe 5-6 years ago?

It's been periodic unstable WiFi connection, random issues with both iOS and Android app, permanently getting 50% of notifications pretty delayed, the stupid beeping and flashing that goes way beyond any gov't requirements and... the general low quality and poor experience with the garage door opener itself long before MyQ... it's time to move on.

Likely going to try Meross as stop gap until I just replace entire garage door with anything not made or sold by Chamberlain or it's many sketchy sub-entities. (Too bad there is not a decent full feature Wyze integration because their opener + camera seems ideal as stop gap).

If Meross sucks but works it will feel like such an upgrade I won't care :)

(What is really strange is I can go to a login at "https://partner-identity.myq-cloud.com" and it works without any issues or anything in background throwing errors (checking in Chrome F12 options). The page does not go anywhere or do anything but I expected the login itself to fail.)

The IT part of me wants to see this working but the rest of me has passed the point of wanting to keep using anything related to Chamberlain no matter how it plays out.

Good luck and I do hope the devs working on this out smart whatever incompetence is taking place on the Chamberlain or actually the sketch external vendor they use to provide the MyQ dumpster fire.

? Maybe if I had said yes to one of the 19,000 Amazon in garage delivery prompts it would not have stopped working ?

jermanoid commented 10 months ago

Interesting. TCP dump says my garage door connects to a remote IP on port 8883 using port a source port of 52280. Which means MyQ uses MQTT on their side.

safadig commented 10 months ago

for those who are copy and pasters when there are fix commands ratsputin's listed commands should work but he didn't line them out removing the pymyq folder got missed i was working so i was multitasking at the same time and overlooked the combined line commands. ratsputin also didn't give the actual commands to clone the repo. since i barely use git-clone (yeah i know i don't usually use other people's code, but i'm somewhat new to this ecosystem so i haven't had time to build my own) and use wget i will give the wget option for you all.

to remove all patches on supervised as well as hassos and docker. I used putty a user after i originally was using the ssh addon so the docker command was not needed for that user

My Needed Commands docker exec -it homeassistant /bin/bash pip uninstall python-myq rm -rf config/pymyq rm -rf /usr/src/homeassistant/homeassistant/components/myq/ wget https://github.com/home-assistant/core/archive/refs/tags/2023.10.3.tar.gz cp -f core-2023.10.3/homeassistant/components/myq /usr/src/homeassistant/homeassistant/components/ rm -rf core-2023.10.3

for those who could not copy the files as shown. Here is an updated version:

docker exec -it homeassistant /bin/bash  
pip uninstall python-myq  
rm -rf config/pymyq  
rm -rf /usr/src/homeassistant/homeassistant/components/myq/  
wget https://github.com/home-assistant/core/archive/refs/tags/2023.10.3.tar.gz  
tar -xvf 2023.10.3.tar.gz core-2023.10.3/tests/components/myq  
cp -rf core-2023.10.3/tests/components/myq /usr/src/homeassistant/homeassistant/components/  
rm -rf core-2023.10.3
wrender commented 10 months ago

Wouldn’t home assistant be connecting to an api on the myq cloud end. I wonder if there is a changelog for their api.

wrmacj commented 10 months ago

A workaround that worked for me. Just bought a MyQ 819LMB of kijiji for $40 and now have MyQ running local with a newer Yellow triangle and older Purple button Chamberlin garage doors. At this point it is a plug and play full featured solution.
This allows for three use vectors. Full access and controll from HASS local Bridge(No internet access), Homekit and the MyQ app. The has been discontinued for some time and is no longer recieving SW support from Chamberlain. Also the chamberlin remotes are dry contact if you want to hook them up to a shelly. You can get control over your doors that way but you would have to add a sensor to the door to know if it is open or close. The 953ESTD works only with yellow triangle garage door openers but will controll three doors. The 953EV will control three doors from Yellow, Green or purple.

shanelord01 commented 10 months ago

Wouldn’t home assistant be connecting to an api on the myq cloud end. I wonder if there is a changelog for their api.

This plugin doesn't use official access to the API. I'm wondering if Chamberlain would be open to a per user developer API, that this plugin could use. At the moment this seems like a losing battle.

As an aside; If I was one of their devs told to block the HA MyQ plugin, I'd wait for the code to be fixed, look at it on the HA github then program a way to block it. Sometimes opensource (in this case where the vendor can look at the code) isn't our friend.

skybasemonkeyha commented 10 months ago

A workaround that worked for me. Just bought a MyQ 819LMB of kijiji for $40 and now have MyQ running local with a newer Yellow triangle and older Purple button Chamberlin garage doors. At this point it is a plug and play full featured solution. This allows for three use vectors. Full access and controll from HASS local Bridge(No internet access), Homekit and the MyQ app. The has been discontinued for some time and is no longer recieving SW support from Chamberlain. Also the chamberlin remotes are dry contact if you want to hook them up to a shelly. You can get control over your doors that way but you would have to add a sensor to the door to know if it is open or close. The 953ESTD works only with yellow triangle garage door openers but will controll three doors. The 953EV will control three doors from Yellow, Green or purple.

i bought one its currently on order, so i'm going the same route as you, for those who also care to know the integration works for a good maybe 10 minutes before giving a time out error (exact time unknown havent paid enough attention) for the next 2 or 3 hours if you disable or renable or reload the integration or if you loose internet connectiving at any time you are going to automatically fall back into that 429 trap and have to wait that out. furthermore, the app using the same external ip will still work when ha has the 429 error, it might have something to do with said user agent. i am looking at the code, and seeing if i can break down a fix myself. it could be that there is some kinda of official vs non-official api calls, what we need to do is break down the app. I am currently decompiling the app to see what we are doing wrong.

wrender commented 10 months ago

Ya. I agree. It does seem like a losing battle. Im trying to weigh my options.

I’m about to sell my house so can’t have unreliable smart home stuff in it. I’m even wondering if having home assistant sold with the house is reliable enough.

All of my lights are zwave so I was thinking of the GoControl garage door opener as a replacement for myq. Hard to know what would be reliable.

lauzca commented 10 months ago

If you did anything to manually try to fix the problem - you need to undo it. i.e. remove the pymyq from the config folder and/or do

pip uninstall python-myq

and pip uninstall pymyq

If you see 'IOS' anywhere in your logs relating to myq, you are stuck on an old version.

Hi, I'm hoping you can give me a little help here. I have been having these issues for over a month. I did try the fix that was put out and now it isn't working. I see above it says to run pip uninstall python-myq. I assume I type that into terminal (not a techy) I get "bash: pip: command not found" when doing that. It says I am running it in [core-ssh ~]$ --- whatever that means. I did however go to the config folder and delete the myq folder. Still no luck....