Closed atruest closed 9 months ago
They would sell more of these boxes if they also opened up a local API to talk to the devices on your local network from your HA server rather than force the cloud service API dependency. I can't image it would be any more or less security. If your cloud account can be hacked your garage can be opened from anywhere in the world. If you protect your home network, someone would have to break into your home network to talk to the device locally to open your door.
Yeah, but now they can double-dip by charging you for the garage door opener AND access via the app - it's what they're doing with Tesla, and what they're doing with IFTTT. Their argument will be "You can ALWAYS just open the MyQ app and open/close for free, and you can ALWAYS fall back to the clicker/remote you put in your car." If you want access to the API for any other reason, you have to pay.
The fact is that the HA integration (along with NodeRed, Homebridge, etc) are all reverse-engineered capabilities to get something for free. It's only a matter of time before Chamberlain puts the API behind a tokenized URL and you have to subscribe to use it. I am personally not skilled enough in programming/coding to resolve this issue, so i'm putting my efforts into an alternate solution (ratgdo). I only use MyQ in HA to monitor status from work, and an automation to automatically close when no motion is detected in the garage after a period of time.
Did something change yesterday? I went to close my garage door in HA and it showed disconnected. I restarted HA and checked the MyQ app (on IOS) and that still works. I signed out of MyQ in HA and now when i try and sign back in it says "Failed to Connect"
Was working perfect after the core update till this morning
Did something change yesterday? I went to close my garage door in HA and it showed disconnected. I restarted HA and checked the MyQ app (on IOS) and that still works. I signed out of MyQ in HA and now when i try and sign back in it says "Failed to Connect"
Was working perfect after the core update till this morning
How did you manage to install HomeAssistant while being unable to read?
I mean, it's an issue for me. I have a Special Needs son who gets home from school at a certain time, and if my wife isn't home, I need access to the garage door to let him in. Previously, HA has been more reliable than the MyQ phone app - but no longer. Now I'm stuck with whether or not I can get the app to work properly - sketchy at best. This is becoming a safety issue for my family (not merely convenience). I fear that the entire industry is going to go this way, which will destroy our HA community.
I mean, it's an issue for me. I have a Special Needs son who gets home from school at a certain time, and if my wife isn't home, I need access to the garage door to let him in. Previously, HA has been more reliable than the MyQ phone app - but no longer. Now I'm stuck with whether or not I can get the app to work properly - sketchy at best. This is becoming a safety issue for my family (not merely convenience). I fear that the entire industry is going to go this way, which will destroy our HA community.
I'm sorry to hear that. If it's that critical for you then I would suggest reviewing the thread and going with one of the mentioned local only options. It's probably better for you to prepare well ahead of time instead of MyQ completely pulling the plug and you scrambling because the integration is permanently broken. No idea if that'll ever happen, but I'd personally prepare for that potential.
@drysart thank you for the updates that allow me to play along in this game of whack-a-mole.
I was late to yesterday's workaround. I made the api.py changes that you outlined, but as you mention here: https://github.com/home-assistant/core/issues/101763#issuecomment-1756494337 things have changed and now I am getting the 403 errors.
Blackstone bought Chaimberland a couple years ago. Anything they buy turns into a paywalled pile of useless dog crap.
@dmperl Have you tried regionalizing the endpoint? As @drysart mentioned in his post that he got working with C#, he was able to get it working by regionalizing to the -east
endpoint. I have a similar Go package for MyQ that I was able to get working (for now) by also regionalizing to -east
.
edit: I suspect this is why the homebrige API is still working btw, they recently implemented regionalized retry logic, so they may also be failing on the initial request, but then when they regionlize to -east
on the retry, it goes through. Just a guess.
edit2: Of course as soon as I say that, my regionalized updates stopped working as well. They may have just gotten around to updating that region. Back to square 1. @drysart I'd be curious if your C# login flow is still working.
I am having the same issue as well. I recently hooked an ESP32 up to my garage door opener and added 2 reed switches to the garage door so I can control it locally. Very glad I did it.
I think I'm gonna do the same thing. This arbitrary up-and-down is frustrating. I hate being reliant on what seems to be a pretty incomplete server implementation....unless you only use their tools.
NOTE: It failed in the app also. I had to restart my device then it worked, but when I tried to use my user/password (to see if that was the problem) it would not recognize the old password. I did a change password and now their app is working again, but HA is not.
Will this be pushed to trunk and available as standard update (relatively) soon? I'm not sure I have the time, and I'm quite sure not the familiarity, to accomplish this without likely screwing it up worse. :)
@brchri
I "regionalized" as follows: const.py ... added -east OAUTH_BASE_URI = "https://partner-identity-east.myq-cloud.com"
then I got the follow error messages at initialization time:
Logger: homeassistant.config_entries Source: components/myq/init.py:42 First occurred: 11:16:56 AM (1 occurrences) Last logged: 11:16:56 AM
Error setting up entry for myq Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/config_entries.py", line 388, in async_setup result = await component.async_setup_entry(hass, self) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/homeassistant/homeassistant/components/myq/init.py", line 42, in async_setup_entry myq = await pymyq.login(conf[CONF_USERNAME], conf[CONF_PASSWORD], websession) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pymyq/api.py", line 594, in login await api.authenticate(wait=True) File "/usr/local/lib/python3.11/site-packages/pymyq/api.py", line 480, in authenticate await self._authentication_task File "/usr/local/lib/python3.11/site-packages/pymyq/api.py", line 493, in _authenticate token, expires = await self._oauth_authenticate() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/pymyq/api.py", line 421, in _oauth_authenticate "code": parse_qs(urlsplit(redirect_url).query).get("code", "")[0],
IndexError: string index out of range
Restored const.py to
OAUTH_BASE_URI = "https://partner-identity.myq-cloud.com"
(now I'm back to the 403 errors. (the only change I made was adding -east as shown above. honest.)
edit 1: now I see that regionalization stopped working for you...
There is an example.py in the root of the Python-MyQ project, that can be used to test any changes without HA. I will try to take a look later and see if I can implement the changes described from @drysart
There is an example.py in the root of the Python-MyQ project, that can be used to test any changes without HA. I will try to take a look later and see if I can implement the changes described from @drysart
https://github.com/Python-MyQ/Python-MyQ/tree/auth_fixes
This branch may be a good starting point for you - it was what I was able to do on my lunch break, but I'm timed out so I can't test it further.
I did add domain cycling in the past, but I made it rather limited. I moved it to request so that it is now global for all requests that are made.
I'm timed out right now, so I can't further test it and I'm almost done w/ my lunch so I need to get back to work
cc @jiffyjaff as well
edit2: Of course as soon as I say that, my regionalized updates stopped working as well. They may have just gotten around to updating that region. Back to square 1. @drysart I'd be curious if your C# login flow is still working.
Nope, the C# code no longer works for login.
@Lash-L I'm not sure region domain cycling is going to fix the problem anymore, it looks like they may have updated the east and west regions to the newer code that doesn't let us in. The homebridge myq-api no longer works either.
I have a feeling this is going to boil down to someone doing some new captures from the iOS or Android apps to see what's required. I spent a little bit of time on it yesterday but I don't have the tools to do it right so I was limited to just looking at decompiled obfuscated code from the Android app and trying to make sense of it, and that only goes so far. It does look like they're sending additional HTTP headers to the get token endpoint (MyQApplicationId
, BrandId
, App-Version
, and they do send a User-Agent
header); but I couldn't find the values they're passing. I wouldn't be surprised if they're checking those headers now.
somebody with more knowledge than me might want to take a look at this repository. it's the base for the node red package, but might give some clues as to what will work.
That's the same api-myq code that homebridge uses; that code is no longer working for me in a test harness. Is the node red package still working? (My expectation is that it's also broken now.)
That's the same api-myq code that homebridge uses; that code is no longer working for me in a test harness. Is the node red package still working? (My expectation is that it's also broken now.)
Well, it has quit working now as well. was working just 5 minutes before i posted that message.
That's the same api-myq code that homebridge uses; that code is no longer working for me in a test harness. Is the node red package still working? (My expectation is that it's also broken now.)
Well, it has quit working now as well. was working just 5 minutes before i posted that message.
stopped for me too! ugh
FWIW I collected ios calls to myq using the charles proxy app. I'm not sure if it helps much. I redacted the cookie and response code but I'll include them in a DM to a maintainer if need-be
Login Call:
HTTP/1.1 302 Found Date: Wed, 11 Oct 2023 17:11:52 GMT Content-Length: 0 Connection: keep-alive Cache-Control: no-cache pragma: no-cache expires: Thu, 01 Jan 1970 00:00:00 GMT location: /connect/authorize/callback?redirect_uri=com.myqops%3A%2F%2Fios&code_challenge=[REDACTED]&scope=MyQ_Residential%20offline_access&client_id=IOS_CGI_MYQ&acr_values=unified_flow%3Av1%20brand%3Amyq%20app_version%3A5.242.0.38913&code_challenge_method=S256&prompt=login&response_type=code&ui_locales=en-US&suppressed_prompt=login set-cookie: idsrv=[REDACTED COOKIE TOKEN]; path=/; secure; samesite=none; httponly request-context: appId=cid-v1:0e843709-4fd8-4c8a-a52a-7eb45ba4930b x-xss-protection: 1; mode=block myq-correlationid: fb95fc49-9db0-44d3-a18e-966494621d24 strict-transport-security: max-age=15724800; includeSubDomains strict-transport-security: max-age=63072000 CF-Cache-Status: DYNAMIC Server: cloudflare CF-RAY: 8148b98af9b253bb-ATL alt-svc: h3=":443"; ma=86400
Callback Call:
HTTP/1.1 302 Found Date: Wed, 11 Oct 2023 17:11:53 GMT Content-Length: 0 Connection: keep-alive location: com.myqops://ios?code=[REDACTED RESPONSE CODE]&scope=MyQ_Residential%20offline_access&iss=https%3A%2F%2Fpartner-identity.myq-cloud.com Cache-Control: no-store, no-cache, max-age=0 pragma: no-cache request-context: appId=cid-v1:0e843709-4fd8-4c8a-a52a-7eb45ba4930b x-xss-protection: 1; mode=block myq-correlationid: b695cbe2-70f3-4f0f-ac1e-ebd041ee09ff strict-transport-security: max-age=15724800; includeSubDomains strict-transport-security: max-age=63072000 CF-Cache-Status: DYNAMIC Server: cloudflare CF-RAY: 8148b990daaa53bb-ATL alt-svc: h3=":443"; ma=86400
@jermanoid Do you have the final call to the /connect/token endpoint? Particularly, the request body + headers. You can redact the code and code_verifier values and cookie values.
Fixed by updating api.py so that the call to get the OAuth token more closely represents a valid OAuth token call. Changes highlighted in yellow:
- Add an authorization header to specify the client ID (base64 encoded value for
IOS_CGI_MYQ:
)- Add an Accept header with
*/*
- Remove the unnecessary
client_id
,client_secret
, andscope
form fields since they're not supposed to be sent on the get token call in the first place- Added
[0]
to the line extracting thecode
form field to get the code as a string rather than as an arrayAs text:
resp, data = await self.request( returns="json", method="post", url=OAUTH_TOKEN_URI, websession=session, headers={ "Authorization": "Basic SU9TX0NHSV9NWVE6", "Content-Type": "application/x-www-form-urlencoded", "Accept": "*/*", }, data={ # "client_id": OAUTH_CLIENT_ID, # "client_secret": OAUTH_CLIENT_SECRET, "code": parse_qs(urlsplit(redirect_url).query).get("code", "")[0], "code_verifier": self._code_verifier, "grant_type": "authorization_code", "redirect_uri": OAUTH_REDIRECT_URI # "scope": parse_qs(urlsplit(redirect_url).query).get( # "code", "MyQ_Residential offline_access" # ), }, login_request=True, )
where can I find this api.py file?
where can I find this api.py file?
Don't bother, it doesn't work anymore. Furthermore, the Homebridge and Node-Red integrations are also now failing.
@jermanoid Do you have the final call to the /connect/token endpoint? Particularly, the request body + headers. You can redact the code and code_verifier values and cookie values.
In attempting to grab the same (Charles Proxy via IOS app), I'm also seeing that the app is now enforcing cooldowns simply on the https://partner-identity.myq-cloud.com/connect/authorize
GET request (the request is from a fresh install of the app, so it has no creds to restrict cooldowns off of; maybe blanket issuing cooldowns by IP now?). I thought that was interesting enough to mention. They're getting aggressive about locking this API down on their end.
I'll attempt a login and publish the redacted headers/cookies/etc once my cooldown expires.
I'm failing to get a good connect/token response. I was trying using a desktop proxy that had some more advanced upstream SSL options, but not having any luck. Yeah @justinlindh, good note, i've gotten rate limited a couple times as well.
My Homebridge wasn't showing errors, but after restarting I am getting API errors
@eyeseesharp
See: https://github.com/home-assistant/core/issues/99947#issuecomment-1712728640
Steps one through 6 will get you to the correct directory: /usr/local/lib/python3.11/site-packages/pymyq
... but as @drothenberger has mentioned, there is no workaround to implement at this time.
@eyeseesharp
See: #99947 (comment)
Steps one through 6 will get you to the correct directory: /usr/local/lib/python3.11/site-packages/pymyq
... but as @drothenberger has mentioned, there is no workaround to implement at this time.
yep i see that now :( ah well, thanks anyway! keeping my eye out here in case something else can be done
I'm failing to get a good connect/token response. I was trying using a desktop proxy that had some more advanced upstream SSL options, but not having any luck. Yeah @justinlindh, good note, i've gotten rate limited a couple times as well.
Yeah, I'm getting SSL handshake errors on the login request when using Charles (even with root CA installed/enabled; SSL requests to login page capture fine, but the CONNECT
request fails). I assume they're using SSL certificate pinning (either their servers or CloudFlare).
FWIW myQ is still working for me through home bridge, I did disable my HA plugin as soon as it stopped working and I saw this issue raised
Their Android app, at least, does certificate pinning; so I assume the iOS app does too.
As an alternate perspective:
I'm still using the previous version of the myq library (without the useragent fix) in an older home assistant version. I have made no changes and not yet upgraded. It had been broken for a while, then it mysteriously started working several days ago, and then stopped working again yesterday. So I think whatever happened is much more likely something on the myq cloud side than related to "when I installed x it broke".
I'm failing to get a good connect/token response. I was trying using a desktop proxy that had some more advanced upstream SSL options, but not having any luck. Yeah @justinlindh, good note, i've gotten rate limited a couple times as well.
Yeah, I'm getting SSL handshake errors on the login request when using Charles (even with root CA installed/enabled; SSL requests to login page capture fine, but the
CONNECT
request fails). I assume they're using SSL certificate pinning (either their servers or CloudFlare).
yeah more than likely need to use a tool like https://github.com/mitmproxy/android-unpinner to unpin the apk
Reverse engineering Android apps isn't in my wheelhouse but no better time than the present to learn something new. I'm taking a stab at it, but don't let that discourage anyone who actually already knows how to do this to do it instead.
Reverse engineering Android apps isn't in my wheelhouse but no better time than the present to learn something new. I'm taking a stab at it, but don't let that discourage anyone who actually already knows how to do this to do it instead.
It is simultaneously a super annoying thing and a super fun thing. Feel free to message me on discord (Same username as here in HA discord) if you have any questions or if I can help you in anyway. Like I said I don't have a ton of time right now to really dive into things, but I have enough time to answer questions
Reverse engineering Android apps isn't in my wheelhouse but no better time than the present to learn something new. I'm taking a stab at it, but don't let that discourage anyone who actually already knows how to do this to do it instead.
It is simultaneously a super annoying thing and a super fun thing. Feel free to message me on discord (Same username as here in HA discord) if you have any questions or if I can help you in anyway. Like I said I don't have a ton of time right now to really dive into things, but I have enough time to answer questions
If you want to have some real fun and stick it to MyQ, we could try and spoof a MyQ enable garage door using an ESP32 or something and trick MyQ into thinking the ESP32 is a real garage door opener and then hook that up to a regular GDO relay so you can get the features of Amazon Key Delivery and that new Walmart delivery feature.
I was finally able to intercept the token call. So a couple of things to note about the current successful auth flow via (Android) app:
/connect/authorize
returns a 302 to Location: /Account/Login
, which when followed returns a 200
. I believe the subsequent form submission used to call to this /Account/Login
path, but now it's posting to /Account/LoginWithEmail
/connect/token
POST call:
code=AC0958D...<redacted>
grant_type=authorization_code
scope=MyQ_Residential offline_access
redirect_uri=com.myqops://android
client_id=ANDROID_CGI_MYQ
code_verifier=OMNfhq...<redacted>
Note that the client_secret
is not present in this body. Also, the following request header was added:
MyQApplicationID: D9D7B...<redacted>
idk if that's significant or sensitive.
So far I'm still getting a 401 on the /connect/token
call despite setting the same parameters, so I'm still troubleshooting, but thought this info might help someone else that's also looking into it.
Note: I am not debugging against the pymyq package, so some of my assumptions about how that package behaves may be a bit off.
Has anybody tried the IFTTT trick to get around this? I have no idea if it will work, nor how it would recognize the devices as Cover's, but the myQ site says they support IFTTT.
I have 3 garage doors that are now disabled from HA. Argh!
Okay I got it working. The key I think is adding the following params to the login call:
Brand: myq
UnifiedFlowRequested: True
ReturnUrl: <dynamic return URL from oauth call>
Also changing the login POST call from /Account/Login
to /Account/LoginWithEmail` is probably key. I'll post a link to my updated Go package for reference when I'm finished with it. Shouldn't be too far out, assuming it doesn't break again.
As soon as I get back to my PC I'll give those headers a try. To be clear, those are on the post to the LoginWithEmail endpoint and not to /connect/token or /connect/authorize?
Correct, and for further clarity, they are not headers, they are body params.
Body params? Interesting, I'd expect that would be a form post corresponding to the login page's HTML form, and the HTML doesn't specify those fields. I wonder if the app is injecting them into the post separately.
Here is what it looks like from Android app:
Headers:
:scheme: https
:method: POST
:path: /Account/LoginWithEmail?returnUrl=%2Fconnect%2Fauthorize%2Fcallback%3Fclient_id%3DANDROID_CGI_MYQ%26scope%3DMyQ_Residential%2520offline_access%26response_type%3Dcode%26redirect_uri%3Dcom.myqops%253A%252F%252Fandroid%26code_challenge%3DKiuQLijmUlk_d5P2-qpmMQa67uKGGxvfTOwJEOAl0vo%26code_challenge_method%3DS256%26ui_locales%3Den-US%26acr_values%3Dunified_flow%253Av1%2520%2520brand%253Amyq%26prompt%3Dlogin%26suppressed_prompt%3Dlogin
:authority: partner-identity.myq-cloud.com
cookie: myqidstid=xxcookie: __cf_bm=hoO
cookie: .AspNetCore.Culture=c%3Den-US%7Cuic%3Den-US
cookie: AntiForgery.MyQ.Identity=CfDJ
cookie: _cfuvid=f.cmmYroxyuFEPTiWc5BIB_dH
cookie: _ga=GA1.1.183427189.169
cookie: idsrv.session=A3977EBD7ABD00
cookie: utag_main=v_id:018b20d7770a001641ca361974cf00076
cookie: _ga_02SSCWB8G4=GS1.1.1697063076.1
content-length: 723
cache-control: max-age=0
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="101", "Google Chrome";v="101"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Android"
upgrade-insecure-requests: 1
origin: null
content-type: application/x-www-form-urlencoded
user-agent: Mozilla/5.0 (Linux; Android 7.1.2; Touchpad) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Safari/537.36
accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
sec-fetch-site: same-origin
sec-fetch-mode: navigate
sec-fetch-user: ?1
sec-fetch-dest: document
accept-encoding: gzip, deflate, br
accept-language: en-US,en;q=0.9
Request Body:
ReturnUrl=%2Fconnect%2Fauthorize%2Fcallback%3Fclient_id%3DANDROID_CGI_MYQ%26scope%3DMyQ_Residential%2520offline_access%26response_type%3Dcode%26redirect_uri%3Dcom.myqops%253A%252F%252Fandroid%26code_challenge%3DKiuQLijmUlk_d5P2-qpmMQa67uKGGxvfTOwJEOAl0vo%26code_challenge_method%3DS256%26ui_locales%3Den-US%26acr_values%3Dunified_flow%253Av1%2520%2520brand%253Amyq%26prompt%3Dlogin%26suppressed_prompt%3Dlogin
Brand=myq
UnifiedFlowRequested=True
Email=xx
Password=xx
__RequestVerificationToken=CfDxx
Right, it's a POST with the body as the populated form fields, via hidden input params. Here's my full list of changes. I just hit my 429 throttling so I can't test further atm. 😒
Ok, verified -- the login code as it existed in Homebridge's api_myq (and in my C# gist from earlier) works with three changes:
Brand
and UnifiedFlowRequested
), into the login POST even though they're not in the HTMLReturnUrl
argument is in the HTML as a hidden field, but the code was previously written to exclude that from the login POST, so it was just a matter of no longer excluding itIOS_CGI_MYQ
to ANDROID_CGI_MYQ
; and the callback url from com.myqops://ios
to com.myqops://android
.Updated gist of C# code is here. Since it's so close to the existing code this should be straightforward to integrate into python_myq too.
Can confirm that I've got this working after adding the changes that @brchri and @drysart mentioned above.
My changes are as follows (I can submit a PR later, if nobody else beats me to it):
const.py
:
OAUTH_CLIENT_ID = "ANDROID_CGI_MYQ"
OAUTH_REDIRECT_URI = "com.myqops://android"
api.py
data.update({"brand": "myq"})
data.update({"UnifiedFlowRequested": "True"})
...
url=f"{OAUTH_BASE_URI}/Account/LoginWithEmail",
I can work on a proper PR later tonight, if nobody beats me to it (feel free to).
think you meant OAUTH_REDIRECT_URI = "com.myqops://android"
I did, sorry. Edited.
can someone post where the lines in the api.py go so the rest of us can test this out please?
ANDROID_CGI_MYQ
was your starting point before or after @drysart's most recent patch?
can someone post where the lines in the api.py go so the rest of us can test this out please?
was in the process of asking the same. there's a lot going on in api.py and i found several lines with data.update calls.
can someone post where the lines in the api.py go so the rest of us can test this out please?
Also is this alone enough to fix or do we also need the header/token changes mentioned previously that (temporarily) fixed things?
The problem
HA 2023.10.1 worked ok for a few days but now similar failure when trying a reload. See error log below
What version of Home Assistant Core has the issue?
2023.10.1
What was the last working version of Home Assistant Core?
No response
What type of installation are you running?
Home Assistant OS
Integration causing the issue
MyQ
Link to integration documentation on our website
No response
Diagnostics information
Logger: homeassistant.components.myq Source: helpers/update_coordinator.py:322 Integration: MyQ (documentation, issues) First occurred: 10:19:24 AM (1 occurrences) Last logged: 10:19:24 AM
Error fetching myq devices data: Error trying to re-authenticate to myQ service: Error requesting data from https://partner-identity.myq-cloud.com/connect/token: 401 - Unauthorized
Example YAML snippet
No response
Anything in the logs that might be useful for us?
Additional information
No response