Recent User-Agent Issues

[dereklucas]

Reading the recent pymyq fixes (#39 #40 #42) it seems clear that MyQ is blocklisting user agents. Initially, the user agent was a default from Python, then to something pulled from mobile Safari, then to a user agent pulled from the iOS app. Someone found that the Android app still uses the default string from the http library they use, but since they've shown a willingness to block one of their own user agents, I don't think we can simply continue pulling from new apps. Instead, we should either use a random string or myQ/[random number] CFNetwork/1107.1 Darwin/19.0.0 possibly randomized at a regular interval.

Unfortunately, if they view this as a problem eventually they will switch from a blocklist to a safelist and then we'll be sunk.

[dseven]

Hmmm. I had assumed that they were already whitelisting, but if appears that you are right - I tried a made up string, and it was allowed. I wonder if they're blacklisting as a means to force users to keep their apps updated, or as a way to keep people like us out......

[ghost]

I guess the risk is they can easily find this project and block whatever gets put in. I could understand if they still charged for premium integration subscriptions but it is a little weird they are doing it.....

[dereklucas]

I suspect that they either have already found this project, or they are looking at usage numbers and blocking anything that looks abnormally high. A randomly generated string that looks similar to the iOS app would keep us off of a high usage list and would probably require a safelist as a next step.

Using the Android user agent might also be a good option since in my experience it is more difficult to get all android users to update quickly. Also since they’re using a default string, it doesn’t look obvious that we’re using it.

It all depends on how likely it is that they’re intentionally targeting this project versus one person blocking things that look off.

[kyotodesertfox]

This product (MyQ) appears to be getting to a point now where this is no longer reliable to use anymore; I realize this is not a problem with HASS, etc, rather the dev's of MyQ.

Are there other garage door solutions that work well with Home Assistant, as I am on the verge of changing products. The amount of times this broke in the past month or so is just absurd, and honestly getting unacceptable. I know that we are doing as much as we can to keep up, but MyQ is just being d's now.

I cannot use the MyQ app either, as it constantly tells me the Sensor is broken; yet it has worked just fine using HASS, instead of the native app (which is amusing to me).

Sooo.....anyone know of any alternatives?

[rct]

1. I think making pymyq as indistinguishable from the Android app as possible is a good strategy as @dereklucas says, "it is more difficult to get all android users to update quickly". I guess at that point we will see if they are willing to force Android users to upgrade by cutting off access from the older apps.

2. How difficult would it be to allow the user agent to be overridden from Home Assistant config files instead of requiring a code change? (This might help until the "arms race" accelerates beyond user agent validation.

3. Anyway to judge how big the population of pymyq users is? Is it only used by Hass or do other HA platforms also use it? If I had known what the situation is, I never would have purchased anything from Chamberlain/Liftmaster.

[kyotodesertfox]

Anyway to judge how big the population of pymyq users is?

I wonder if HA Dev's can see these numbers on their end. Surely they monitor usage metrics like this?

If I had known what the situation is, I never would have purchased anything from Chamberlain/Liftmaster.

More true words were rarely spoken. I feel the same way.

[brbeaird]

Are you guys 100% sure user agent is required? I've been using my own integration for awhile now that does not send user agent at all.

[dereklucas]

@brbeaird It is not required. But, Python sets a default user agent. I’m not familiar enough with it to know how easily that can be disabled. But, if they’re willing to block specific user agents, it’s probably not significantly more work to block no user agent.

[brbeaird]

I do not think they're explicitly blocking certain agents. I think it is more that they use the agent string as a way to force clients to stay upgraded, and it seems if the agent isn't there, that logic is bypassed (for now at least).

[dseven]

Interesting. With no User-Agent specified in the headers, one of the form Python/3.7 aiohttp/3.6.2 gets added by default. This can be turned off by adding skip_auto_headers={'USER-AGENT'} to the request() call. With that in place, it does seem to be working (at least today).

[kyotodesertfox]

Wouldn't mind a PR ;-)

[dseven]

I can do a PR for the skip_auto_headers thing if there's a consensus that it's a good approach .. I guess I could put it up for review anyway....

[dseven]

master should be working currently

[brbeaird]

Well now I'm having issues again connecting with whatever I try. Anyone else seeing this?

[kyotodesertfox]

At the moment mine is currently Connected.

Unsure if there is a reconnection cycle or not to force a new session, but I'm up as of this writing.

I'm not going to restart HASS either in case I do lose it.

[dseven]

Still working for me with the current master branch ("okhttp/3.10.0") and with the above change (header disabled completely).

[dseven]

I've been using this to test basic connectivity - it's basically copied from the README. It should work in a homeassistant environment, if you can get a shell - just run it with python3

import asyncio
from aiohttp import ClientSession
import pymyq

async def main() -> None:
    async with ClientSession() as websession:
        myq = await pymyq.login('myqdevice@mydomain.com', 'xxx', websession)
        devices = myq.devices
        for dev, info in devices.items():
            print('%(device_family)s: %(name)s' % info.device_json)

asyncio.get_event_loop().run_until_complete(main())

[brbeaird]

That works. I was trying to replicate manually in Postman but still can't login that way. I wonder what the difference is.

On Fri, Jul 3, 2020 at 8:49 AM iain MacDonnell notifications@github.com wrote:

I've been using this to test basic connectivity - it's basically copied from the README. It should work in a homeassistant environment, if you can get a shell - just run it with python3

import asyncio from aiohttp import ClientSession import pymyq

async def main() -> None: async with ClientSession() as websession: myq = await pymyq.login('myqdevice@mydomain.com', 'xxx', websession) devices = myq.devices for dev, info in devices.items(): print('%(device_family)s: %(name)s' % info.device_json)

asyncio.get_event_loop().run_until_complete(main())

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/arraylabs/pymyq/issues/46#issuecomment-653557463, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXHM2CQKJLZFNI4WNS3SU3RZXOU7ANCNFSM4OKKYGYA .

[brbeaird]

Figured it out. Mostly. The "Username" and "Password" attributes are case-sensitive now and begin with capital letters. I still can't get it to work directly in Postman for some reason but was successful via other methods. Either way, I'm good for now. Thanks!

[codypet]

So is Myq working in 0.112.2 and you are working on a more permanent fix, or is 0.112.2 still have MyQ down? I was waiting to upgrade, but might just go ahead.

[kyotodesertfox]

It seems to be working under master branch so yeah I would give it a go if you can.

I'm on the latest and I'm working fine.

[dseven]

Home Assistant 0.112.0 has a requirement for pymyq 2.0.5, so it should be good (until something else changes!)