"Alexa Media Player failed to login" when adding integration for amazon.co.uk: `Not logged in: An exception of type ContentTypeError occurred`

alandtse / alexa_media_player

This is a custom component to allow control of Amazon Alexa devices in Home Assistant using the unofficial Alexa API.

Apache License 2.0

1.51k stars 289 forks source link

"Alexa Media Player failed to login" when adding integration for amazon.co.uk: `Not logged in: An exception of type ContentTypeError occurred` #2469

Open deosrc opened 2 months ago

deosrc commented 2 months ago

IMPORTANT: Please search the issues, including closed issues, and the FAQ before opening a new issue. The template is mandatory; failure to use it will result in issue closure.

Describe the bug

When adding the AMP integration using the amazon.co.uk region, login to amazon is successful but then the integration states:

To Reproduce

Remove the AMP integration
Re-add the AMP integration setting the amazon region to amazon.co.uk
Login to amazon through the proxy page
Once the proxy page closes, the error is displayed

Expected behavior

Should be able to add the integration for the amazon.co.uk region.

Screenshots

System details

Home-assistant (version): 2024.8.1
alexa_media (version from const.py or HA startup): 4.12.7
alexapy (version from pip show alexapy or HA startup): Not sure how to find this in HAOS. Doesn't seem to be in the logs?
Amazon 2FA is enabled (y/n). : y

Logs Please provide logs.

Apologies if this is excessively redacted.

2024-08-18 16:17:20.261 DEBUG (MainThread) [alexapy.alexalogin] 5 cookies successfully exchanged for refresh token for domain .amazon.co.uk
2024-08-18 16:17:20.263 DEBUG (MainThread) [alexapy.alexalogin] Attempting to discover CSRF token
2024-08-18 16:17:20.383 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /spa/index.html
2024-08-18 16:17:20.626 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /api/devices-v2/device?cached=false
2024-08-18 16:17:20.684 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /templates/oobe/d-device-pick.handlebars
2024-08-18 16:17:20.728 DEBUG (MainThread) [alexapy.alexalogin] No csrf token found

2024-08-18 16:17:21.124 DEBUG (MainThread) [alexapy.alexalogin] GET: 
https://alexa.amazon.com/api/bootstrap returned 200:OK with response <CIMultiDictProxy('Content-Type': 'application/json', 'Content-Length': '172', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Sun, 18 Aug 2024 15:17:21 GMT', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'x-amz-rid': '29EZT64H7XHEE5M0K0ER', 'Set-Cookie': 'csrf=-701784766; expires=Fri, 18-Aug-2034 15:17:20 GMT; Domain=.amazon.com; Path=/', 'x-amzn-RequestId': '29EZT64H7XHEE5M0K0ER', 'x-amzn-alt-domain': 'https://alexa.amazon.co.uk/logout', 'X-Cache': 'Miss from cloudfront', 'Via': '1.1 ce6dd6cf8e3b2bf695c066e441ad568e.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'LHR61-P6', 'X-Amz-Cf-Id': 'l0vWnUXdmXysduXb9GVKjiCRjCUb0_4RDTxQiw8OZTSXDrilcU50bA==')>

2024-08-18 16:17:21.167 DEBUG (MainThread) [alexapy.alexalogin] GET: 
https://alexa.amazon.co.uk/api/bootstrap returned 401:Unauthorized with response <CIMultiDictProxy('Content-Length': '0', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Sun, 18 Aug 2024 15:17:21 GMT', 'Strict-Transport-Security': 'max-age=0; includeSubDomains;', 'x-amz-rid': 'GS2055T72KC6WFXSRBD8', 'loginUrl': 'https://www.amazon.co.uk/ap/signin?showRmrMe=1&openid.return_to=https%3A%2F%2Falexa.amazon.co.uk%2Flogin&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=amzn_dp_project_dee_uk&openid.mode=checkid_setup&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&', 'x-amzn-RequestId': 'GS2055T72KC6WFXSRBD8', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 6cd1874959ee48e26855209aa18a4014.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'LHR50-P7', 'X-Amz-Cf-Id': 'a_kCzoW7gXbKbd_13B1UiHWlFsEnNgovCnc9TFi6p1N7mxi0Kq7pjw==')>

2024-08-18 16:17:21.168 DEBUG (MainThread) [alexapy.alexalogin] Not logged in: An exception of type ContentTypeError occurred. Arguments:
(RequestInfo(url=URL('https://alexa.amazon.co.uk/api/bootstrap'), method='GET', headers=<CIMultiDictProxy('Host': 'alexa.amazon.co.uk', 'User-Agent': 'AmazonWebView/Amazon Alexa/2.2.556530.0/iOS/16.6/iPhone', 'Accept': '*/*', 'Accept-Language': '*', 'DNT': '1', 'Upgrade-Insecure-Requests': '1', 'Accept-Encoding': 'gzip, deflate, br', 'Cookie': 'at-main=<REDATED>; lc-main=en_GB; sess-at-main="\\"<REDACTED>\\""; session-id=258-2185812-7023702; session-id-time=2354714240l; session-token=<REDACTED>; ubid-acbuk=130-0156096-0538242; ubid-main=130-0156096-0538242; x-main=<REDACTED>')>, real_url=URL('https://alexa.amazon.co.uk/api/bootstrap')), ())

Additional context

I have found a partial workaround to add the amazon.co.uk config entry, but it does not result in a functioning AMP integration. Instead, the integration prompts to be reconfigured (possibly one of the other issues such as #2453), and has some strange behaviour possibly as a result of the same account being registered in two config entries (conflicting pickle file?)

The workaround to add the config entry:

Add the AMP integration but using the amazon.com region.
Once logged in to amazon, the AMP integration will be added, but will fail to start.
Click the 3 dot menu for the failing entry and select "System options"
Turn off both options and save:
Click "Add entry" and complete the setup this time for amazon.co.uk region
The integration should add successfully, but will then prompt to be re-configured:
~~Remove the amazon.com entry.~~

Edit: Removing the entry seemingly removes both the amazon.com and amazon.co.uk entries. Possibly a conflicting ID (email?) causing some strange behaviour.

danielbrunt57 commented 2 months ago

Could you try this:

Delete the integration's config entry
Delete /config/,storage/alexa_media..pickle
In HACS, delete the custom_component and restart HA
In HACS, install Alexa Media Player custom component v4.12.8
Restart HA
in ssh/terminal session:
- docker exec -it homeassistant bash
- cd /usr/local/lib/python3.12/http
- replace cookies.py with cookies.zip
- cd ../site-packages/alexapy
- replace alexalogin.py with alexalogin.zip
- This adds alexapy merge request !380 which contains the corresponding async def delete_cookiefile(self) which AMP 4.12.8 will call to delete the pickle file when removing the config entry.
Restart HA
Clear browser cookies & cached files
Try adding the integration with:
- Seconds between scans 360 - with http2push active, this effectively becomes 1 hour (3600 seconds)
- For now, do not enable Include devices connected via Echo
Verify at this point that the integration is successfully configured
Three dots System options
- Disable Enable newly added entities but leave Enable polling for updates enabled

deosrc commented 2 months ago

@danielbrunt57

I have:

Removed all AMP config entries.
Checked /config/.storage, pickle file was not present
Updated AMP to 4.12.8 and restarted
Installed and configured advanced SSH addon to gain access to docker in HassOS
Replaced the contents of http/cookies.py and site-packages/alexapy/alexalogin.py within the homeassistant docker container.
Restarted HA
Launched chromium and cleared all browsing data, then restarted the browser.
Login to HA using IP address (to avoid any differences between default URL in AMP and alternate URLs such as homeassistant.local)
Attempt to add AMP integration using amazon.co.uk region and 360 seconds between scans
- Same result after Amazon login and OTP entry. Failed to login.
Attempt workaround which still seems to be working.

I forgot to re-enable debug logging when first trying this, but enabling it afterwards and attempting to add the AMP integration yielded mostly the same 401 Unauthorized log messages.

I'm not sure if there are more URLs now showing 401 or if I just missed these before. It looks like the same log section so perhaps related to the changes:

2024-08-18 21:35:31.988 DEBUG (MainThread) [alexapy.alexalogin] 5 cookies successfully exchanged for refresh token for domain .amazon.co.uk
2024-08-18 21:35:31.990 DEBUG (MainThread) [alexapy.alexalogin] Attempting to discover CSRF token
2024-08-18 21:35:32.100 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /spa/index.html
2024-08-18 21:35:32.178 DEBUG (MainThread) [alexapy.alexalogin] Unable to load page for csrf: <ClientResponse(https://alexa.amazon.co.uk/api/language) [401 Unauthorized]>
<CIMultiDictProxy('Content-Length': '0', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Sun, 18 Aug 2024 20:35:32 GMT', 'x-amz-rid': 'WRNKE3AZMZ9DXJHAX25T', 'loginUrl': 'https://www.amazon.co.uk/ap/signin?showRmrMe=1&openid.return_to=https%3A%2F%2Falexa.amazon.co.uk%2Flogin&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=amzn_dp_project_dee_uk&openid.mode=checkid_setup&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&', 'x-amzn-RequestId': 'WRNKE3AZMZ9DXJHAX25T', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'Strict-Transport-Security': 'max-age=0; includeSubDomains;', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 3d8e500d44b557879a1086daf1dc3aaa.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'LHR50-P7', 'X-Amz-Cf-Id': 'HZXUb3Z6ORWiaDjGmhw8Zv8WJkPGC8h5tHRw_AzLcKlr13nXlC_qrA==')>

2024-08-18 21:35:32.294 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /api/devices-v2/device?cached=false
2024-08-18 21:35:32.354 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /templates/oobe/d-device-pick.handlebars
2024-08-18 21:35:32.398 DEBUG (MainThread) [alexapy.alexalogin] Unable to load page for csrf: <ClientResponse(https://alexa.amazon.co.uk/api/strings) [401 Unauthorized]>
<CIMultiDictProxy('Content-Length': '0', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Sun, 18 Aug 2024 20:35:32 GMT', 'x-amz-rid': 'RHX33MM85WX8E01P5DYE', 'loginUrl': 'https://www.amazon.co.uk/ap/signin?showRmrMe=1&openid.return_to=https%3A%2F%2Falexa.amazon.co.uk%2Flogin&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=amzn_dp_project_dee_uk&openid.mode=checkid_setup&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&', 'x-amzn-RequestId': 'RHX33MM85WX8E01P5DYE', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'Strict-Transport-Security': 'max-age=0; includeSubDomains;', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 e852f14db89debadcfac3bde6e826efe.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'LHR50-P7', 'X-Amz-Cf-Id': 'ZZHYEbLe3Z4yrJguiY_mAIR03ftuSWd4OmLx4ppkR3AMWHh9l_Fvmw==')>

2024-08-18 21:35:32.398 DEBUG (MainThread) [alexapy.alexalogin] No csrf token found

From the details of the log message I have managed to reproduce this in RestFox (Postman equivalent). I built up the request from the log message for the request to https://alexa.amazon.co.uk/api/bootstrap which is failing and it returns the same 401 Unauthorized response with no body content. I only have to change the TLD from .co.uk to .com and it returns a 200 OK instead, just as the logs from HA indicate. There is a header X-Cache: Error from cloudfront appearing the response which may be relevant.

deosrc commented 2 months ago

@danielbrunt57 This seems promising so feel free to leave this issue as is for now and I'll keep looking into it when I get chance. Summary below for info and for my own reference. Any guidance is of course welcome.

It occurred to me that the login is a proxy, so the cookies would probably be similar to the normal amazon website. I tried https://alexa.amazon.co.uk/api/bootstrap in my browser where I am already logged in and received the correct response.

I've then transferred the browser cookies into the RestFox request I had created above, and received the 200 OK response expected from amazon.co.uk. I then started comparing the cookies from the browser request with the cookies from the log mesesages in HA. There were more in the browser so I started removing the cookies one by one that were missing from the HA request. As soon as I removed the x-acbuk cookie, the request started failing.

To double check, I returned all of the cookies and it once again returned 200 OK, I then removed only the x-acbuk cookie, and the request failed. This seems a promising leads on a potential cause.

I logged in to amazon.co.uk again to grab the set-cookie header. There doesn't look to be anything obvious to make this cookie different, but I'm not familiar enough with it to know.

set-cookie: x-acbuk=<REDACTED>; Domain=.amazon.co.uk; Expires=Mon, 18-Aug-2025 21:26:21 GMT; Path=/; Secure

:warning: For the following discussion of the logs, it should be noted that I have not reverted the changes to the python files, and have not yet restarted HA (to clear the logs) since performing various other steps adding and removing the AMP config. I can't be sure which attempt these messages were generated from. This is far from conclusive and could be a dead-end. I just wanted to post an update before signing off given this seems a promising atm.

I have 2 log messages shortly before the failing /api/bootstrap call showing the x-acbuk cookie. The one below is from "registration" which is first, it is then shortly followed by a similar one for "Exchange cookie json". The value looks to be different in each, but I need to confirm this.

(formatted for clarity and easier redaction)

2024-08-18 21:35:31.435 DEBUG (MainThread) [alexapy.alexalogin] Received registration data:
{
    "success": {
        // ...
        "tokens": {
            "website_cookies": [
                // ...
                {
                    "Path": "/",
                    "Secure": "true",
                    "Value": "\"<REDACTED>\"",
                    "Expires": "13 Aug 2044 20:35:31 GMT",
                    "Domain": ".amazon.co.uk",
                    "HttpOnly": "false",
                    "Name": "x-acbuk"
                },
                // ...
            ],
            // ...
        },
        //...
    }
}

There is another log message shortly after this which may (or may not) be related given there are 5 entries in "website_cookies".

2024-08-18 21:35:31.988 DEBUG (MainThread) [alexapy.alexalogin] 5 cookies successfully exchanged for refresh token for domain .amazon.co.uk

The code for the log message above looks to be here: https://gitlab.com/keatontaylor/alexapy/-/blob/dev/alexapy/alexalogin.py?ref_type=heads#L1187

This log message seems a little confusing as the code looks to be exchanging a refresh token for cookies, rather than cookies for a refresh token?

Between this log message and the failing api/bootstrap call are only attempts to determine the CSRF token which is never found.

danielbrunt57 commented 2 months ago

@chrisvblemos Could you step in on this since it appears you have far more knowledge re: cookies, tokens and CSFR (based on alexapy merge request !378) than I do as mine is next to nil...

danielbrunt57 commented 2 months ago

I just had this comment on an alexapy merge request (that I cancellled) from alandtse and I think it could very well relate to your issue...

Alan Tse @alandtse · 1 hour ago Maintainer The more likely cause of auth errors is the fact only like 5 cookies are saved now instead of every cookie that we found. That was the other change on that release.

chrisvblemos commented 2 months ago

I'll take a look later if I can. But I think @deosrc is on the right track. My fix to alexapy here was made based on the same observations. since he is using the latest version with this fix, further investigation is needed to find out why aiohttp is not using the x-acbuk here.

jleinenbach commented 2 months ago

amazon.de: As it works for me: Do we use the same minimum aiohttp version?

There were cookie fixes: https://github.com/aio-libs/aiohttp/releases

Esp. this for my version 3.10.0:

Implemented filter_cookies() with domain-matching and path-matching on the keys, instead of testing every single cookie.
This may break existing cookies that have been saved with CookieJar.save(). Cookies can be migrated with this script::

import pickle
with file_path.open("rb") as f:
    cookies = pickle.load(f)

morsels = [(name, m) for c in cookies.values() for name, m in c.items()]
cookies.clear()
for name, m in morsels:
    cookies[(m["domain"], m["path"].rstrip("/"))][name] = m

with file_path.open("wb") as f:
    pickle.dump(cookies, f, pickle.HIGHEST_PROTOCOL)

deosrc commented 2 months ago

@jleinenbach pip show aiohttp is showing as v3.10.3 for me. I believe it was updated in HA 2024.8.2 which I have updated to but the issue is still present for me.

Are you using amazon.de for the region? Based on the x-acbuk cookie name, this may be an amazon.co.uk specific issue. amazon.com doesn't seem to have the same problem.

I made some adjustments to the logging in alexalogin.py using the steps mentioned by @danielbrunt57 to try get some more diagnostics (adjusting some existing messages and adding output of filter_cookies("f{self._prefix}{self.url}") in various places) but I'm not seeing them take effect. I deleted the __pycache__ to be sure but it isn't being regenerated. pip show alexapy seems to indicate it is the correct location so I'm not sure what is happening.

I don't have any other version of alexalogin.py in the alexapy directory. The steps I'm using:

SSH to HassOS Advanced SSH addon
docker exec -it homeassistant bash
cd /usr/local/lib/python3.12/site-packages/alexapy
vi alexalogin.py
Edit file and then exit using :wq
Restart HA

The container shows as having restarted based on the uptime, and the changes are still there after reconnect so I'm not sure why they aren't being used. Any ideas?

Other things I've tried:

Reproducing the exchange token request in RestFox, all seems to work as expected and the x-acbuk token is one of the 5 cookies returned.

Writing a short python script (locally, not in HA container) to check filter_cookies is returning the cookie for what should be the inputs being used. Also seems to return as expected.

import aiohttp
import asyncio
from http.cookies import SimpleCookie
from yarl import URL

async def main():

    async with aiohttp.ClientSession() as session:
        raw_cookie = SimpleCookie()
        cookie_name = "x-acbuk"
        cookie_value = "\"<REDACTED>\""
        raw_cookie[cookie_name] = (
            cookie_value
            if not (
                cookie_value.startswith('"') and cookie_value.endswith('"')
            )
            # Strings are returned within quotations, strip them
            else cookie_value[1:-1]
        )
        raw_cookie[cookie_name]["domain"] = ".amazon.co.uk"
        raw_cookie[cookie_name]["path"] = "/"
        raw_cookie[cookie_name]["secure"] = True
        raw_cookie[cookie_name]["expires"] = "14 Aug 2044 15:57:18 GMT"
        raw_cookie[cookie_name]["httpOnly"] = False

        session.cookie_jar.update_cookies(raw_cookie, URL(".amazon.co.uk"))
        cookies = session.cookie_jar.filter_cookies(URL("https://alexa.amazon.co.uk/api/bootstrap"))
        print("Test") # Debugger breakpoint here...

asyncio.run(main())

If I can't get the alexalogin.py changes to take effect, next thing I can think to try is to expand the test script to a full request to /api/bootstrap and see if it uses the cookies.

danielbrunt57 commented 2 months ago

5. Replaced the contents of... and site-packages/alexapy/alexalogin.py within the homeassistant docker container.

async def delete_cookiefile(self) is now merged in alexapy 1.29.0 so you can just edit AMP's manifest.json from 1.28.2 to 1.29.0.

danielbrunt57 commented 2 months ago

My aiohttp is also 3.10.3

chrisvblemos commented 2 months ago

I don't have any other version of alexalogin.py in the alexapy directory. The steps I'm using:
1. SSH to HassOS Advanced SSH addon

2. `docker exec -it homeassistant bash`

3. `cd /usr/local/lib/python3.12/site-packages/alexapy`

4. `vi alexalogin.py`

5. Edit file and then exit using `:wq`

6. Restart HA
The container shows as having restarted based on the uptime, and the changes are still there after reconnect so I'm not sure why they aren't being used. Any ideas?

HA doesn't reinstall packages on restart. Also, making changes directly to the locally stored python packages is not recommended.

@deosrc Fork and then clone or just download the alexapy repo inside your /config path while in SSH (this folder is mapped to the homeassistant docker). Make your changes as desired. Then, attach to the docker as you did and run pip install -e /config/path/to/alexapy. Restart. Repeat these steps every time you make a change to alexapy.

emufan commented 2 months ago

I have the same (?) problems with amazon.de. At least the error messages are looking similar.

2024-08-20 10:56:50.146 DEBUG (MainThread) [alexapy.alexalogin] 5 cookies successfully exchanged for refresh token for domain .amazon.de
2024-08-20 10:56:50.147 DEBUG (MainThread) [alexapy.alexalogin] Attempting to discover CSRF token
2024-08-20 10:56:50.254 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /spa/index.html
2024-08-20 10:56:50.459 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /api/devices-v2/device?cached=false
2024-08-20 10:56:50.518 DEBUG (MainThread) [alexapy.alexalogin] CSRF token not found from /templates/oobe/d-device-pick.handlebars
2024-08-20 10:56:50.568 DEBUG (MainThread) [alexapy.alexalogin] No csrf token found
2024-08-20 10:56:50.866 DEBUG (MainThread) [alexapy.alexalogin] GET:
https://alexa.amazon.com/api/bootstrap returned 200:OK with response <CIMultiDictProxy('Content-Type': 'application/json', 'Content-Length': '171', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Tue, 20 Aug 2024 08:56:50 GMT', 'x-amz-rid': '6KKHP4FZ41X4BCB9HTB6', 'Set-Cookie': 'csrf=30060786; expires=Sun, 20-Aug-2034 08:56:50 GMT; Domain=.amazon.com; Path=/', 'x-amzn-RequestId': '6KKHP4FZ41X4BCB9HTB6', 'x-amzn-alt-domain': 'https://alexa.amazon.de/logout', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'X-Cache': 'Miss from cloudfront', 'Via': '1.1 918126cde448a82c2757fd37058ea3ec.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'HAM50-C1', 'X-Amz-Cf-Id': 'iO606tajoBCwq-Th8kkZq5uCE5wd6oikuf2eMvoQSlz_thiO78utvg==')>
2024-08-20 10:56:50.919 DEBUG (MainThread) [alexapy.alexalogin] GET:
https://alexa.amazon.de/api/bootstrap returned 401:Unauthorized with response <CIMultiDictProxy('Content-Length': '0', 'Connection': 'keep-alive', 'Server': 'Server', 'Date': 'Tue, 20 Aug 2024 08:56:50 GMT', 'Strict-Transport-Security': 'max-age=0; includeSubDomains;', 'x-amz-rid': 'Y477TQPNJ78YCCZWW972', 'loginUrl': 'https://www.amazon.de/ap/signin?showRmrMe=1&openid.return_to=https%3A%2F%2Falexa.amazon.de%2Flogin&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.assoc_handle=amzn_dp_project_dee_de&openid.mode=checkid_setup&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&', 'x-amzn-RequestId': 'Y477TQPNJ78YCCZWW972', 'Vary': 'Content-Type,Accept-Encoding,User-Agent', 'X-Cache': 'Error from cloudfront', 'Via': '1.1 eabf0052502240e2b09c2e962490cabc.cloudfront.net (CloudFront)', 'X-Amz-Cf-Pop': 'HAM50-P1', 'X-Amz-Cf-Id': '-PQKFYqt386Wb4iwH8OjLJsELQZkdVXwXQyUcQujr8-7eTfUZivdVQ==')>
2024-08-20 10:56:50.920 DEBUG (MainThread) [alexapy.alexalogin] Not logged in: An exception of type ContentTypeError occurred. Arguments:
(RequestInfo(url=URL('https://alexa.amazon.de/api/bootstrap'), method='GET', headers=<CIMultiDictProxy('Host': 'alexa.amazon.de', 'User-Agent': 'AmazonWebView/Amazon Alexa/2.2.556530.0/iOS/16.6/iPhone', 'Accept': '*/*', 'Accept-Language': '*', 'DNT': '1', 'Upgrade-Insecure-Requests': '1', 'Accept-Encoding': 'gzip, deflate, br', 'Cookie': 'at-main=foo; lc-main=de_DE; sess-at-main="\\"b9BhEQzsyFcjYE/PA63XLpaDdONUfxzIgr+6vpYGL8E=\\""; session-id=257-9085617-4147239; session-id-time=2354864210l; session-token=bar; sid="\\"NPowU5s/wYMuNgqu4a7Dfg==|Sw777im+40ufbBsvCgpED22f/VD0l8vNEXo7L+h+hAE=\\""; ubid-acbde=135-3594605-4904817; ubid-main=135-3594605-4904817; x-main="\\"XDSi40t1MtVOVBV8nTfUOpNBgcWl?PaQ\\""')>, real_url=URL('https://alexa.amazon.de/api/bootstrap')), ())

async def delete_cookiefile(self) is now merged in alexapy 1.29.0 so you can just edit AMP's manifest.json from 1.28.2 to 1.29.0.

tried this already as well.

HA 2024.8.2 AMP v4.12.10

deosrc commented 2 months ago

I'm not sure how, but AMP is working again for me. Entity updates, announcements, everything.

My container may be in a bit of a mess at the moment. As @chrisvblemos correctly points out, editing packages inside the container is not a good idea.

:warning: Before making these changes, be aware that they are in the /cofing directory so will persist across updates. If they break HA, updating/re-creating the container will likely not fix it. If you are unsure, wait for an AMP update to be pushed out.

I believe my active changes are:

Updated /config/custom_components/alexa_media/manifest.json to use a fork of alexapy with some logging changes:

"requirements": ["alexapy@git+https://gitlab.com/deosrc1/alexapy.git@bugfix/unauthorized", "packaging>=20.3", "wrapt>=1.14.0"],

Move/rename/remove the alexapy* directories from /config/deps/lib/python3.12/site-packages so that the dependency re-downloads (see additional)
Restart HA
Attempt to add the integration

There shouldn't be anything in that branch to affect the functionality other than adjusting some logging, but the integration added successfully, and everything seems to be working again. Perhaps it was some of the other recent changes which have now been applied from my steps above.

I'll try clean up my container and check to see if it is still working.

Additional

@chrisvblemos Thanks for the info. It put me on track to learn more about the internals and debugging of HA.

HA doesn't reinstall packages on restart.

I was still seeing some strange behaviour from the packages. It seems this might not be true: https://developers.home-assistant.io/docs/creating_integration_manifest/#requirements

The requirements seems to be checked when adding an integration. At one point when I had an invalid environments setup, attempting to add the AMP integration failed because it couldn't grab the pip package. There may be checks to see if the package is already installed before attempting. I haven't looked into it too closely.

Also, making changes directly to the locally stored python packages is not recommended.

Absolutely. My python knowledge and especially setting up HA container for debugging a pip package is fairly limited. I attempted setting up debugpy after this comment, but the breakpoints weren't being hit. Possibly because they are in a pip module? Even setting justMyCode to false and the path mappings didn't seem to work. It's possible I just had a configuration issue somewhere though.

I'm aware I'm making a mess of my container at the moment. It will at least destroy any (except within /config) when I update to the next version. I'll need to be careful until then.

Fork and then clone or just download the alexapy repo inside your /config path while in SSH (this folder is mapped to the homeassistant docker). Make your changes as desired. Then, attach to the docker as you did and run pip install -e /config/path/to/alexapy. Restart. Repeat these steps every time you make a change to alexapy.

:+1: This seems a much better way, unfortunately see the above about how HA does requirements differently.

chrisvblemos commented 2 months ago

@deosrc

Is there any chance you weren't using the latest alexapy version before using your fork? Not because you did something wrong, but to check if HA is not automatically updating the dependency (I don't think this is the case, but better safe than sorry).

Also, could you check if everything works if you roll back alexapy to this change? If you want to keep your way of using branches in the manifest.json, create a new branch and do the rollback. This is to see if any of the latest changes are in fact fixing this issue.

deosrc commented 2 months ago

Is there any chance you weren't using the latest alexapy version before using your fork? Not because you did something wrong, but to check if HA is not automatically updating the dependency (I don't think this is the case, but better safe than sorry).

I'm honestly not sure. The alexa media version in the initial post is definitely accurate (although I've since upgraded to .8) , but I didn't know how to check the alexapy version at the time.

Also, could you check if everything works if you roll back alexapy to this change? If you want to keep your way of using branches in the manifest.json, create a new branch and do the rollback. This is to see if any of the latest changes are in fact fixing this issue.

I think I've worked out how the dependencies work now so should be able to target a specific alexapy commit to try this out and report back.

Given my deps folder also includes some older versions of python though, I'm thinking it might be worth creating a backup of the folder, and letting HA reacquire the dependencies. I should also look at re-creating the HA container to make sure I'm working with a clean container.

alandtse commented 2 months ago

The requirements seems to be checked when adding an integration. At one point when I had an invalid environments setup, attempting to add the AMP integration failed because it couldn't grab the pip package. There may be checks to see if the package is already installed before attempting. I haven't looked into it too closely.

This is correct. HA will only check component dependencies when adding the integration. HA upgrades will wipe out all custom dependencies, so forcing an add (even if not completed), is the best way to restore dependencies. I believe that was in the wiki at some point (or should be put there).

philtarry commented 2 months ago

Same issue here with amazon.co.uk

is there a simple workaround?

arachnetech commented 2 months ago

My Alexa Media player has been failing to log in too. I've just tried deleting and re-adding the integration, and even clearing all my 2FA settings, but I still get the 'Alexa Media Player failed to log in' error. I updated to HA 2024.8.3 (I was previously on 8.1) and tried again, but I still can't make it work.

Finding this open issue, hopefully this is a general issue with the integration which will be fixed in a future update? Or is this only affecting isolated accounts?

LindsayReid commented 2 months ago

I'm pretty sure it's affecting thousands!

On Sun, 1 Sept 2024 at 11:09, David Miller @.***> wrote:

My Alexa Media player has been failing to log in too. I've just tried deleting and re-adding the integration, and even clearing all my 2FA settings, but I still get the 'Alexa Media Player failed to log in' error. I updated to HA 2024.8.3 (I was previously on 8.1) and tried again, but I still can't make it work.

Finding this open issue, hopefully this is a general issue with the integration which will be fixed in a future update? Or is this only affecting isolated accounts?

— Reply to this email directly, view it on GitHub https://github.com/alandtse/alexa_media_player/issues/2469#issuecomment-2323268298, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFWRDXL7DFW4LLQ7JKWVLDZULRUHAVCNFSM6AAAAABMWPTCBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGI3DQMRZHA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

jleinenbach commented 2 months ago

Try this:

Upgrade HA 2024.8.1 and 2024.8.2 to HA 2024.8.3 if not already done
Upgrade Python's cookie.py and check if it was successful: https://github.com/alandtse/alexa_media_player/issues/2453#issuecomment-2317105135
Install PythonScriptPro with HACS: https://github.com/AlexxIT/PythonScriptsPro/tree/master

add this to your configuration.yaml

python_script:
requirements:
- alexapy>=1.29.0
- aiohttp>=3.10.5
- aiofiles>=24.1.0

Deactivate all automations that use alexa_media.update_last_called
Go to developer-tools/yaml (HA WebGUI) and check your configuration
Reboot
Reconfigure AMP: Set the scan interval from 60 to 90, make sure to add the / at the end of your URL if it is missing.

andrea-git commented 2 months ago

I'm pretty sure it's affecting thousands! … On Sun, 1 Sept 2024 at 11:09, David Miller @.> wrote: My Alexa Media player has been failing to log in too. I've just tried deleting and re-adding the integration, and even clearing all my 2FA settings, but I still get the 'Alexa Media Player failed to log in' error. I updated to HA 2024.8.3 (I was previously on 8.1) and tried again, but I still can't make it work. Finding this open issue, hopefully this is a general issue with the integration which will be fixed in a future update? Or is this only affecting isolated accounts? — Reply to this email directly, view it on GitHub <#2469 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFWRDXL7DFW4LLQ7JKWVLDZULRUHAVCNFSM6AAAAABMWPTCBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGI3DQMRZHA . You are receiving this because you are subscribed to this thread.Message ID: @.>

The only thing "sure" (for now) you can do is disable system options, like this:

LindsayReid commented 2 months ago

I decided that there was nothing more important in the latest HA update that I needed more than using Alexa Media Player so, as I always keep backups, I went back to Core 2024.7.4 and am waiting for all the mess to be sorted out! I am also steering clear of Alexa Media Player updates and am staying with version 4.12.5 for the time being!

It's taking some time to get this sorted out.

On Mon, 2 Sept 2024 at 13:50, andrea-git @.***> wrote:

I'm pretty sure it's affecting thousands! … <#m1798710548594273036> On Sun, 1 Sept 2024 at 11:09, David Miller @.> wrote: My Alexa Media player has been failing to log in too. I've just tried deleting and re-adding the integration, and even clearing all my 2FA settings, but I still get the 'Alexa Media Player failed to log in' error. I updated to HA 2024.8.3 (I was previously on 8.1) and tried again, but I still can't make it work. Finding this open issue, hopefully this is a general issue with the integration which will be fixed in a future update? Or is this only affecting isolated accounts? — Reply to this email directly, view it on GitHub <#2469 (comment) https://github.com/alandtse/alexa_media_player/issues/2469#issuecomment-2323268298>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFWRDXL7DFW4LLQ7JKWVLDZULRUHAVCNFSM6AAAAABMWPTCBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGI3DQMRZHA https://github.com/notifications/unsubscribe-auth/ABFWRDXL7DFW4LLQ7JKWVLDZULRUHAVCNFSM6AAAAABMWPTCBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRTGI3DQMRZHA . You are receiving this because you are subscribed to this thread.Message ID: @.>

The only thing "sure" (for now) you can do is disable system options, like this:

image.png (view on web) https://github.com/user-attachments/assets/3731c259-120d-423a-a76d-36ac421b75b6

— Reply to this email directly, view it on GitHub https://github.com/alandtse/alexa_media_player/issues/2469#issuecomment-2324686447, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFWRDRS6R4ZBJIOMQHHFV3ZURNJBAVCNFSM6AAAAABMWPTCBWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRUGY4DMNBUG4 . You are receiving this because you commented.Message ID: @.***>

cb12tre commented 1 month ago

Due to the ongoing issues I encountered after installing Home Assistant 2024.8 and 2024.9, I decided to restore an old backup of version 2024.7.4. I can confirm that this way, Alexa Media Player works without any problems.

philtarry commented 5 days ago

Any progress or eta regarding this issue?

danielbrunt57 commented 5 days ago

@deosrc

Removing the entry seemingly removes both the amazon.com and amazon.co.uk entries. Possibly a conflicting ID (email?) causing some strange behaviour.

I believe that could cause problems inside the code. Two config entries with the same email could be confused. The required way to configure it is to specify the amazon domain account where the echo devices are registered. Anything else and you're playing with fire. Mine were originally registered in amazon.com but I moved a few months back to my amazon.ca account (same email) and had to remove/re-add the AMP config entry. Perhaps you might need to reset your password for your amazon.co.uk account, and maybe the 2SV app secret?

@philtarry A lot of code has changed since August and there is still a pending MR for alexapy: !398

deosrc commented 4 days ago

@deosrc

Removing the entry seemingly removes both the amazon.com and amazon.co.uk entries. Possibly a conflicting ID (email?) causing some strange behaviour.

I believe that could cause problems inside the code. Two config entries with the same email could be confused.

It's a while since I've looked at this but I figured that was happening. I was surprised it allowed it and didn't throw errors during the setup, but I guess it's not something you would expect people to do under normal circimstamces.

I've been running without issues for a while now and across updates of both HA and AMP, so problem seems to be solved for me. Unfortunately I'm not sure what eventually fixed it.