bbindreiter / node-red-contrib-alexa-remote2-applestrudel

Node-Red Nodes for interacting with Alexa
https://flows.nodered.org/node/node-red-contrib-alexa-remote2-applestrudel
MIT License
74 stars 20 forks source link

"SyntaxError : Unexpected End of JSON Input" during initialization #53

Closed nrout-HA closed 1 year ago

nrout-HA commented 2 years ago

Hi all, I'm having issues initializing my account. Not sure when this started happening. I think the past one or two days. When I initialize, it asks me to sign into the browser. After I do that, the alexa init node shows the SyntaxError. I'm clueless as to how to troubleshoot this, so any suggestions appreciated.

My init flow is:

[{"id":"5db06587.9762bc","type":"inject","z":"b7f6e999.c59f18","g":"9e5610cb.2df3c","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":"5","topic":"","payload":"Started!","payloadType":"str","x":180,"y":180,"wires":[["e565b92b.653278"]]},{"id":"e565b92b.653278","type":"change","z":"b7f6e999.c59f18","g":"9e5610cb.2df3c","name":"set","rules":[{"t":"set","p":"payload","pt":"msg","to":"alexa","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":350,"y":180,"wires":[["549f4dd9.40faf4"]]},{"id":"549f4dd9.40faf4","type":"alexa-remote-init","z":"b7f6e999.c59f18","g":"9e5610cb.2df3c","name":"","account":"5ed557f8.3cce28","option":"fresh","x":580,"y":180,"wires":[[]]},{"id":"5ed557f8.3cce28","type":"alexa-remote-account","name":"alexaaccount","authMethod":"proxy","proxyOwnIp":"192.168.0.100","proxyPort":"3453","cookieFile":"NodeRedAlexaAuth.txt","refreshInterval":"6","alexaServiceHost":"pitangui.amazon.com","amazonPage":"amazon.com","acceptLanguage":"en-US","onKeywordInLanguage":"","userAgent":"","useWsMqtt":"on","autoInit":"off"}]

ctoujoma commented 2 years ago

Hi!

Im having the same problem, since yesterday, I was just about to open an issue also, Getting "Unexpected End of JSON Input" after retrieving the cookie with proxy method, Other methods wont work also..

Thanks for your time,

bbindreiter commented 2 years ago

Will check asap. Amazon API change possibly

bbindreiter commented 2 years ago

5.0.9 pushed with updated dependencies (https://github.com/Apollon77/alexa-remote/releases/tag/v4.1.2), please report back whether this fixes your issues.

I did a fresh proxy init with pitangui.amazon.com and it worked fine.

ctoujoma commented 2 years ago

Not working for me.. Im with "alexa.amazon.es", i've tried also with 'pitangui' and still nothing. Same message, it appears right after success on login screen, Just tell me if i can provide more info

bbindreiter commented 2 years ago

Does a login on https://alexa.amazon.com work for you?

ctoujoma commented 2 years ago

Yes, it redirects me to alexa.amazon.es

i tried also with a bowser with no credentials stored, it made me login this time, after success, it redirected me to alexa.amazon.es again,

bbindreiter commented 2 years ago

Thanks, I'll take a look on the login tomorrow and how to debug it best.

ctoujoma commented 2 years ago

Thanks! If i could help just let me know

nrout-HA commented 2 years ago

The update did work for me. I was/am using pitangui. Thanks for the update 👍

bbindreiter commented 2 years ago

@ctoujoma have you set acceptLanguage to es-ES ?

ctoujoma commented 2 years ago

Yes sir, image

I could try with a fresh install..

bbindreiter commented 2 years ago

The log on your machine contains the node-red network communication. On my raspi this is in the syslog (/var/log/syslog). You can see what's being returned on your login. Please make sure that you wipe out all sensitive data before posting. Thank you!

ctoujoma commented 2 years ago

Thanks for your patience, Found the logs, they are huge, and not sure if github have private msgs.. I dont get why its giving me a 401 error when apparently it succesfully logins and retrieves the cookie

22 Feb 20:02:10 - [info] [alexa-remote-account:Alexa] intialising "Alexa" with the PROXY method and NO saved data... 
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Remote: starting 
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Remote: {"authMethod":"proxy","initType":"proxy"}
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Remote: Use as User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Remote: Use as Login-Amazon-URL: amazon.es
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Remote: Use as Base-URL: alexa.amazon.es
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Remote: No cookie given, generate one
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Use as Login-Amazon-URL: amazon.es
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Use as Base-Amazon-URL: amazon.com
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Use as User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Use as Accept-Language: es-ES
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Proxy mode disabled
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Proxy Init: loaded temp data store ass fallback former data
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Proxy Init: reuse frc from former data
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Proxy Init: reuse map-md from former data
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Proxy Init: reuse deviceId from former data
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Proxy-Server listening on port 3456
22 Feb 20:02:10 - [debug] [alexa-remote-account:Alexa] Alexa-Remote: Error from retrieving cookies
22 Feb 20:02:10 - [warn] [alexa-remote-account:Alexa] open 192.168.1.108:3456 in your browser
22 Feb 20:02:14 - [debug] [alexa-remote-account:Alexa] Router: / / GET / {"host":"192.168.1.108:3456","connection":"keep-alive","upgrade-insecure-requests":"1","user-agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","accept-encoding":"gzip, deflate","accept-language":"es-ES,es;q=0.9,ca;q=0.8","cookie":"X-OPENMEDIAVAULT-LOGIN- etc......................

(cookie info, a lot of repeated entries)

then this one

22 Feb 20:02:23 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Proxy detected SUCCESS!!
22 Feb 20:02:23 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Proxy catched cookie: X-OPENMEDIAVAULT-LOGIN-%242y%2410%24D9DTQ
22 Feb 20:02:24 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Response (200)
22 Feb 20:02:24 - [debug] [alexa-remote-account:Alexa] Register App Response: {"response":{"success":{"extensions":{"device_info":{"device_name":" .........................

and it ends with this:

22 Feb 20:02:24 - [debug] [alexa-remote-account:Alexa] Alexa-Cookie: Response (401)
22 Feb 20:02:24 - [debug] [alexa-remote-account:Alexa] Get User data Response: ""
22 Feb 20:02:24 - [debug] [alexa-remote-account:Alexa] Alexa-Remote: Error from retrieving cookies
22 Feb 20:02:24 - [error] [alexa-remote-account:Alexa] Unexpected end of JSON input
bbindreiter commented 2 years ago

@Apollon77 maybe you can help out here. In the example above, the base url in Alexa-Remote is correctly set to alexa.amazon.es but in Alexa-Cookie it's reverted to amazon.com. I don't see the parameter baseAmazonPage being set here: https://github.com/Apollon77/alexa-remote/blob/master/alexa-remote.js

Maybe I'm completely wrong but could be the cause for the error? At least that's the only idea I have.

Apollon77 commented 2 years ago

Yes. You need to make sure the locale is set correctly. The login itself is always done against amazon.com and the tokens are then later exchanged for es. So, is the locale correct?

I can also have a look at code later but this would be important question first

bbindreiter commented 2 years ago

https://github.com/bbindreiter/node-red-contrib-alexa-remote2-applestrudel/issues/53#issuecomment-1047497980`

It's es-ES. Didn't know that the login is always done against amazon.com. As you can see in the log above the login itself works but when there's an additional request that gives a 401.

I tried to but couldn't reproduce it with my account (US and German). I think it'd be a quick fix if I'd know how to reproduce.

nrout-HA commented 2 years ago

I'm getting the error again whenever I restart node red. What's weird is one time I tried initializing again the next morning, and it worked. I tried looking for the nodered logs, but I'm not sure where they are. I'm running node-red as an addon in Home Ass OS, on a Rasp 3B+. Any suggestions? edit : oh, duh. It's on the Home Ass supervisor page... Let me try logging some debug info...

ctoujoma commented 2 years ago

On the Add-on page, check on the last tab, i found them there. Also, on the configuration tab, add this "log_level: debug"

nrout-HA commented 2 years ago

I'm seeing debug output similar to what you have.

alexa] Alexa-Remote: starting initialisation:
alexa] Alexa-Remote: {"authMethod":"proxy","initType":"proxy"}
alexa] Alexa-Remote: Use as User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
alexa] Alexa-Remote: Use as Login-Amazon-URL: amazon.com
alexa] Alexa-Remote: Use as Base-URL: pitangui.amazon.com
alexa] Alexa-Remote: No cookie given, generate one
alexa] Alexa-Cookie: Use as Login-Amazon-URL: amazon.com
alexa] Alexa-Cookie: Use as Base-Amazon-URL: amazon.com
alexa] Alexa-Cookie: Use as User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36
alexa] Alexa-Cookie: Use as Accept-Language: en-US
alexa] Alexa-Cookie: Proxy mode disabled
alexa] Proxy Init: loaded temp data store ass fallback former data
alexa] Proxy Init: reuse frc from former data
alexa] Proxy Init: reuse map-md from former data
alexa] Proxy Init: reuse deviceId from former data
alexa] Alexa-Cookie: Proxy-Server listening on port 3456
alexa] Alexa-Remote: Error from retrieving cookies
alexa] open 192.168.0.111:3456 in your browser
Alexa-Cookie: Proxy catched cookie: session-id=...
Alexa-Cookie: Proxy catched parameters: {"openid.assoc_handle":"amzn_dp_project_dee_ios","openid.claimed_id":"https://www.amazon.com/ap/id/amzn1.account....
Handle token registration Start: {"loginCookie":"session-id=....
Alexa-Cookie: Register App
{"host":"api.amazon.com","path":"/auth/register","method":"POST","headers":{"User-Agent":"...
Alexa-Cookie: Sending Request with {"host":"api.amazon.com","path":"/auth/register","method":"POST","headers":{...
Alexa-Cookie: Response (200)
Register App Response: {"response":{"success":{"extensions":{"device_info":{"device_name":"...
Alexa-Cookie: Get User data
{"host":"alexa.amazon.com","path":"/api/users/me?platform=ios&version=2.2.443692.0","method":"GET","headers":{"User-Agent":"...
Alexa-Cookie: Sending Request with {"host":"alexa.amazon.com","path":"/api/users/me?platform=ios&version=2.2.443692.0","method":"GET","headers":{"User-Agent":...
Alexa-Cookie: Response (401)
Get User data Response: ""
Alexa-Remote: Error from retrieving cookies
26 Feb 11:47:03 - [error] [alexa-remote-init:635ac7f756d1bcbe] Unexpected end of JSON input
nrout-HA commented 2 years ago

I also sometimes see ""ENOENT: no such file or directory, open 'NodeRedAlexaAuth.txt'"" in my nodered debug. That's the filename I have under the Alexa Account in nodered.

bbindreiter commented 2 years ago

@flashbacck thanks for sharing your log. Like @Apollon77 I thought it's because of some locale/baseurl mismatch but in your case everything is set to amazon.com and en-US. Can you confirm that your amazon alexa account really is from amazon.com with locale en-US?

Could it be some 2FA that's enabled?

nrout-HA commented 2 years ago

It is correctly set at amazon.com and en-US. I do have 2FA enabled. I don't think I've ever had the alexa-remote2 module ask for it though. It always just went through with the password. Is that not supposed to happen?

thorrrr commented 2 years ago

Hi moved over onto this fork and followed the guide which has 90% worked had to set a new cookie file and change name then do it incognito mode and it worked kind of created a new cookie file and i use 2FA which it asked for and accepted. It is connecting but i am getting error logs from NR See here for settings. I have tried US settings but that really upsets it and i get many more logs. It used to work if i just set language to en-US but no longer. Not sure if Amazon have change Auth

My errors in log are here How can i get rid of this error please ?

nrout-HA commented 2 years ago

Interesting. I browsed to the amazon login using incognito, and after entering password, it asked for the 2FA. I entered that, and now nodered reports the error:

"Error while checking Authentication: Error: getaddrinfo ENOTFOUND https://pitangui.amazon.com"

The last few lines of the debug log are:

 Alexa-Cookie: Response (200)
 Alexa-Cookie: Add Cookie csrf = 1981
 Alexa-Cookie: Result: csrf=1981
 Final Registration Result: {"loginCookie":"
 Alexa-Remote: Sending Request with {"host":"https://pitangui.amazon.com","path":"/api/bootstrap?version=0","method":"GET","timeout":10000,"headers":{}}
 Alexa-Remote: Response: Error: Error: getaddrinfo ENOTFOUND https://pitangui.amazon.com
 Error while checking Authentication: Error: getaddrinfo ENOTFOUND https://pitangui.amazon.com
nrout-HA commented 2 years ago

ok... I tried initializing in Firefox. And it worked. I was previously doing it in Chrome. I don't know what the hell is going on O_o

ctoujoma commented 2 years ago

I've just tried with Edge, it worked! It asked for the OTP code, so maybe was some issue with temp files in Chrome or smz

Apollon77 commented 2 years ago

WTF ... Now I understand nothing anymore ... it should not be brower specific ?!

To the place where the error happens: It is "after"the login was successfull where the process queries the logged in user to get the marketplace country to then exchange the tokens to the correct country page

@flashbacck "getaddrinfo ENOTFOUND" is an DNS issue on your side somewhere! the nodejs process can not get the IP for that hostname, so thats something different

nrout-HA commented 2 years ago

WTF ... Now I understand nothing anymore ... it should not be brower specific ?!

To the place where the error happens: It is "after"the login was successfull where the process queries the logged in user to get the marketplace country to then exchange the tokens to the correct country page

@flashbacck "getaddrinfo ENOTFOUND" is an DNS issue on your side somewhere! the nodejs process can not get the IP for that hostname, so thats something different

Ok, good to know. I am running Adguard, maybe something happened there while I was trying to get it to work.

rubberychicken commented 2 years ago

I have the same "Unexpected end of JSON input" error. I have tried on different node-red installations, different computers and different browsers (edge, chrome, firefox) in both regular and private mode.

I am using AUS alexa.amazon.com.au amazon.com.au en-US

Similar to the other logs above everything seems to go ok until the last step where we get this:

4 May 04:55:53 - [debug] [alexa-remote-account:7a49d4ae7b286636] Alexa-Cookie: Response (401) 4 May 04:55:53 - [debug] [alexa-remote-account:7a49d4ae7b286636] Get User data Response: "" 4 May 04:55:53 - [debug] [alexa-remote-account:7a49d4ae7b286636] Alexa-Remote: Error from retrieving cookies 4 May 04:55:53 - [error] [alexa-remote-account:7a49d4ae7b286636] Unexpected end of JSON input

bdydrp commented 2 years ago

I get the same error also Node set correctly Will try and get logs tonight image

rodrigoadachi commented 2 years ago

My case is a little different

Service Host: pitangui.amazon.com Page: amazon.com.br Language: pt-BR

Return: "Unexpected end of JSON input"

Captura de tela 2022-06-22 221650

Yesterday I got no problems with this setup above.

rodrigoadachi commented 2 years ago

I managed to delete the entire cookie and re-login

kuradi commented 2 years ago

i'm getting these errors... what is going on here? ` (node:17) TimeoutOverflowWarning: 2592000000 does not fit into a 32-bit signed integer.

26 Jun 10:52:33 - [info] [alexa-remote-account:f0e7f266622b6a8f] auto refreshing cookie...

26 Jun 10:52:33 - [info] [alexa-remote-account:f0e7f266622b6a8f] intialising with the PROXY method and saved data...

26 Jun 10:52:35 - [warn] [alexa-remote-account:f0e7f266622b6a8f] 401 Unauthorized

Timeout duration was set to 1. `

bdydrp commented 2 years ago

Just revisiting this as i still get same error using Australian credentials Screenshot_20220715-234006_Samsung Internet

Apollon77 commented 2 years ago

Now after all the other fun sorted out look into this here.

Can anyone please use thelatest alexa.remote version (or latest node version from here with the latest alexa-remot in it) and do that again?

I need:

bdydrp commented 2 years ago

ok, i think i have the logs you require! Node is up to date (as far as i can tell) Node Settings: image

Logs: NodeRed Logs - Alexa.txt

Apollon77 commented 2 years ago

Can you delete the cookie please and such ... I simply think that there is no special logic in for au right now ... but I also miss some more loglines from alexa-remote "bootup" ...there should be more logs.

Additionally I would like to see the fill log of "getting cookies via proxy" too please. You can also send as email with referecne to this issue to iobroker@fischer-ka.de

Apollon77 commented 2 years ago

@bbindreiter which of the fields from config are passed to which library fields? especially the "Service host" and "page" ones?

bbindreiter commented 2 years ago

Service Host -> alexaServiceHost -> @bdydrp try without https:// prefix (not sure if that makes a difference, but it should be without it) Page -> amazonPage

Full example (using JSON.stringify):

{
  "proxyOwnIp": "pi-network.lan",
  "proxyPort": "3456",
  "alexaServiceHost": "pitangui.amazon.com",
  "amazonPage": "amazon.com",
  "acceptLanguage": "en-US",
  "onKeywordInLanguage": "on",
  "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15",
  "useWsMqtt": true,
  "refreshCookieInterval": 0,
  "proxyLogLevel": "warn",
  "cookieJustCreated": true,
  "bluetooth": false,
  "setupProxy": false,
  "proxyOnly": true,
  "cookie": {
    "loginCookie": "xxx",
    "frc": "yyy",
    "map-md": "xxx",
    "deviceId": "aaa",
    "deviceSerial": "bbb",
    "refreshToken": "xxx",
    "tokenDate": 1658856084228,
    "macDms": {
      "device_private_key": "ppp",
      "adp_token": "{ppp"},
    "amazonPage": "amazon.com",
    "localCookie": "ppp",
    "csrf": "-201645313",
    "deviceAppName": "ioBroker Alexa2"
  },
  "amazonPageProxyLanguage": "en_US"
}
bdydrp commented 2 years ago

Service Host -> alexaServiceHost -> @bdydrp try without https:// prefix (not sure if that makes a difference, but it should be without it) Page -> amazonPage

Full example (using JSON.stringify):

{
  "proxyOwnIp": "pi-network.lan",
  "proxyPort": "3456",
  "alexaServiceHost": "pitangui.amazon.com",
  "amazonPage": "amazon.com",
  "acceptLanguage": "en-US",
  "onKeywordInLanguage": "on",
  "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15",
  "useWsMqtt": true,
  "refreshCookieInterval": 0,
  "proxyLogLevel": "warn",
  "cookieJustCreated": true,
  "bluetooth": false,
  "setupProxy": false,
  "proxyOnly": true,
  "cookie": {
    "loginCookie": "xxx",
    "frc": "yyy",
    "map-md": "xxx",
    "deviceId": "aaa",
    "deviceSerial": "bbb",
    "refreshToken": "xxx",
    "tokenDate": 1658856084228,
    "macDms": {
      "device_private_key": "ppp",
      "adp_token": "{ppp"},
    "amazonPage": "amazon.com",
    "localCookie": "ppp",
    "csrf": "-201645313",
    "deviceAppName": "ioBroker Alexa2"
  },
  "amazonPageProxyLanguage": "en_US"
}

I dont normally have https://!! I added that last time to see if it made a differrence - obvioulsy not!! It doesnt work withor without it!

Have new logs...This is everything i could see related to Alexa nodes from NR restart to error!! Node Red logs - Alexa V2.txt

Apollon77 commented 2 years ago

So Yes Service host without https:// please, but this is not the topic

In fact you need to set "baseAmazonPage" to "amazon.com.au" ... at least this would be my first try ... I know this is "undocumented" on alexa-remote level, but existing in alexa-cookie lib :-) In fact it is amazon.com for ALL except amazon.com.jp (known till now), so I assume that also amazon.com.au is a special case there. and most likely we need a second change (assumed).

So @bdydrp up for an experiment? ;-)

1.) Find the location of alexa-cookie2 in your npm tree (e.g. "find . -name alexa-cooke.js"). Hopefully you only find it once, else show the locations please 2.) open that file (maybe create a backup copy before) in an editor 3.) Find that line https://github.com/Apollon77/alexa-cookie/blob/master/alexa-cookie.js#L139 4.) AFTER that line add a new line

_options.baseAmazonPage = 'amazon.com.au';

5.) find these lines https://github.com/Apollon77/alexa-cookie/blob/master/alexa-cookie.js#L153-L155 6.) AFTER the line with the closing bracket ("}") insert the following

            else if (amazonDomain === 'au') {
                _options.baseAmazonPageHandle = '_' + amazonDomain;
            }

Retry to generate cookie and please send new log

bdydrp commented 2 years ago

OK - Give me a bit, and i will get what you need!!

bdydrp commented 2 years ago

Alright - I think i have made the correct chnages to the .js file Now when i log in, i get a new error msg. "No CSRF Found" image

Here is the logs, and i have also included the .js file - incase you wish to check it is correct as per your instructions alexa.js changes.txt

I have emailed you the .js file, as i cant upload it here

Apollon77 commented 2 years ago

Ok, another try. Please remoive the lines added above in Step 6 again and try again please

bdydrp commented 2 years ago

ok - now i get this image

Apollon77 commented 2 years ago

Ok, then something else is different ... only option now is to sniff the Alexa App traffic to see whats different. In fact that means:

bdydrp commented 2 years ago

Ok... Damn australia alexa! Lol

I would be, but i dont use ios... only androids here! Let me look into it anyway as i'd still like to help out anyway possible

Apollon77 commented 2 years ago

On android this root cert installing is a mess and I have no oidea who got that ever working :-(

Alternative would be set a new password, tell it to me, I try and then when I did it you change password ...

bdydrp commented 2 years ago

Im happy to change password and let you test...

Will send you an email Cheers

Apollon77 commented 2 years ago

alexa-remote2 5.7.4 n it's way.