Closed MauriceW67 closed 1 year ago
Hmm… I might have an idea on why this is no longer working, but I have no way to test this. I believe Amazon changed the way logins work, and instead of stopping on a certain page it goes automatically through everything on its own, which for the user is faster, but in our case blazes past the part we need to login properly.
I might come up with a solution, but it'd be a blind jump without anyone testing, I'd be developing without hardware.
@Varstahl I don't mind testing stuff for you and sending log files if required.
But isn't it strange that there are people on the Kodi forum for whom 4K seems to be working? Or do you think they managed to register their device earlier?
Possibly earlier. If they have the authentication already in place then it wouldn't matter, and the update would provide them with new features.
I'll get to it, but it's going to take a while to find the time needed to work on it.
So I just tried this with a VPN. The sign in button on primevideo.com is just a redirect to the amazon.XY page of that country?
In this case the code wouldnt work, because it relies on adding params to the URL that will trigger an oauth login. That oauth login returns a token that can be used to register a device and get the real token for playback.
If you add params to the primevideo.com domain it will just ignore them and redirect you to the standard amazon login for a browser. The oauth params wont be added properly, and you wont get the token for registering a device.
I have two ideas how this could be solved: You could open the primevideo page, wait for the redirect, read the URL and add the params to that. That should work, but it might be a bit messy. You probably also need to scrape out the amazon domain, and use that for oauth and registering instead of primevideo.com.
The other idea is more speculation, but could you just add all the other amazon domains to the list? e.g. amazon.nl, amazon.it, etc. Registering a device should work on any platform, not just Android. And by registering a device you can also get cookies for browser based playback.
Sadly I dont have a primevideo.com account so all I can do is guess.
@StollD Thanks for taking the time to help investigate.
You are right, when I click the signin button on primevideo.com I am redirected to this URL:
I wouldn't mind adding params to this URL in order to get my device registered, since it's probably a one-time action anyway. But I have no clue what changes I need to make to the URL to get this to work :)
Regarding your second possible solution, maybe @Varstahl or @Sandmann79 can comment on this?
So I just tried this with a VPN. The sign in button on primevideo.com is just a redirect to the amazon.XY page of that country?
Not necessarily, depends on the country. You are usually redirected to a domain. The "problem" is that one of the URLs has the openid.oa2
parameters:
https://github.com/Sandmann79/xbmc/blob/24b1505aed816135c1d99d32a6fd1ad0a9e2bc37/plugin.video.amazon-test/resources/lib/network.py#L675-L678
Which at the moment skips through. So my plan is to alter the getURL
function to provide a redirection callback, so that we can proceed or interrupt the flow as needed, from here:
https://github.com/Sandmann79/xbmc/blob/24b1505aed816135c1d99d32a6fd1ad0a9e2bc37/plugin.video.amazon-test/resources/lib/network.py#L185-L187
Instead of forwarding the allowRedirects
, on a callback present we manually call back the given closure for every redirect, and continue/abort looping through as needed. This offers the best compromise because it allows us to operate without knowing the details of every country served by PrimeVideo (there's a ton), and could work in our favour in the future, if something like this appears again.
With that done, when we have the correct match, endpoint and URL, we can just inject what we need and abort the redirection loop early, which should make logins faster and save bandwidth on top of the device registration.
That's my plan, didn't exactly have the time to implement it yet, though.
Edit: sorry, forgot a response:
I wouldn't mind adding params to this URL in order to get my device registered, since it's probably a one-time action anyway. But I have no clue what changes I need to make to the URL to get this to work :)
If you can run a browser and register your domain that way, you can extract the cookies and inject them in a file in user data. No need to change the code for that. Storage location details in the wiki.
The params skip through because its not doing an oauth login, its just doing a regular login.
For UHD playback to work, you need to register a device with Amazon. Doing that allows you to generate a token that you use instead of the cookies. You can still use the cookies, but they won't give you UHD manifests, Amazon will treat you like you are a browser.
To register a device you need the code from openid.oa2.authorization_code
. But that value will not be returned if you are doing a regular login. You need to specifically request it, by adding theese params to the login request.
Some of them are already in the URL that primevideo.com will redirect you to, but not all. And since these values are appended to an URL that discards them, the login flow is the same as it was before, you just get cookie auth and no registered device, because the code to register a device was never requested.
Ah, I see, makes sense. I'll factor that information in.
I tried to implement my first idea (parsing the amazon domain from the primevideo redirect and using it to log in) into my minimal plugin. I have tried the requests I added in the python interpreter, but not the full plugin.
https://github.com/StollD/plugin.video.tmsp-amazon/commit/3d65d55d0554690d600fa905620d8fda6ccc6250
@MauriceW67 If you are feeling adventurous, this might work for you. Or it might not work at all.
@StollD I tried to replace the auth.py file inside the plugin with you "hack" version on my Shield and then logging on
Unfortunately it throws an error:
2022-07-01 16:39:59.758 T:14163 ERROR
Ah, I see, makes sense. I'll factor that information in.
@Varstahl Did you make any progress yet? Not trying to pressure you, just curious :)
Nah, fell sick end of week. It's all in my head but need to actually write it down.
@StollD How would I go about testing your "hack"? You mentioned something about the Python interpreter? Is this possible on an Nvidia Shield running Kodi?
Replacing the auth.py file inside your plugin didn't work obviously :)
Replacing the file is the correct way, my idea just doesnt work. I only tested the code to determine the amazon.xy domain from the primevideo redirect, not if that domain will actually allow login the same way as amazon.com/uk/de
Replacing the file is the correct way, my idea just doesnt work. I only tested the code to determine the amazon.xy domain from the primevideo redirect, not if that domain will actually allow login the same way as amazon.com/uk/de
Ok, thanks for trying. It was worth a shot :)
I don't know if it will help, but I made a screenshot of the debug screen of the Prime Video app on my Shield:
https://i.postimg.cc/BQyF7ncR/Prime-Video-20220709-103313.jpg
Interestingly, the authEndpoint URL points to api.amazon.co.uk, but all the other endpoints point to api.amazonvideo.com.
As you can see my geoLocation is detected as NL.
@Varstahl I found sort of a semi-workaround:
After I noticed on the debug screen of the app that my authEndpoint URL pointed to amazon.co.uk, I tried to login from the addon while connected to the UK using NordVPN on my Shield. This seemed to work because after jumping through all the verification hoops (including a text message on my phone with an approval URL), a new Android device became visible in my Prime Video account on the UK version of the web page.
I was then able to play videos in 4K, but the addon seemed to use my "UK" profile, because my watchlist was empty for example. Also, the show Reacher - Season 1 shows up after a search, but does not show any episodes. However other shows such as The Boys and Wheel of Time work just fine.
I then checked my device list on the primevideo.com web site without a VPN and noticed that the newly registered device had also become visible there.
So I thought I would relog into my account in the addon without the VPN. This resulted in my watchlist becoming available again, but unfortunately the shows that worked before in 4K were now back to 1080p.
So it does work kind of, but it's not ideal :)
Maybe this info will help with your research.
@MauriceW67 After testing, I think 4K\HDR currently only works under the TLD login. I tried logging in in the UK or US (Add-On Settings - Connection - Amazon Account Region) and it doesn't seem to need to be under the corresponding region ip. For example, if you are not in the US, you can still login in to Amazon.com. Prime Video accounts can be logged in under TLD (except for Japan, which is a completely separate account system). But TLD and Prime Video have different content libraries, so we still need to wait for the developers to fix PV 4K\HDR support.
@Liqianyu You are probably right. But like I said I was able to play 4K content using my NL PV account without VPN. As long as I have first logged on with VPN (connected to UK in my case) which made the device registration work.
Unfortunately this means the addon now "sees" the video content of the UK version of PV. But for shows that are available in both UK and NL, I can still play them in 4K, including Dutch subtitles.
So, I've been working on this all day, and I've got a prototype working. The bad news is, I have no idea if it works (probably doesn't at all). As far as I could see, this whole ordeal revolves around the need to set openid.assoc_handle
to amzn_piv_android_v2_[country]
, but the problem is that those are only available for the usual 4. The rest has amzn_prime_video_sso_[country]
.
If any of you wants to try, you can replace your network.py with this version. Just don't keep your hopes up.
If any of you have the hardware and the knowledge to install the PrimeVideo app on Android and give me the network trace of the login, feel free to share, because atm I don't have enough to work with.
@Varstahl I tried the new network.py, but as you already expected it doesn't work. I was able to login without errors, but 4K video was not available and no device was registered in my PV account.
Regarding the network trace, I was able to capture one on my Shield using the app PCAPdroid. However, all traffic is of course TLS encrypted. Is that still useful or do you need a decrypted capture?
Regarding the network trace, I was able to capture one on my Shield using the app PCAPdroid. However, all traffic is of course TLS encrypted. Is that still useful or do you need a decrypted capture?
Ah yeah, if we MitM it probably won't work because of CA verification 🤦♂️. Yeah, I would need it decrypted to be able to tell which variables are used and what endpoints, in sequence. Dang it.
The app that I mentioned (PCAPdroid) does come with an MITM addon, which I was able to install successfully.
However it also requires a CA certificate to be installed in the Shield's CA store and I cannot figure out how to do this.
I have found solutions such as these:
adb shell am start -a "android.intent.action.VIEW" -d "file:///sdcard/Download/PCAPdroid_CA.crt" -t "application/x-x509-ca-cert"
adb shell am start -n com.android.certinstaller/.CertInstallerMain -a android.intent.action.VIEW -t application/x-x509-ca-cert -d file:///sdcard/Download/PCAPdroid_CA.crt
But they all give me a toast message on the Shield saying "Couldn't install because the certificate file couldn't be read".
Do you have a clue on how to install a certificate on the Shield?
Took me some time, saw your message but was in the middle of a 4+ hours meeting. I'm not very experienced with Nvidia Shield, it's one of those objects that'd be cool to have but that I don't really want to buy, together with Chromecasts and Firesticks, so it was kind of a hard sell for me to actually buy one for this addon's development alone. I would think that the reason it doesn't work is explained in 3.4.1 of the PCAPdroid documentation. It's one of the things that made it hard for me to debug this stuff, on top of the fact that my main (and only?) PrimeVideo running platform is Tizen.
I would assume that those apps, by default, reject all third party certificates. Assuming one takes the time to unpack the apk, modify the XMLs to allow outern CAs and repack the apk, then it should work. I've done something similar in the past, ~I just don't know where to start with this though.~ Scratch that, after unpacking the APK with apktool
, the specific file we're looking for seems to be res/xml/network_security_config.xml
:
<?xml version="1.0" encoding="utf-8"?>
<network-security-config>
<base-config cleartextTrafficPermitted="false">
<trust-anchors>
<certificates src="system" />
</trust-anchors>
</base-config>
<debug-overrides>
<trust-anchors>
<certificates src="user" />
<certificates src="@raw/mockwebserver_root_ca_cert" />
</trust-anchors>
</debug-overrides>
<domain-config>
<domain includeSubdomains="true">localhost</domain>
<domain includeSubdomains="true">127.0.0.1</domain>
<trust-anchors>
<certificates src="@raw/mockwebserver_root_ca_cert" />
</trust-anchors>
</domain-config>
</network-security-config>
So we could add all the various TLDs to the include list and repack. Or we could add a brand new CA to the XML, by adding a new certificate, like @raw/totally_not_mitm_cert
, place the .pem file res/raw
, and add it to res/values/public.xml
, original/META-INF/MANIFEST.MF
and original/META-INF/TEST_SIG.SF
(we'd need to re-sign the files though? not sure if necessary). Or we could overwrite the res/raw/mockwebserver_root_ca_cert.pem
with the CA from PCAPdroid instead, but still would need signing I think (maybe).
Have I ever done it? No. Does it sounds fun? Possibly :D
I think that adding TLDs is possibly the safest bet, but I don't know how the SHA signatures work. Could removing it from the TEST_SIG.SF
work? Does the MANIFEST.MF
entry need to match the file? Probably, I'd assume, but I'm not an android expert, and it's not something I'll be able to discover at 2am on a working day, hehe.
Feel free to read more/experiment with any of these. Might get us closer, or not, what do I know, I'm a tired old man.
Just found this, might work or not, I have no idea:
Basically just remove the signatures, create a new keystore to sign the apk with, and resign it. I remember doing this very thing many moons ago, but I'd be hard press to remember the specifics at this moment.
@Varstahl My test result is the same as MauriceW67. And to MITM grab the packets, I think need to modify the application or Root. I don't know if I can use other devices and methods instead.
Japan region login is a bit strange, as my European Prime subscription can be logged in the Japan region. As far as I know Amazon Japan uses a separate account system. At least amazon.co.jp other regional accounts can't log in. Does this mean that Prime Video accounts are globally interoperable even for the Japan region?
@Varstahl Thanks for looking into this further.
I did find an older guide on how to capture traffic using PCAP Remote:
https://www.exandroid.dev/2021/03/21/capture-all-android-network-traffic/
However, when I tried to make the changes to the APK and recompiling it, it would no longer install on the Shield. I used the APKEasyTool instead of apktool though, so that may have something to do with it.
The changes to the APK you are suggesting are going a bit over my head though :) Maybe if I spend more time learning/investigating I would start to understand, but for the moment it's not clear to me.
It would be easier to maybe use an Android phone instead of a Shield, since that would be easier to root and/or install certificates on. But I don't think Android phones support 4K playback?
@MauriceW67 If the phone is available, you can go directly through the Android emulator. Some emulators have Root access built in. then there are some methods and software to bypass SSL Pinning, for example https://github.com/Fuzion24/JustTrustMe Or decompile and modify the app, for example https://github.com/felHR85/Uncertify
Shield TV should be able to use SHIELD Developer OS Images to get Root. I think it's best to test it via mobile first, because odds are Android will use a very similar login.
@Varstahl The Fire TV Stick 4K is now on sale for Prime members for 27 euros. Or the 4K Max version for 37 euros if you're interested :).
I'd even be willing to donate one if you think it would help you debug the issue.
Does this mean that Prime Video accounts are globally interoperable even for the Japan region?
Amazon's accounts are global, you can use your account on any of amazon's websites. The streaming services might be geolocked depending on where you're accessing what. PrimeVideo is global, so you can technically access it from everywhere, but the content and the languages you see depend on the georegion where you're locked into.
If the phone is available, you can go directly through the Android emulator
I was thinking that I couldn't do it before for 4k testing, but then I just realized that I don't need to, we know 4k is working o if I can just test the endpoints then that would kind of solve the problem. I have a couple emulators I can try it on, I'll see if it works (after work ofc).
@Varstahl The Fire TV Stick 4K is now on sale for Prime members for 27 euros. Or the 4K Max version for 37 euros if you're interested :)
hnng… is this the day where I give in? Honestly I don't know if it would help, at least in this specific regard. ~I'll think it over. Possibly past the Prime Day, hahaha, wouldn't be uncharacteristic of me.~ Fuck it, I gave in, I'll receive it tomorrow.
hnng… is this the day where I give in? Honestly I don't know if it would help, at least in this specific regard. ~I'll think it over. Possibly past the Prime Day, hahaha, wouldn't be uncharacteristic of me.~ Fuck it, I gave in, I'll receive it tomorrow.
LOL, sorry didn't mean to make you spend extra money... Just thought I'd mention it since the discount is pretty substantial. Just noticed that the Chromecast with Google TV is also discounted to 35 euros (at least in Italy, in NL there is no discount).
Eh, you know, I only have one TV which supports 4k, and it's a big ass TV I bought for my elderlies with PrimeVideo already on it, so I'm not that into 4k. But still, at this price I might as well. I've always wanted to reverse engineer some parts of Widevine I didn't have access to. Until now 😈
@Varstahl your code from the android-uhd-fix branch prefers the parameters in the redirect over the ones in params, am I reading that correctly? i.e. it won't overwrite a value with the one from params, it will only add values from params.
Have you considered only retaining the assoc_handle, and blindly replacing everything else with the values from params?
Have you considered only retaining the assoc_handle, and blindly replacing everything else with the values from params?
There's only 2 parameters overlapping, the assoc_handle
which if changed results in a 404, and the return_to
URL. I don't remember if I tested with the return_to
alone. Although it probably wouldn't work as it uses different endpoints, and I'm not sure if those endpoints exist on other servers, although it's possible (better worded: they resulted in 404, but it might be a 404 due to query parameters, so it's possible that with the correct redirects those endpoint work for primevideo too).
Every other parameter is either missing, or equal to the one in params
. It should also report in the logs eventual collisions. But yes, I can default to log and overwrite, I should be able to push the change later today for more testing.
Every other parameter is either missing, or equal to the one in
params
. It should also report in the logs eventual collisions. But yes, I can default to log and overwrite, I should be able to push the change later today for more testing.
Did you manage to test the new changes yet that you had in mind?
Not yet, I also have the Firestick 4K Max on my desk, still packaged. Real life's brutal, yo. Even if my plan could work, it would still mean I need to reverse the original prime app, might take more time than simply coding. I'll see what I can do.
If you want to do the quick test (that I still don't think it's gonna work, but I might be mistaken), you can take this few lines of code: https://github.com/Sandmann79/xbmc/blob/b83e9ba7376f97c3899619ed4ab100d6fcde2835/plugin.video.amazon-test/resources/lib/network.py#L680-L684 And change them into this:
elif k in ['openid.assoc_handle']:
# Prefer the values returned by the official endpoint
pass
else:
query[k] = [v]
Log('Login data conflict on {}: {} => {}'.format(k, query[k], v))
@Varstahl I tested your change and as you expected it didn't work :)
After logging in and entering the OTP code I got a popup from the addon saying "We're sorry. The Web address you entered is not a functioning page on our site. Go to amazon.com's HomePage"
Yeah, expected as much. You can put back the ['openid.return_to', 'openid.assoc_handle']
, and leave the rest modified, but I don't think that'll help either. Can't hurt, I guess.
Changed that line back and I could login again, but no 4K unfortunately.
By the way did you read that Amazon is going to revamp their PV interface:
https://www.engadget.com/amazon-prime-video-redesign-announced-204019411.html
Hopefully that won't affect the addon too much...
haha, yeah. I had flashbacks. I'm pretending not to know for the foreseeable future. I've went through 2 of their renovations already, with API endpoints being brought up and shut down. I wouldn't surprise if they completely destroy everything overnight.
I can MiTM decrypt HTTPS traffic on tvOS, does that help?
Not sure if they use the same endpoints but sure, feel free to share a full login. Censor the obviously sensitive parts like the password values if you share them in public.
@MauriceW67 did you manage in the end to register this add on?
@cralcubo No not with the Prime Video account.
But as I wrote above (https://github.com/Sandmann79/xbmc/issues/632#issuecomment-1179692637) there is sort of a workaround by first logging in with the TLD forced to .co.uk.
You then get an Android device registered in your "UK" profile which can be used to play 4K videos, even if your own account is PV.
The downside is that the "UK" profile is separate from your PV profile with regards to your watchlist etc. Also, some shows that are paid in the UK, while included with PV will not play (example: Las Vegas).
Another thing is that you need to enable the beta feature "Use web API", otherwise some new shows such as LOTR - Rings of Power will show up with an empty episode list.
@Varstahl Did you make any progress at all with this? :)
I barely bootstrapped the Firestick ç_ç
Seriously, my time is being consumed by a void. Took some holidays and thought "I'm gonna fix the addon and change this chip from the headphones". Did neither. And the time off is already past. Raising a tiny human prototype's hard, yo.
@Varstahl hi, is there any progress and any chance of this problem being fixed in the near future? Thank you.
Having the same issue on my Shield, no 4k/HDR/DV. Also seems to be affecting some 1080p streams, maxing out playback at 720p (not all).
For my Dutch PV account this is now solved when using 0.9.7 beta 7:
https://github.com/Sandmann79/xbmc/issues/674#issuecomment-1518524373
So, my spanish account can finally play 4K/HDR. But for some reason, the video is stuttering a lot. I've tried different refresh rates, although Kodi auto changes it according to the content (I've tried playing "The Boys" and the resolution auto changes to 2160p/24. The stuttering does not happen on other addons using Input stream, like Netflix or Disney+ (on Kodi, I mean). It also does not happen on the official Prime Video app.
Not sure if it's because of the addon or not. Does anyone else have the same problem? My device is a Fire TV 4K Max. I've also tried changing the playback setting from Input Stream to Android, same problem.
Thanks!
@pmsobrado That issue has been reported already:
https://github.com/Sandmann79/xbmc/issues/676 https://github.com/xbmc/inputstream.adaptive/issues/1197
Unfortunately the devs of InputStream:Adaptive have not responded yet.
For me it only happens with HDR10 material. The Lord of the Rings show is in Dolby Vision and works fine. Also, if I disable HDR10 in settings and play The Boys, it plays fine in 4K SDR.
Maybe you could confirm to the devs that you have the same issue in the second link?
@MauriceW67 i have Fire Cube, and the boys jerks for me, although HDR10 is switched off. Whitelist is empty, my tv (S95b) switching to 4k 24hz. can u test this?
Addon used
Account type
System Setup (please provide the following information):
Upload Logs
Debug log: https://paste.kodi.tv/inehevowow.kodi avod-login.log: https://paste.kodi.tv/pikeqeciwi.kodi avod-login-mfa.log: https://paste.kodi.tv/uniqokukix.kodi
Describe the bug
After enabling the 4K/HDR10/DV options in the addon and trying to play 4K episode, a popup appears telling you that the device is not registered at Amazon for 4K playback and that the user should login again from the addon.
After doing this, 4K playback still does not work. Also the primevideo.com web site does not show any additional devices that have registered.
Note: My Prime Video account is a Dutch one and in the browser I need to logon to primevideo.com