FHD playback / UHD listing

[jgimness]

I see a commented out HighDef parameter, is that what enables Ultra HD listings to be retrieved? Any info would be appreciated.

[Sandmann79]

No, some years ago this parameter displayed the HD content. To get UHD content listed, you have to use the HideNum=F parameter. But it makes no sense, because the devicetypeID that is used for playback doesn't support UHD.

[jgimness]

Thanks for the response. Tried with HideNum=F and no luck. Any other hints? Maybe I need a valid devicetypeID that can actually supports UHD?

[Sandmann79]

Maybe for listing it, but not for playback. You cannot change the playback devicetypeid, that's the only one which is currently working.

[greystashh]

Are there any news for uhd?

[Varstahl]

@greystashh some android reverse engineering has probably to be done, but I didn't find the time yet.

[greystashh]

Have you been able to look at it in the meantime?

[Varstahl]

Nope, it's still on my list of things to do for the addon but still haven't been able to look into it.

[jgimness]

Any updates? Just a side note, it looks like instantwatcher (which I believe has partner-level API access) only has 4 items listed as UltraHD (used to be a lot more) https://instantwatcher.com/a/search

[mirh]

https://github.com/asciidisco/plugin.video.netflix/issues/86#issuecomment-342261005 https://www.amazon.com/gp/help/customer/display.html?nodeId=201648150 Ideas

EDIT: https://www.amazon.com/gp/help/customer/display.html?nodeId=201460880 this seems to cover way more devices (and even though it's still far from 100% comprehensive, at least it gets near to recognizing my XZ2 can access HDR)

[Varstahl]

Yeah, one of my devices that I was targeting to reverse engineer the 4K video playback is on that amazon's support list. The problem that peak3d is describing is different though: it's not enough to "have" those in the lists, if you can't playback. And you can't reproduce those videos, without a full reverse engineering of the Widevine drm encryptions that is shipped in those devices, which is different from what is publicly available.

Short example: we have support for FHD movies in Amazon VOD, but nothing is able to reproduce them. If we can't playback FHD why bother with UHD?

It sucks, but that's Google for you.

[mirh]

I thought that was just a matter of various WV levels..

[Varstahl]

I'm not exactly "in the know", but when I talked with some devs (under NDA) they hinted at the fact that even for PrimeVideo.com, several chromium-based browsers are not able to decode properly videos, even when taking the Widevine straight from Chrome and putting it in the other browser, because along with the the DRM management they would need to ship with particular codecs, that all bundle for a fair amount of millions of dollars.

I admit I never even began reverse engineering my TV's handshake, but from what I understood even if we were to replicate and get both 4K and up to date Widevine from whatever source, chances are it will never ever be rendered to video. FHD decoding can maybe work on Windows 10, if what I heard is correct, but 4K probably won't on any platform.

[mirh]

Ok, it's true that amazon's streaming "parameters" are bonkers, but for as much as codecs are concerned the only peculiarity should be the usage of H265. End. How could they get hw accelerated otherwise? Nothing that isn't already handled by inputstream imho.

Then, as I was saying.. I know instead *DRM* is the thing pretty famous to be finicky.

---

I've spent all day trying to understand what particular kinds Prime Video have been using, but to no definitive avail. Of course Widevine plays a role... With L3 being the minimum requirement to do *any thing at all*. But then for some reason, computers can get away with 1080p easily, while on android not even L1 seems to be enough for 720p sometimes (though that could be as well some coarse device model check...).

Proceeding with the analysis, a breadcrumb trail, led me inside the android apk, whose biggest library is literally libAIVPlayReadyLicensing.so.

So, I was almost going to call it a day, they may be using the same super pesky hardware-baked thing of netflix (which in this case they aren't offering on x86, just out of laziness or whatever) And it would also explain why a lot of android TVs are fine with it - since they are all reportedly required to support it (even though I couldn't find anything in the official documentation... which is really poor in documenting drm)

But I could only find PlayReady SL2000 certificates inside of com.amazon.avod.thirdpartyclient.apk\assets\PlayReady\.... Which is not actually the playready 3.0 I was theorizing about. ... and at that point I realized android TVs uses a completely different application (com.amazon.amazonvideo.livingroom).. Unfortunately protected AF, but that could still cover my hypothesis ....

Until the last blow arrived, when I eventually found out Xperia XZ Premium can actually do 4K with the normal app. On the "normal" android. And even though its oreo firmware has a vendor\lib\qcdrm\playready\ folder... I know from both drminfo and exoplayer tests on its sibilings sony phones with the same board, that it always was inert.

So... TL;DR I'm back to square one. FHD anywhere shouldn't be rocket science to be honest anyway.

p.s. when you'll eventually get to look into UHD titles, take also notice they are total asses with their interface/api.

[Varstahl]

Although I have quite some old expertise in reverse engineering, when it comes to this sort of things I usually go completely platform agnostic and just sniff the living sh*t out of the network transmission through a transparent proxy, and reverse engineer the protocol from there. That's the way I reverse engineered PrimeVideo's everything, and it worked (kind of) okay. Even then, replicating what FireFox and Chrome do to get and decode the streams doesn't let Kodi reproduce videos in the same way. I'm relatively new to Kodi as a whole, so if I had to analyse both Kodi and IS.A's sources it would take me months.

I've been away from the scene from quite a long time, and I especially know nothing of android reversing, so my only bet in this would be to either transparent proxy a TLS connection between my TV and amazon servers hoping that they don't use certificate pinning, or be stuck with about nothing. Which is the reason why I still haven't done anything, as I severely doubt that's going to work. (Because they use pinning, right?)

Anyways, I have a few days off work in front of me, so if you guys have suggestions or attack vectors, or want to set-up a mini reverse engineering collective effort, let me know.

[Varstahl]

Also, thanks for the additional information, I had no idea that (at least on some platforms) Microsoft's PlayReady 3.0 was used. Which explains why Windows 10 is fucking required to play 4k.

* exhausted sigh *

[mirh]

That's for netflix though, as I said. Aside of this specific instance though, I know of no particular "merit" of W10 on anything. (and, even then, there seems to be plenty of samples floating around for playback, at least under UWP...)

Then, if you want to work on the android app, I know there are plenty of xposed tools that can uncheck/unpin certs on the fly. But at least for 1080p everything should be already more than doable on x86 alone.

[Varstahl]

I'll copy my tests out of #224 in here, as it might be useful later on:

---

After a few tests [with operatingSystemName parameter to catalog/GetPlaybackResources API call, ndr.], selecting anything other than Windows (and a few other written below), with this specific capitalization, reduces the number of videos available for playback. If, for example, we select Linux, it hides all the high movie resolution streams (that are unplayable anyway, in our case), but along with it takes a few of the high resolution TV series that would be playable. Using the Windows parameter, seems to restore all video qualities.

With a bit of data mining, and several tests later, I found the following:

operatingSystemName	Max. playable	Max. available
Unknown	1088x464
Android	1088x464
BlackBerry	1088x464
Chrome OS	1088x464
Firefox OS	1088x464
Google TV	1088x464
iOS	1088x464
Linux	1088x464
Mac OS X	1088x464	1920x816
Mac OS PPC	1088x464
Windows	1088x464	1920x816
Windows Phone	1088x464
Windows RT	1088x464	1920x816
Xbox One	1088x464

As far as movies go, Mac OS X, Windows and Windows RT have the full 1920 width available, although at least on my Win7 machine, only streams up to 1088x464 were playable. Anything higher would result in audio and a freezed frame. All streams reported as H264.

[…]

---

I think I noticed it depending on a few parameter combinations, some resolutions are added or removed to the list, but nothing higher than 1088 will ever play. I also noticed that HLS is also supported, but it returns AVC1 encoded videos. IS.A fails with a Could not open / parse mpdURL, uncertain if that's because I might be lacking some specific codecs.

[mirh]

Seen? It wasn't that hard after all 🙃 (I wonder if hls is what the official applications use then, or if there's still even more to discover)

AVC1's just normal plain H.264 anyway...

[Varstahl]

Seen? It wasn't that hard after all

To do what? Playback still doesn't work. Neither DASH nor HLS, MPD nor M3U8, AVC1 or not plays above 1088x464. And since I'm a complete noob when it comes to video streaming technologies, I have no idea how to analyze the issue.

I've just poked and prodded to solve a different problem, but we're nowhere near where we should be, for the moment.

Edit: ERROR: AddOnLog: InputStream Adaptive: DecodeVideo: kNoKey for key <KEY> At least now we also have something to relate the problem to.

[jgimness]

@Varstahl Thanks for your investigation. I'm actually more interested in finding out WHAT is available in UltraHD vs. actually being able to play content in UHD (I'm a data geek). Are you able to get anything out of the API that shows UHD "possible" content? I've tried many device types to no avail ...

[Varstahl]

The only surefire way is to: A. Obtain a valid apk and reverse engineer that B. Find if there's a possibility to sniff the traffic and reverse the protocol (generally this is my preferred route)

I've spent the past 8 hours setting up everything for a MitM attack, and I mostly succeded except for the fact that I came late to the party. Samsung has been bashed for insufficient security, and apprently they now took things ultra safely. I can sniff all the traffic no problem, but when it comes to HTTPS it has a reject by default policy, even in the browser. So, I either find a way to download/modify/sideload browser/amazon apks, or I need a way to inject a root certificate onto my TV.

It's now 4:30 am and I'm tired af, so I'm headed to bed.

Also, to elaborate better:

Are you able to get anything out of the API that shows UHD "possible" content?

Once upon a time @Sandmann79 found a way to do it for amazon.de, but several endpoints behaviour changed dramatically during the last few months. I don't think I can find out UHD streams with what I have at the moment. The only way would be to crack my damn TV open, but as far as the first attempt went, I fell short TLS. I can tcpdump and maybe bruteforce the TLS, but it's quite unrealistic.

Don't know, I'm sad, I'm going to bed.

Also, for added fun: Amazon removed the 4k section even on my freakin' 4k TV, listed as supported even on Amazon's website. Didn't test 4k content though, yet, so don't know about that. But even if I could previously find out of an URL API endpoint to discover 4k streams, now I just can't. Yay.

[mirh]

https://blog.netspi.com/four-ways-bypass-android-ssl-verification-certificate-pinning/ https://techblog.mediaservice.net/wp-content/uploads/2017/10/HackInBo-2017-Winter-Edition-Federico-Dotta-Advanced-mobile-penetration-testing-with-Brida-141017.pdf#page=13

[Varstahl]

Interesting, but Samsung 4k TVs run on Tizen. I can develop something for it and load it into the TV, but it's not Android. It's also a minimal distro, with no rooting possible (as of now). Also since I can't tell which are the endpoints for the samsung website, I'm not really able to download the package off the main repository, so even that didn't work out for me yet. Thanks for the reference though, I didn't know of Frida, that's an interesting read.

[mirh]

XZ premium 4K compatibility broke with Pie, and it seemingly never had playready to begin with. amazon broke compatibility with standard android

So my last, definitive, eventual, guess is just that they are performing stupid server-side checks, and nothing more than L1 should be required.. "high-end-security" wise at least.

[Varstahl]

I can tell you that they are doing a number of client/server side checks. Different codecs, lists, streams, and whatnot may or may not be displayed, and may or may not be served. Even with Widevine no more than L1 should be needed (for tv series at least), except that movies won't go higher than 480p (L3?). And since a lot of the checks and the "device" parameters are server/client side assessed, but in the most obscure way possible, I still can't find a way to wrap my head around the issue.

As far as HD movies go, Chrome is (possibly?) able to unlock the streams (I didn't actually check the stream contents yet so it might just pop up the HD and have a 480p quality, for all I know). But IS.A and most other browsers aren't able to decode them. Multiple Widevine licenses? I have no idea, and every time I see a change on Amazon's backend I die a little.

Hell, I'm even using Vivaldi on a daily basis, which is basically an "Opera" skin on top of the Blink/Webkit, and it still doesn't work. PrimeVideo on Vivaldi only works if you spoof the UA to match Opera, not even Chrome.

I have so many questions, and not enough answers… I just keep hating Amazon every day a little more.

[mirh]

I get the "HD light" (whatever it might actually entail) on firefox on windows.. And as I reported above, if you spoof enough the user agent, even on android you can basically get the same.

So.. to be honest, now that we solved the 4K "standards" conundrum, I think the only mystery left is why (just like with netflix tbh) 720p/1080p requires L1, while on "desktops" you can get away with the lowest security level

[Varstahl]

After you said that on Firefox 1080p ran find I got curious and installed a brand new portable version to dedicate to reverse engineering. At a glance, these are my findings:

• Firefox is using Widevine CDM 4.10.1196.0-win-x64, grabbing it from here: https://redirector.gvt1.com/edgedl/widevine-cdm/4.10.1196.0-win-x64.zip (apparently it's for Edge?);
• There are new APIs exposed that weren't there in the last two revisions, such as /gp/video/api/storefrontEnrichItems. Might reduce load times dramatically, but I'll have to look deeper at it;
• Firefox is reportedly showing as supporting 1080p HD streams on Win7 x64. I'm fairly sure that is correct, but I'll double check with the actual stream CDN location.

So, yeah, contrary to popular belief FHD might (could?) actually be obtained under Kodi, unless there's something weird going on with the Widevine library itself.

I'll analyze the handshakes and whatnot, maybe we can sort this out once and for all.

[Varstahl]

Kodi decided to do “The right thing”™, so now I can't debug fucking anything coming out of it. None of the systems they listed to bypass certificate validation works. Fucking geniuses, the lot of them. What am I supposed to do now, revert to Alpha1?

This day started badly and it's gonna end poorly…

Edit: Kodi is now ignoring system CAs and is shipping with a CA bundle inside systems/certs. If you guys need to proxy/bypass, you can put your self signed root CA in there. All values back to nominal.

[Varstahl]

Soo, after a few tests:

1. fetched the server certificate through a CAQ= query, and embedded it into the request. No dice.
2. took away our version of the Widevine CDM and used the Firefox/Edge, only to see nothing. No dice.
3. tried to modify the Widevine queries, but even though the result seem equal and perfectly fine, no dice.
4. plenty of InputStream Adaptive: DecodeVideo: kNoKey for key <KEY1> or InputStream Adaptive: DecryptSampleData: Decrypt failed with error: 2 for key <KEY1/KEY2>
5. judging by the MPDs the video streams have the exact same properties, but they just won't decode (again, google being google?)

We're going from nothing gained to everything lost. I'm exhausted, out of ideas, and extremely unfamiliar with Widevine DRM, no matter how much I try to tackle this problem I get nowhere every time. Short of bombarding @peak3d with questions and mpds, I've tried mostly everything. I don't think analysing/re-compiling IS.A would get me any further anyway.

[mirh]

(what's the difference between VOD and prime video?) Aaanyway as I have kind of hinted in #217...... Maybe we'd all have better to wait for inputstream to handle its last woes. Maybe check the last cookie PR they have there in the meantime idk..

[Varstahl]

Just like every other DRM it's a game of cat and mouse. There's a lot of money at stake, I doubt we'll ever have a proper solution that will last long enough. Which is still pissing me off, because I now have TWO different .DLLs that are able to decrypt the content, but nothing that works with either of them.

I'm half tempted to download the damned firefox sources and see if there's anything I can reverse engineer to retro-fit into IS.A, Kodi, or whatever the hell would make it work.

[mirh]

Lolwat. You aren't trying to crack it (fun fact) You are just trying to play to its rules.. except amazon cannot get their shit stable together.

[Varstahl]

Lolwat. You aren't trying to crack it

That's the theory, but then again what's the actuality of it? Like peak3d said, there are a number of hardware vendors (Google included) who have vested interests in keeping the most significant portions of the DRM out of public usage (buy our chromecast/pvr/whatever to see Netflix/Prime/YouTube in 4K!). Just like Google killed Edge, I've heard many reports of them throwing wrenches around.

I'm also fairly sure that what is going around Widevine is quite illegal, although it's slightly dressed up as "open" and "community friendly", whatever that means.

You are just trying to play to its rules.. except amazon cannot get their shit stable together.

Then let me ask you this, unless there are recent developments I'm not aware of, why is YouTube not working with high definition videos, using Widevine CDM? YouTube's owned by Google, and so is the Widevine DRM, and so is the Widevine CDM distribution. Then why doesn't it "work"?

Netflix's stuff is the only thing that works because they decided it should. Amazon seems to be using Google's suggested settings, and the world's suffering for it.

OH, but if you buy the 4k TV, or Google's Chromecast, or Amazon's Fireshtick or what have you, then everything suddenly works.

And if I'm to be true to myself, in all honesty, I don't even care for FHD or 4k. It's just that their keeping content locked up by blocking access to portions of implementation details makes me nauseous. And the more it goes on, the more it pisses me off.

I know this sounds quite the rant — because it is —, so let's cheer things up :tada:

[Varstahl]

Throwing these for future reference:

• https://archive.mozilla.org/pub/firefox/releases/64.0.2/
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Architecture_Overview.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Proxy_Integration.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Encryption_API.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_HLS.pdf
• https://integration.widevine.com/documentation/Widevine%20DRM
• https://integration.widevine.com/diagnostics

Other Widevine docs (unused atm, but while I'm at it…):

• https://storage.googleapis.com/wvdocs/Widevine_DRM_Getting_Started_Devices.pdf
• https://storage.googleapis.com/wvdocs/Chrome_EME_Changes_and_Best_Practices.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Partner_License_SDK.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Device_Provisioning_Models.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Getting_Started_Device_Management.pdf
• https://storage.googleapis.com/wvdocs/Widevine_CAS_Overview.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Using_Shaka_Packager_and_Player.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Encoding_and_Packaging.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Working_With_Foreign_Keys.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Getting_Started.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Android_Using_Key_Rotation.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Android_Vendor_Extensions.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Group_License.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Using_License_Renewal.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Key_Rotation_Encryption.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_License_Renewal_Overview.pdf
• https://storage.googleapis.com/wvdocs/Widevine_DRM_Accessing_the_CDM_Repository.pdf

Also, @mirh:

I think the only mystery left is why (just like with netflix tbh) 720p/1080p requires L1, while on "desktops" you can get away with the lowest security level

Your question is answered on page 13 of the pdf I linked above. I'll post an excerpt:

They can't enforce L1 on PCs, because there is a big number of users, including myself, who strongly oppose and refuse to buy any platform with TTX/TPM. I specifically nitpicked my Mobo, socket and processor to have no trusted computing on it. Although hardware vendors keep pushing it, market's nowhere near the point where they can enforce something like that. Here's hoping they never will.

[Varstahl]

So, I've found the tools and I started analysing the content exchanges between Kodi and the server, and Firefox and the server. It looks pretty much the same except for the little thing down here:

Kodi	Firefox
platformVerificationStatus	PLATFORM_UNVERIFIED	PLATFORM_SOFTWARE_VERIFIED

Which is to say the CDM/OEMCrypto module doesn't trust our code, so it won't work. Possibly nothing we do will ever fix that. Did I mention that CDM/OEMCrypto are also closed source, and independently controlled by Google? Yeah… I think I did…

What am I even supposed to look for… The CDM would most likely access the parent's hierarchy and declare "I don't trust this guy", is there even anything we can do?

Edit: added the proxy integration PDF in the references post above. Page 22, Verified Media Path (VMP).

Update: so, long story short, clientId in widevine requests is deprecated in favor of encryptedClientId, through keys obtained via a widevine request with a CAQ= request body. As soon as I find out if that's even possible through IS.A/CDM I'll let you know. Also, if anybody can help, I'd greatly appreciate.

[mirh]

Yes, again, I know L1 requires some whatever sort of hardware "secure predisposition", while L3 is "a program just like another". So this has to mean that, given my FF report, *to the very least* (notwithstanding recommendations) there must be an L3-secured HD stream. Therefore either L1 requirements on phones is bogus (at least for non-4K stuff that is), or the app is using different APIs that ends up offering you different streams.

As for your rant, again there isn't just DRM in the grand scheme of things. In fact, movies aside, I cannot think of once that youtube required me it. And I'm not sure what you are talking about that it doesn't work. Once people managed to understand "enable inputstream DASH" means they have to do it, everything seems perfect. If >1080p doesn't work it is just because (dejavu) inputstream doesn't support VP9 in a webm container. That seems to be the only different standard from netflix, for as much as they are concerned.

p.s. also which x86 system released in the past 5 years hasn't some sort of, if not TPM, at least TEE?

Which is to say the CDM/OEMCrypto module doesn't trust our code, so it won't work. Possibly nothing we do will ever fix that.

Ehrm.. Docs just say FF value means L3. Which I'm not sure how couldn't be available anywhere. UNVERIFIED doesn't even mean TAMPERED, it just entails the "verification code" isn't even plugged in. (to be fair, it seems like linux's CDM doesn't support it yet, but if netflix is working then VMP ain't the hard problem)

[Varstahl]

First of all, I ask everyone's forgiveness if I've been sounding relatively rude or just plain stupid. I've just realized that I'm increasingly fever-ish, and since it usually slowly turns me into a braindead grumpy cat I just want to reiterate that I'm not trying to attack anything/anyone, I'm just trying to solve this problem with increased frustration. So, if you'll excuse the tone I used so far, I'll try to explain myself better.

So this has to mean that to the very least there must be an L3-secured HD stream

Correct. Firefox streams are L3, as the license handshake I analysed suggests.

Therefore either L1 requirements on phones is bogus (at least for non-4K stuff that is), or the app is using different APIs that ends up offering you different streams.

I believe we provide android with the necessary native streams, but even if we were to pass them L3 protected H264 I'm unsure whether or not it could decrypt them. Imagining Android has all the codecs it needs, it's actually the OEM Decrypter, not even the WidevineCDM, that decides whether or not the content will be decrypted, with a whitelist based approach (afaik). It should be relatively easy to pass L3 encrypted content for any platform, previous studies suggest it won't matter anyway.

In fact, movies aside, I cannot think of once that youtube required me it

I was directly referencing the YouTube movie streaming issue, with problems decoding >480p with any Widevine CDMs. It was a bit ago, so my information might be outdated.

p.s. also which x86 system released in the past 5 years hasn't some sort of, if not TPM, at least TEE?

I have no idea, but I sure will hang tightly on my esa-cored ht-enabled TXT-free processor. I'm philosophically against any form of trusted computing, and no amounts of "necessary evil" or "inevitable doom" would convince me otherwise. On this, I'm strongly with Stallman.

Docs just say FF value means L3. Which I'm not sure how couldn't be available anywhere.

Correct, but part of the can/can't relies on CDMs that must be shipped/certified with binaries (roughly, not literally). This could absolutely not be our case, you're perfectly correct on the assumption. But then, again, both the Widevine CDM and the OEM Decrypter work with a whitelist based approach, and the extent of their "validation" is out of our scope. I know it works like this because it's in the specification, but they obviously omitted the details. I'm trying to analyse the docs to see if I can adapt our software to the problem we're facing.

UNVERIFIED doesn't even mean TAMPERED, it just entails the "verification code" isn't even plugged in

Exactly. The problem is that I actually requested and attached the server PSSH, but it doesn't work. So it's one of the following:

1. Just as for the linux version, the cdm we're currently using doesn't fully support VMP/client certification/whatever it is that it should.
2. Replacing the current IS.H wvcdm with Firefox's version should work, but doesn't because of reasons I'm not yet clear with.
3. wvcdm won't ever work with our custom software because we're not whitelisted with the decrypter.
4. The solution's simple as heck, but I can't see it because I'm increasingly feverish.

The good news is that I found quite a bit of docs, so at least I now can learn more about it and hopefully figure out what we/cdm/is.a are doing wrong. If I don't fall asleep on my desk due to the fever:thermometer:, that is :laughing:

P.S.: I've added a bunch of documentation on the post above, if you guys are curious.

[Varstahl]

~Just to be on the safe side, instead of hardcoding the PSSH keys I implemented an automatic fetch and then attached it to IS.A~. I'm also using Firefox/Edge WidevineCDM. It still doesn't encrypt the client information, so something is still off. I'm downloading FF's sources, maybe I can find some information there.

Edit: actually the answer may be a Widevine CryptoSession, Kodi ref., Android MediaDrm. Also added another couple links above.

Edit 2: after a day of building a proxy I realized while reading the documentation that the PSSH are embedded inside MPDs. It could be useful if we want to switch to HLS eventually, but it's just wasted code atm.

Edit 3: cryptosession might be an android-only thing. The solution might be EME?

Edit 4: tried to sniff Widevine API calls, turns out Widevine is not so keen on sharing and is protected against DLL injection on the target. Currently there are two possible reasons why it's not working:

1. wvcdm needs a cryptoproxy which is not yet implemented in IS.A
2. wvcdm actually checks if the parent executable is signed by a reputable CA

I'll try to recompile FF to check if the latter's the problem, in which case we're doomed.

[Varstahl]

I've been hammering at it for the past 5 days, recompiled everything recompilable and have discovered the following:

1. PLATFORM_UNVERIFIED does not prohibit 1080p reproduction
2. I still have no idea why IS.A fails and what's so different with Firefox

On the bright side, we now know we don't need VMP, so that's out of the way (at least I think, but _UNVERIFIED speaks loudly in 1080p). Also, I now have Firefox, Kodi and InputStream.Adaptive in a compilable (and working) form. If it's anything that can be fixed by coding, I'll get to the bottom of this, even if I have to lose my mind over it for another week or three.

[aers]

Update: so, long story short, clientId in widevine requests is deprecated in favor of encryptedClientId, through keys obtained via a widevine request with a CAQ= request body. As soon as I find out if that's even possible through IS.A/CDM I'll let you know. Also, if anybody can help, I'd greatly appreciate.

Encrypted client ID just required the server's service certificate to be loaded into the CDM, it will then automatically encrypt the client ID. This is used to protect the client ID from being in plaintext; the service cert is simply a RSA keypair generated by the service, allowing them to decrypt the client ID encrypted with it. "SetServiceCertificate" or something like that should be the function call.

Requesting the service certificate for any widevine service takes sending the bytes "08 04", which is probably CAQ= base64'd as you said.

As an aside VMP doesnt work at all without encrypted client IDs, since encrypting the client ID is used to hide the VMP data from being read by, well, you guys.

Also there was an earlier discussion about Amazon 4K. Amazon uses Playready for 4K on the majority of devices, even Android. They don't even serve regular 4K via Widevine, only 4K HDR. Playready is done using SL2000 certs (not SL3000 certs, aka Playready 4.0, which is only really supported by Windows 10 and maybe some very new devices right now). Those certs you found in the apk are generic test certs I believe.

PS: there's no difference in protocol for L1 vs L3; the level of your device is verified via a giant intermediate cert list that basically says whether or not your client is L1 or L3 based on some fields in the clientID. All ChromeCDM keys are L3, and Firefox uses ChromeCDM.

[aers]

Went to take a look at inputstream's CDM adapter. You want this:

void SetServerCertificate(uint32_t promise_id, const uint8_t* server_certificate_data, uint32_t server_certificate_data_size);

It needs to be called before generating the license request with the service cert.

[Varstahl]

the service cert is simply a RSA keypair generated by the service, allowing them to decrypt the client ID encrypted with it. "SetServiceCertificate" or something like that should be the function call.

I was unsure of the use of what Google calls PSSH, since even in the entirety of the documentation I read it wasn't really specified. I read back IS.A's source code and noticed there isn't any SSC calls, so the server certificates we provide are just never used for that. It seemed trivial enough, up until the point where the entire "promises" management needs to be implemented. I'll see what I can do.

Requesting the service certificate for any widevine service takes sending the bytes "08 04", which is probably CAQ= base64'd as you said.

I just checked and yes, that is correct.

As an aside VMP doesnt work at all without encrypted client IDs, since encrypting the client ID is used to hide the VMP data from being read by, well, you guys.

It's so true it hurts.

Also there was an earlier discussion about Amazon 4K. Amazon uses Playready for 4K on the majority of devices, even Android. They don't even serve regular 4K via Widevine, only 4K HDR. Playready is done using SL2000 certs (not SL3000 certs, aka Playready 4.0, which is only really supported by Windows 10 and maybe some very new devices right now).

Very interesting, I'll have to dig in the subject after I'm done with regular FHD.

PS: there's no difference in protocol for L1 vs L3; the level of your device is verified via a giant intermediate cert list that basically says whether or not your client is L1 or L3 based on some fields in the clientID. All ChromeCDM keys are L3, and Firefox uses ChromeCDM.

Yup, while analysing the sources I found the ChromeCDM as well inside Firefox, which also supports hardware proxying, if I read correctly.

@aers thank you so much for sharing the information, here's hoping I can put them to good use :)

[peak3d]

1.) Forcing the 08 04 request for android for encrypted messaging is done with setting private_mode attribute. There is no such attribute for libwidevine, if you guys find how it is triggered, pls. let me know.

Sure you can retrieve the server certificate by your own in addon and pass it using inputstream.adaptive listitem property, but I'm sure there is a way you don't have to do it this way.

2.) 4K streams are usually HEVC encoded. libWidevine does only decode H264 and VP9 (s/w) so it will be unlikely that 4K streams will be played using the internal decoder. There is only one chance for libwidevine + 4K: implement platform verification (as done in chromeos) to say that your system is "secure" and use libwidevine only for decryption. I doubt, that this will work in any way, because of many reasons (HDCP)

3.) would be worth a try setting inputstream.adaptive.license_flags=persistent_storage and watch traffic in kodi log

Edit: @Varstahl : regarding PSSH: https://w3c.github.io/encrypted-media/format-registry/initdata/cenc.html#common-system

[aers]

I was unsure of the use of what Google calls PSSH, since even in the entirety of the documentation I read it wasn't really specified. I read back IS.A's source code and noticed there isn't any SSC calls, so the server certificates we provide are just never used for that. It seemed trivial enough, up until the point where the entire "promises" management needs to be implemented. I'll see what I can do.

PSSH is called init data internally (PSSH is a legacy term). It is used to identify which key is being requested in a license request; the init data is effectively just straight up included in the license request message sent to the server.

You can probably see this if you use protobuf raw decode on the widevine messages, they are all protobufs ('08 04' is actually a valid protobuf message, and in widevine's protobuf definition its "request service cert").

1.) Forcing the 08 04 request for android for encrypted messaging is done with setting private_mode attribute. There is no such attribute for libwidevine, if you guys find how it is triggered, pls. let me know. Sure you can retrieve the server certificate by your own in addon and pass it using inputstream.adaptive listitem property, but I'm sure there is a way you don't have to do it this way.

ChromeCDM and the CDM adapter don't handle requests on their own. It's the responsibility of the player javascript to provide the service certificate to the CDM via setServerCertificate: https://developer.mozilla.org/en-US/docs/Web/API/MediaKeys/setServerCertificate

I don't know how this is implemented in the code inputstream uses, I'm not really familiar with any of this Kodi stuff, just widevine in general. But libwidevine dll from Chrome will not do any requests like this on its own.

2.) 4K streams are usually HEVC encoded. libWidevine does only decode H264 and VP9 (s/w) so it will be unlikely that 4K streams will be played using the internal decoder. There is only one chance for libwidevine + 4K: implement platform verification (as done in chromeos) to say that your system is "secure" and use libwidevine only for decryption. I doubt, that this will work in any way, because of many reasons (HDCP)

It is impossible to use ChromeCDM to decrypt 4K because no service will return a 4K license from a L3 request, and in addition all 4K licenses will have HW_ALL mode flagged, so you can't even use SW_DECRYPT mode in ChromeCDM.

Amazon licenses for ChromeCDM are flagged SW_DECODE which is why CDM can only decode them and output raw frames, and not decrypt them, btw.

[Varstahl]

Sure you can retrieve the server certificate by your own in addon and pass it using inputstream.adaptive listitem property, but I'm sure there is a way you don't have to do it this way.

Assuming the server certificate is the same one that's streaming through the MPD (and it should, given Google's specs), there is no need to fetch it differently. We would need to manually get it only if we were using HLS streams instead, just like Netflix. Since the PSSH is already parsed from within the stream it doesn't seem necessary. The only problem, in regard to VMP, is that the PSSH is not passed to the Widevine CDM through the SSC.

would be worth a try setting inputstream.adaptive.license_flags=persistent_storage and watch traffic in kodi log

Will do

It is impossible to use ChromeCDM to decrypt 4K because no service will return a 4K license from a L3 request, and in addition all 4K licenses will have HW_ALL mode flagged, so you can't even use SW_DECRYPT mode in ChromeCDM.

At the moment in the 4k department we can't even get the list of videos that support them. There was a small-ish chance when Amazon had the 4k section on its TV apps, but I didn't reverse engineer it in time, and now that it's gone is all up in the air. If we manage to make 4k videos available, I guess people with secure devices might still find use for it. After all, in theory, we're here to use the content we're already paying for, not breaking the DRM it's shackled by, although the two often seem one and the same nowadays…

[aers]

Amazon's MPD does not include service cert. Its really easy to check Chrome/Firefox's behavior, if its making the 0804 request to Amazon's widevine2license endpoint or whatever its called then its loading it separately.

PSSH is not!! service cert, PSSH is init data.

Netflix actually includes the cert in their manifest responses (aptly named "cert") to avoid doing multiple requests.

VMP is impossible to make work in Kodi without doing things that would get you in legal trouble with Google, but you can definitely do privacy mode just by loading the service cert.

At the moment in the 4k department we can't even get the list of videos that support them. There was a small-ish chance when Amazon had the 4k section on its TV apps, but I didn't reverse engineer it in time, and now that it's gone is all up in the air. If we manage to make 4k videos available, I guess people with secure devices might still find use for it. After all, in theory, we're here to use the content we're already paying for, not breaking the DRM it's shackled by, although the two often seem one and the same nowadays…

Theres no support for 4k on MPD endpoint, only 4k HDR, unless this changed in 1.5 years (okay maybe it changed but it has been that long since I investigated). You could feasibly support 4K HDR on android TV and other such things but Amazon can block this easy.

[peak3d]

1.) Forcing the 08 04 request for android for encrypted messaging is done with setting private_mode attribute. There is no such attribute for libwidevine, if you guys find how it is triggered, pls. let me know. Sure you can retrieve the server certificate by your own in addon and pass it using inputstream.adaptive listitem property, but I'm sure there is a way you don't have to do it this way.

ChromeCDM and the CDM adapter don't handle requests on their own. It's the responsibility of the player javascript to provide the service certificate to the CDM via setServerCertificate: https://developer.mozilla.org/en-US/docs/Web/API/MediaKeys/setServerCertificate

My own Chromium compilation logs every CDMAdapter call, I'm sure that Server Certificates are managed.

For Amazon initialization starts with reading a fileIo object, but I had no time so far to investigate what information is readthere. Inputstream.adaptive returns 0 byte from this read, Chromium provides some bytes.

I'm quite sure that if we know / understand what file is read at initialization time, that server certificate is requested automatically.

Edit: What Im 100% sure about is that 08 04 license challenge comes out of libwidevine before the real license challenge is provided. 08 04 is not provided using kodi / inputstream.adaptive, but from log traffic the only difference is the file.

[Varstahl]

Amazon's MPD does not include service cert. Its really easy to check Chrome/Firefox's behavior, if its making the 0804 request to Amazon's widevine2license endpoint or whatever its called then its loading it separately.

You're correct, I'm going stupid. For what it concerns Firefox though, it's not easy, because funnily enough, the behaviour is not reproduced when compiling from the sources. When Firefox is compiled and ran it doesn't encrypt the request and returns a PLATFORM_UNVERIFIED. So there's something missing from the picture.

In the grand scheme of things probably none of this matters, since Firefox is also able to decode FHD with no VMP. I'd be glad enough to figure out how to make 1080p play.

Theres no support for 4k on MPD

We are able to get a variety of streams, it's just that at the moment we don't know where to look.

My own Chromium compilation logs every CDMAdapter call, I'm sure that Server Certificates are managed.

I wonder if a compiled version of Chromium fetches the server certificate, because Firefox doesn't.

For Amazon initialization starts with reading a fileIo object, but I had no time so far to investigate what information is readthere. Inputstream.adaptive returns 0 byte from this read, Chromium provides some bytes.

I have a log of everything going out and coming into my machine, but I didn't find anything specific to RSA certificates. I can check back.

I'm quite sure that if we know / understand what file is read at initialization time, that server certificate is requested automatically.

I've been hammering at this from a while and I haven't figure out yet. I've got access to PrimeVideo.com, a few compilers, and traffic logs. If there's anything specific you want me to take a look at or provide you, just ask.

In the meanwhile I'll dive back into the logs. Although I could swear I checked everything and didn't see the server cert, unless it was hidden somewhere.

[aers]

When Firefox is compiled and ran it doesn't encrypt the request and returns a PLATFORM_UNVERIFIED. So there's something missing from the picture.

VMP is disabled since your files aren't signed so it probably doesn't bother to enable privacy mode, although it really still should use privacy mode... Chromium will act the same way, probably.

BTW what version of ChromeCDM are you using in Kodi? Version number of widevinecdm.dll

[peak3d]

@aers https://github.com/peak3d/inputstream.adaptive/blob/master/wvdecrypter/cdm/media/cdm/cdm_adapter.cc#L56

But not everything is implemented,