mkb79 / audible-cli

A command line interface for audible package. With the cli you can download your Audible books, cover, chapter files.
GNU Affero General Public License v3.0
423 stars 45 forks source link

Atmos Download (Future Feature Request… if possible) #155

Open CoolJoe72 opened 11 months ago

CoolJoe72 commented 11 months ago

So it would be awesome to enable the Audible Dolby Atmos option for downloading I'm not sure if something would need to be added to the api or even what codec format they are using but it seems it can be downloaded in the iOS app and some select android devices.

Digging around the api found this in /1.0/library/B0C66LN3JW but it showed the standard stereo available_codecs

"asset_details": [ "is_spatial": true,"name": "Dolby"]

This is all new to me and I'm not even sure what I'm looking for.

mkb79 commented 11 months ago

I'm absolutely low on time this week. But you can try out yourself if you can download and decrypt the atmos audio.

You have to make a POST request to https://api.audible.de/1.0/content/B004UVB7KC/licenserequest. B004UVB7KC must be replaced with your ASIN.

The body for the request above is

{
  "use_adaptive_bit_rate" : true,
  "supported_media_features" : {
    "codecs" : [
      "mp4a.40.2",
      "mp4a.40.42",
      "ec+3"
    ],
    "drm_types" : [
      "Mpeg",
      "Hls",
      "HlsCmaf",
      "FairPlay"
    ]
  },
  "response_groups" : "content_reference,chapter_info,last_position_heard,pdf_url,certificate",
  "consumption_type" : "Streaming",
  "spatial" : true
}

I does not know why the iOS Audible app makes a Streaming request but it download and saves the audio file.

Edit: The iOS Audible app makes a download request. It looks so

{
  "quality" : "High",
  "response_groups" : "chapter_info,content_reference,last_position_heard,pdf_url",
  "consumption_type" : "Download",
  "supported_media_features" : {
    "codecs" : [
      "mp4a.40.2",
      "mp4a.40.42",
      "ec+3"
    ],
    "drm_types" : [
      "Mpeg",
      "Adrm",
      "FairPlay"
    ]
  },
  "spatial" : true
}

Edit: In both cases a m3u8 playlist file is provided for downloading using the FairPlay format. After that, the client made a request to https://api.audible.de/1.0/content/B004UVB7KC/drmlicense with a licenseChallenge body to receive the license. I does not know how to decrypt FairPlay DRM.

CoolJoe72 commented 11 months ago

Well it was worth a shot, and thank you all your time you put into it. I wasn't expecting a response other than yeah that would be neat and maybe marked as a future option once it was figured out in like 6 months or more.

Looks like I'll have to do some research now I know what direction to go in, when I get some free time.

Thank you.

devnoname120 commented 10 months ago

Audible for Android also supports Dolby Atmos. Does it also download a FairPlay format? Because to the best of my knowledge FairPlay is only available on Apple devices so it may be using another (weaker?) DRM.

mkb79 commented 10 months ago

@devnoname120

The iOS and Android Audible apps are requesting the same API to make a license request. Therefore, you just have to find out which request body the Android app uses.

For iOS this body is sent:

{
  "quality" : "High",
  "response_groups" : "chapter_info,content_reference,last_position_heard,pdf_url",
  "consumption_type" : "Download",
  "supported_media_features" : {
    "codecs" : [
      "mp4a.40.2",
      "mp4a.40.42",
      "ec+3"
    ],
    "drm_types" : [
      "Mpeg",
      "Adrm",
      "FairPlay"
    ]
  },
  "spatial" : true
}

I do not have an Android device to decrypt the HTTPS traffic sent from the Audible app to the API. So you can using the script below to play around to make a licenserequest and test some codecs and drm_types and check which response the API will give:

import json

from audible import Authenticator, Client

auth_file_path = "..."  # FILL OUT
asin = "..."  # FILL OUT

auth = Authenticator.from_file(auth_file_path)

with Client(auth) as client:

    body = {
        "quality": "High",
        "response_groups": "chapter_info,content_reference,last_position_heard,pdf_url",
        "consumption_type": "Download",
        "supported_media_features": 
            {
                "codecs": [
                    "mp4a.40.2",
                    "mp4a.40.42",
                    "ec+3"
                ],
                "drm_types": [
                    "Mpeg",
                    "Adrm",
                    "FairPlay"
                ]
            },
        "spatial": True
    }

    lr = client.post(
        f"content/{asin}/licenserequest",
        body=body,
    )
    print(json.dumps(lr, indent=4))

Known drm_types are Mpeg, PlayReady, Hls, Dash, FairPlay, Widevine, HlsCmaf, Adrm. And known codecs are ec+3, ac-4, mp4a.40.42, mp4a.40.2. Maybe these helps a little bit?!

mkb79 commented 10 months ago

If you remove FairPlay from supported drm_types it will download the book in AAXC (Adrm mode)! I does not know if it using Dolby Atmos. But it uses the mp4a.40.2 codec in booth cases. Tried it with asin 3844535306 and german marketplace .

Edit: Oh it seams these audiobook is not available in Dolby Atmos.

Edit: Tried it again with an Atmos title. It will only download with ec+3 and only as FairPlay protected title. Maybe you have more luck.

devnoname120 commented 10 months ago

@mkb79 I can't test right now. Did you try with ac-4 and WideVine? This seems to me to be the combo that will most likely work for Android content.

mkb79 commented 10 months ago

I've tried it with the codecs ec+3 and ac-4. Drm_types where set to Mpeg, PlayReady, Hls, Dash, Widevine, HlsCmaf, Adrm (only removing FairPlay). This will give me a HTTP 404 error with the message: Unable to retrieve asset details from Sable(AssetInfos), for marketplaceId:AN7V1F1VY261K, asin:B0BGYDYQ38, acr:null, skuLite:OR_ORIG_002267, version:LATEST, aaaClientId:ApolloEnv:AudibleApiExternalRouterService/EU/Prod. Maybe B0BGYDYQ38 is not available on Android devices in Dolby Atmos or they are using some other codecs/drm_types there.

mkb79 commented 10 months ago

I've documented some new API endpoints related to the FairPlay DRM.

Steps to download Dolby Atmos titles:

  1. The client makes a licenserequest.
  2. It receives the response containing a URI to a m3u8 file
  3. The client makes multiple GET requests to this URI with the User-Agent: AppleCoreMedia/1.0.0.20G75 (iPhone; U; CPU OS 16_6 like Mac OS X; de_de) and receives content from type `application/vnd.apple.mpegurl
  4. Client requests the FairPlay certificate.
  5. Client makes a GET request to dpm.demdex.net and receives JSON content (Edit: these step may be not iOS App related)
  6. Client make a POST request to drmlicense endpoint
  7. The client request build a new URI from the URI from 2 and the filename taken from the m3u8 file and receives the audio/mp4 file.

Edit: I can’t replay step 6 currently. I does not know how to build the license challenge. If I follow the other steps from above, leaving step 6 out, I receive a 403 status error. So step 6 must be the import part. Maybe the challenge is build using the cert from point 4 and other things?

mkb79 commented 10 months ago

@devnoname120 I’ve found out how I can extract the uri from the m3u8 file and download the Dolby Atmos mp4 file. Now I've to find out how to decrypt these file.

devnoname120 commented 10 months ago

@mkb79 What's the output of file dolby_atmos_file.mp4?

Note: step 5 is probably irrelevant as dpm.demdex.net is an endpoint used by Adobe Experience Platform Identity Service.

mkb79 commented 10 months ago

The m3u8 playlist contains the mp4 file location. With these information you can build the correct URI to the mp4 file. These file must be downloaded using a special Range header specify the byte range like bytes=0-1368458706 for asin B0BGYDYQ38. These audiobook has a size of 1368458706 bytes. If you forgot the Range header I've got an 403 error. So this header is mandatory.

After downloading you have a mp4 file which is encrypted via SAMPLE-AES. Now you need the key for decryption.

devnoname120 commented 10 months ago

Here is the full licenserequest JSON body definition on Android:

{
  "supported_media_features" : {
    "drm_types": [
      "adrm",
      "hls",
      "play_ready",
      "mpeg",
      "dash",
      "widevine"
    ],
    "codecs": [
       "mp4a.40.2", // AAC_LC
       "mp4a.40.42", // XHE_AAC
       "ec+3", // EC_PLUS_3
       "ac-4", // AC_4
    ],
    "chapter_titles_type": "flat/tree",
  },
  "spatial" : true, // true/false,
  "consumption_type" : "streaming/download",
  "rights_validation": "ownership/radio/aycl",
  "quality" : "low/normal/high/extreme",
  "version": "[version]",
  "acr": "CR![id_of_28_characters]", // Some kind of id for license?
  "use_adaptive_bit_rate": true, // true/false
  "playback_start_ms": 123,
  "playback_end_ms":  456,
  "response_groups" : "content_reference,chapter_info,pdf_url,last_position_heard,ad_insertion",
  "file_version": "[version]"
}

Not sure whether it differs from the one you see on iOS or not. Just posting this for my future reference I'll dig more into the Atmos stuff on Android.

mkb79 commented 10 months ago

Thank you very much. That seams a licenserequest for streaming purposes. use_adaptive_bit_rate is typically for streaming requests. acr is the amazon content reference. This value is unique for each book/asin/codec?/version combination.

Important for me is the response for a download license request for an Dolby Atmos book.

devnoname120 commented 10 months ago

Unfortunately I don't have an Android device that supports Dolby Atmos… Neither do I own a Dolby Atmos book.

devnoname120 commented 10 months ago

@mkb79 Can you retry with this user agent?

Audible, Android, 3.58.0, samsung, SM-S906B, g0sxeea, 13, 1.0, WIFI

This corresponds to a Samsung Galaxy S22 (it has native Dolby Atmos support).

I cobbled up this user agent by looking at how the Audible app constructs it and filling it with the information I was able to find on the internet. I'm not 100% sure I didn't make any mistakes while building it though.

mkb79 commented 10 months ago

@devnoname120 Specifying an User-Agent makes no different.

But I'm checked the downloaded book B0BGYDYQ38 with MediaInfo

General
Complete name                            : audio.mp4
Format                                   : MPEG-4
Format profile                           : Base Media / Version 1
Codec ID                                 : mp41 (iso8/isom/mp41/dash/cmfc)
File size                                : 1.27 GiB
Duration                                 : 3 h 57 min
Overall bit rate mode                    : Constant
Overall bit rate                         : 769 kb/s
Encoded date                             : 2023-03-22 12:22:54 UTC
Tagged date                              : 2023-03-22 12:22:54 UTC

Audio
ID                                       : 1
Format                                   : E-AC-3
Format/Info                              : Enhanced AC-3
Commercial name                          : Dolby Digital Plus
Codec ID                                 : enca / ec-3
Duration                                 : 3 h 57 min
Bit rate mode                            : Constant
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Compression mode                         : Lossy
Service kind                             : Complete Main
Encoded date                             : 2023-03-22 12:22:54 UTC
Tagged date                              : 2023-03-22 12:22:54 UTC
Encryption                               : Encrypted

So downloading Dolby Atmos titles is no problem. But decrypting FPS is the next step. I know how I can receive the FairPlay cert. The drmlicense request will then receive the license which can be used for decryption.

devnoname120 commented 10 months ago

@mkb79 I'm not familiar with FairPlay but it's a tough nut to crack. I'll try to trick the Audible app into thinking that my phone supports Dolby Atmos and dump the resulting HTTP requests. That will be for another time though!

devnoname120 commented 10 months ago

For my future reference here is the list of available Dolby Atmos audiobooks: https://www.audible.com/public-collections/1998b1ba-07e8-470f-8581-f97365772fe0

Mbucari commented 9 months ago

@devnoname120 @mkb79

Widevine is only available on android devices. For you to be able to request Widevine DRM, your audible client must be registered as an android device. Currently, audible-cli only registers as an iPhone. To register as an Android device, use the following registration body:

Android Registration Body Change the registration body to the following: ```Python body = { "requested_token_type": [ "bearer", "mac_dms", "website_cookies", "store_authentication_cookie", ], "cookies": {"website_cookies": [], "domain": f".amazon.{domain}"}, "registration_data": { "domain": "DeviceLegacy", "app_version": "141028", "device_serial": serial, "device_type": "A10KISP2GWF0E4", "device_name": ( "%FIRST_NAME%%FIRST_NAME_POSSESSIVE_STRING%%DUPE_" "STRATEGY_1ST%Audible for Android" ), "os_version": "google/sdk_gphone64_x86_64/emu64xa:14/UPB5.230623.003/10615560:userdebug/dev-keys", "software_version": "130050002", "device_model": "sdk_gphone64_x86_64", "app_name": "com.audible.application", }, "auth_data": { "use_global_authentication": "true", "client_id": build_client_id(serial), "authorization_code": authorization_code, "code_verifier": code_verifier.decode(), "code_algorithm": "SHA-256", "client_domain": "DeviceLegacy", }, "requested_extensions": ["device_info", "customer_info"], } ```

I don't own an android Device that supports Dolby Atmos, but I was able to modify the Audible apk to allow downloading Atmos files. You can download it here.

Steps to download Dolby Atmos titles using Zero-G as an example:

  1. The client makes a licenserequest. a. Request URL: https://api.audible.com/1.0/content/B07K4VYQ5X/licenserequest b. Request Body

      {
        "supported_media_features": {
          "drm_types": [
            "Widevine",
            "Adrm",
            "Mpeg"
          ],
          "codecs": [
            "mp4a.40.2",
            "mp4a.40.42",
            "ec+3",
            "ac-4"
          ],
          "chapter_titles_type": "Tree"
        },
        "spatial": true,
        "consumption_type": "Download",
        "quality": "High",
        "response_groups": "content_reference,chapter_info,pdf_url,ad_insertion"
      }
  2. It receives the response containing a URI to a Dash MPD file

    Example MPD File Zero-G MPD File ```Xml AAAAsXBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAAJESECe96GDiepAsn9qaoEPE/BESEFvn1UjnJquHS2LOtGVV88cSEBwnhnUCfvJndrBf6z7afIsaB0F1ZGlibGUiUGNpZDoNCko3M29ZT0o2a0N5ZjJwcWdROFQ4RVE9PSxXK2ZWU09jbXE0ZExZczYwWlZYenh3PT0sSENlR2RRSis4bWQyc0YvclB0cDhpdz09 AAAAVHBzc2gBAAAAmgTweZhAQoarkuZb4IhflQAAAAMnvehg4nqQLJ/amqBDxPwRW+fVSOcmq4dLYs60ZVXzxxwnhnUCfvJndrBf6z7afIsAAAAA ../../../../base/or_orig_000434/47096871/cenc/g1/or_orig_000434_48_320-ac4.mp4?ss_sec=20&use_token_based_signing=true ```
  3. Client makes a GET request for the first Initialization bytes of the file. (Found in the MPD file. Value is range='0-1117' in the example.) These bytes are the mp4 file's ftyp and moov boxes.

  4. Client make a POST request to drmlicense endpoint Body:

    {
      "consumption_type": "Download",
      "drm_type": "Widevine",
      "licenseChallenge": "..."
    }

    licenseChallenge is the Base64 encoded bytes returned from [ExoMediaDrm.KeyRequest.getData()](https://exoplayer.dev/doc/reference/com/google/android/exoplayer2/drm/ExoMediaDrm.KeyRequest.html#getData())

  5. Client downloads the rest of the mp4 file.

I don't have any widevine experience, but a good place to start reverse engineering Audible's implementation is in com.audible.widevinecdm.WidevineL3CdmFactory.

mkb79 commented 9 months ago

@Mbucari

Thank you very much for sharing your findings. Since I don't have an Android device, I unfortunately couldn't find out the exact registration body. This will help a lot.

Now we have two possible approaches (Widevine or FairPlay) to decrypt Atmos titles. Maybe some of them is successful.

devnoname120 commented 9 months ago

Hey @Mbucari and thank you for the payloads. I didn't update you guys in this thread but in the meantime I did a pretty thorough reverse engineering of the Audible Android app. I was also able to defeat the Amazon MAP certificate pinning and insert my own mitmproxy CA inside their hardcoded base64 + gzipped BKS bundle of certificates.

I updated my local fork of audible-cli in order to support Android auth, but (at least on my side) they use a different format of certificates (not PEM) for payload signatures and I haven't fixed that part entirely yet. I additionally dumped a L3 Widevine CDM so I should be ready to decrypt the actual resulting Dolby Atmos payload — unless there are more protections on top of Widevine.

I wanted to do a PR or at least a complete PoC before updating you, but to avoid duplicate work it I could maybe release my findings before doing a PoC when I get the chance to. On Oct 16, 2023, 23:52 +0200, mkb79 @.***>, wrote:

@Mbucari Thank you very much for sharing your findings. Since I don't have an Android device, I unfortunately couldn't find out the exact registration body. This will help a lot. Now we have two possible approaches (Widevine or FairPlay) to decrypt Atmos titles. Maybe some of them is successful. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

Mbucari commented 9 months ago

Hey @Mbucari and thank you for the payloads.

You're welcome @devnoname120. FYI, my patched apk removed cert pinning so you can use Http Toolkit to get all https traffic.

I updated my local fork of audible-cli in order to support Android auth, but (at least on my side) they use a different format of certificates (not PEM) for payload signatures and I haven't fixed that part entirely yet.

I forgot to mention this. On Android the private key is formatted like so (Asn.1 values)

PrivateKeyInfo SEQUENCE (3 elem)
  version Version INTEGER 0
  privateKeyAlgorithm AlgorithmIdentifier SEQUENCE (2 elem)
    algorithm OBJECT IDENTIFIER 1.2.840.113549.1.1.1 rsaEncryption (PKCS #1)
    parameters ANY NULL
  privateKey PrivateKey OCTET STRING
    SEQUENCE (9 elem)
      INTEGER 0
      INTEGER (RSA Modulus)
      INTEGER (RSA Public Exponent)
      INTEGER (RSA D)
      INTEGER (RSA P)
      INTEGER (RSA Q)
      INTEGER (RSA DP)
      INTEGER (RSA DQ)
      INTEGER (RSA InverseQ)

The PrivateKey octet string is equivalent to the PEM delivered by iPhone registration.

I additionally dumped a L3 Widevine CDM so I should be ready to decrypt the actual resulting Dolby Atmos payload — unless there are more protections on top of Widevine. I wanted to do a PR or at least a complete PoC before updating you, but to avoid duplicate work it I could maybe release my findings before doing a PoC when I get the chance to.

At the moment I have no idea what any of that means, but it sounds very impressive! Do you have some links to widevine documentation that could help explain this? My searches only yielded high-level info which seems pretty useless for reversing.

Mbucari commented 9 months ago

@devnoname120 I really wish I could be more helpful with the encryption key, but I don't have python code for it.

You can use this JavaScript parser to decode the base64 and get the integer values: https://lapo.it/asn1js/

And you can see my c# Asn.1 decoder here: https://github.com/rmcrackan/AudibleApi/blob/a630d6f04b2840d68b532a782eab3f46ec14aac0/AudibleApi/Cryptography/PrivateKey.cs#L54C1-L54C1

devnoname120 commented 8 months ago

Quick update: I made good progress. I have a PoC that authenticates, gets the Atmos content license object, extracts the pssh from the MPD, sets up a new Widevine L3 session with my dumped CDM keys, gets a new challenge from the CDM, and sends a Widevine L3 license request to Audible with that challenge.

Currently Audible refuses to grant my L3 license request. I double-checked my license request and it looks correct — I think that the CDM keys that I extracted are just not approved by Audible. Next step is getting my hand on a rooted physical Android physical in order to extract new Widevine CDM keys and move on to the next step.

Mbucari commented 8 months ago

@devnoname120 I have a couple of old android devices that I could root and, with your instruction, dump CDM keys. Hell, I'd be willing to gift one of them to you if you'd like.


Can you explain what these keys are? If you dumped a working CDM and we use them in our audible decryptors, won't they just be revoked? And when they revoked, would we have to buy a new device to get a new, valid CDM?

devnoname120 commented 8 months ago

@Mbucari That would be great, thanks! Getting my hand on CDM keys would be enough. These keys would only be for my personal use. In order to get the keys you would need to follow these instructions: https://github.com/lollolong/dumper

Can you explain what these keys are?

They are used to simulate a Widevine L3 device. From that simulated device we create a challenge that is sent to Audible's server, which in turn returns personalized decryption keys that the simulator deciphers and then returns back to us. I'm not in my sharpest mental state right now so I hope it makes sense.

If you dumped a working CDM and we use them in our audible decryptors, won't they just be revoked? And when they revoked, would we have to buy a new device to get a new, valid CDM?

You're right, if we included dumped keys they would be banned pretty fast. Users would have to dump their own keys or use a Widevine proxy service such as https://getwvkeys.cc/ or one based on pywidevine's remote CDM feature. I don't know any good proxies yet

devnoname120 commented 7 months ago

@Mbucari Do you have any updates on the Widevine CDM dumps? 🙏

Mbucari commented 7 months ago

Do you have any updates on the Widevine CDM dumps? 🙏

Not yet, sorry. Thanks for the reminder. I'll get to it this week.

Mbucari commented 7 months ago

@devnoname120 I sent you a CDM dump. I tried using to get the decryption keys, but I got the following response from Audible:

{
  "message": "Widevine license denied due to: UNTRUSTED_DEVICE_CERTIFICATE",
  "reason": "UntrustedDeviceCertificate"
}

I got that CDM from the web browser at bitmovin. I got another CDM from audible.com in the web browser, and that one also resulted in an UNTRUSTED_DEVICE_CERTIFICATE error. I tried using the dumper to get a CDM from the Audible app, but it only found the "device_info" and not the "private_key", so I couldn't dump it.

devnoname120 commented 7 months ago

@Mbucari I have the exact same error when using the CDM that you sent me.

Interestingly when I use the CDM that I dumped from the Android emulator, Audible returns a 500 HTTP status code and the following response:

{"message":"There was an internal server error."}

I didn't try with a Chrome CDM though.

Can you give us the output of the following steps?

  1. Install the DRM Info app, open it, and take a screenshot.
  2. Install the liboemcryptodisabler Magisk module and reboot. This forces the use of Widevine L3 rather than Widevine L1. Take another screenshot from the DRM info app.
  3. Use the dumper again on the Audible app and report back. Send me by email the device_client_id_blob and/or device_private_key.

I wonder if Audible only accepts devices that actually support Dolby Atmos when requesting ec+3.

Edit: Maybe Audible actually requires Widevine L1 for Dolby Atmos audiobooks.

Mbucari commented 7 months ago

@devnoname120

Original DRM Info

Screenshots ![Screenshot_20231208-161008](https://github.com/mkb79/audible-cli/assets/37587114/cfdb0626-d2c9-48da-9761-d78a90622419) ![Screenshot_20231208-161121](https://github.com/mkb79/audible-cli/assets/37587114/9d97d4b6-5961-4fa3-b5c1-e90af0e4c392)

DRM Info after installing liboemcryptodisabler

Screenshots ![Screenshot_20231208-162615](https://github.com/mkb79/audible-cli/assets/37587114/0fa3f935-7036-4267-801b-2961c8e246dc) ![Screenshot_20231208-162625](https://github.com/mkb79/audible-cli/assets/37587114/0b7488a9-e951-4811-8e98-932979087b8e)

I've emailed you the new CDM which I retrieved from the audible.com web player.

Some more Audible Widevine info

I've been doing some more reverse engineering work on the app, and I don't think this approach will work. Here's what I've learned so far:

Audible uses a native library to manage its CDM. com.audible.widevinecdm.WidevineCdmNative is the java native interface. Here's that interface:

Here's the WidevineCdmNative interface ```Java public static native WidevineCdmNative create(EventListener eventListener, String productName, String deviceType, String deviceUniqueId, String cdmDirectory, String cryptoDirectory); public native Status closeSession(byte[] sessionId); public native byte[] createSession(SessionType sessionType); public native byte[] decrypt(byte[] sessionId, byte[] inBuffer, byte[] iv, byte[] keyId, int i); public native void destroy(); public native Status forceRemoveStoredSession(byte[] sessionId); public native Status generateLicenseRequest(byte[] sessionId, byte[] initData); public native long getExpiration(byte[] sessionId); public native HashMap getKeyStatuses(byte[] sessionId); public native ProvisioningRequest getProvisioningRequest(); public native Status handleProvisioningResponse(byte[] response); public native boolean isProvisioned(); public native String[] listStoredLicenses(); public native Status loadStoredSession(byte[] sessionId); public native Status removeProvisioning(); public native Status removeStoredSession(byte[] sessionId); public native Status updateLicense(byte[] sessionId, byte[] response); public native String version(); ```

Audible creates a singleton instance of WidevineCdmNative with the following parameters:

productName = "Amazon Audible" deviceType = "A10KISP2GWF0E4" deviceUniqueId = [Device ID used when the device was registered] cdmDirectory = "/data/user/0/com.audible.application/files/cdm" cryptoDirectory = "/data/user/0/com.audible.application/files/crypto"

When playing media, Audible first checks if a device key has already been provisioned by calling isProvisioned(). If not It generates a ExoMediaDrm.ProvisionRequest using the values retrieved from getProvisioningRequest(). It then calls MediaDrmCallback.executeProvisionRequest, and the response data is used to call handleProvisioningResponse() which stores stores the device-unique credentials in cdmDirectory. See MediaDrm.

To do a license request, a new DRM session is created with createSession(), and a call is made to generateLicenseRequest() with the session ID and the pssh data as initData. Then updateLicense() is called (but I don't know where data for the byte[] response argument comes from) and then decrypt() is called repeatedly to decrypt the media. Again, I don't know where the decrypt() come from.

My theory is that everything from createSession() through decrypt() is just standard CDM operations, so if we get out hands on the actual CDM used by Audible then we can use existing tools to generate license requests and decrypt the data. But first we need the device key.

PROBLEM: I have no experience reverse engineering on unix, and we need to reverse engineer how the native audible shared libraries encrypt and store the CDM so we can decrypt and use it. It uses three libraries that are packed in the .apk:

Ideally we will be able to reverse engineer getProvisioningRequest() so we can make our own requests to the provisioning server for the device-unique credentials. But even if I manage to do that, I have no idea what to do with those credentials.

devnoname120 commented 7 months ago

@Mbucari With regard to step 3. I wanted you to try again getting the CDM from the Audible app (not the web browser). With the module I made you install it should make sure that the Audible app uses L3 rather than L1, which may be the reason why you couldn't dump it earlier. Could you try that?

I'm looking at the rest of your comment.

devnoname120 commented 7 months ago

@Mbucari By the way for comparison here is what I see in the valid license request sent by my ZTE Axon 7 (which has official Dolby Atmos support) from the Audible app (v3.58 iirc or maybe v3.59, it's an “old” dump):

Client → Additional Info:

company_name = "Audible, Inc."
model_name = "A10KISP2GWF0E4"
oem_crypto_build_information = "OEMCrypto Level3 Code 22435 Oct 29 2021 17:43:38"
oem_crypto_security_patch_level = "0"
product_name = "Amazon Audible"
widevine_cdm_version = "15.3.0"

It's Widevine L3 so it's good news at least.

Client → Root of Trust:

Owner =  "widevine.com"
System ID = "22435"
SerialNumber = Base64: [REDACTED]
Created = "2023-10-08T23:28:23.0000000+00:00 (Sun, 08 Oct 2023 23:28:23 GMT)"

Given the Created date I think it's a device/registration root certificate that is either provisioned by Amazon when using the app or just generated by the app itself. So I guess we need to extract that certificate in order to sign our requests with that one.

Mbucari commented 7 months ago

@devnoname120 I tried dumping the audible app cdm, but dumper could not find the private key (even after following you instructions to disable L1). I plan to retry using different app versions, but I won't have time until after the new year.

devnoname120 commented 5 months ago

I won't be able to dedicate time to this in the foreseeable future. If anyone wants my jadx project with the reverse engineered Audible Android app feel free to ask me. I documented a lot of stuff except the native Widevine Cdm itself (i.e. dynamically-loaded .so Widevine libraries).

As I see it the easiest way forward would be to:

szescxz commented 3 months ago

Poked around this a little bit and can confirm that Audible's license server is indeed restricted to its own app-level CDMs if it's an L3 without VMP. This is an obvious sign that Audible might take actions once its CDMs get extracted and published to public (although it's just an L3).

Some tips for dynamic analysis (with Frida on Audible Android app version 3.73.0):

devnoname120 commented 3 months ago

@szescxz Yes I agree that extracting their CDMs and publishing them standalone wouldn't be a great idea. Likewise, extracting the private certificates/keys embedded in the CDMs (https://arxiv.org/pdf/2204.09298.pdf) and then publishing them online would probably not be risk-free. I don't know the track record of Amazon on these kinds of issues.

szescxz commented 3 months ago

extracting their CDMs and publishing them

By "extracting CDMs" I mean exactly extracting a valid DRM certificate / private key pair.


Anyway, if @mkb79 is still interested in implementing this, the path forward should be

* while ffmpeg can also do the job by supplying decryption_key, to my knowledge they don't support multiple keys yet. The audiobook I've tested only requires 1 key and will work with ffmpeg, but according to the sample MPD above, other audiobooks might require multiple keys to decrypt.

mkb79 commented 3 months ago

@szescxz I'm interesting in an implementation. But first, I have to prepare my audible and audible-cli packages. I'll have to support Android device registration and then downloading Atmos titles.

devnoname120 commented 3 months ago

@szescxz Were you able to decrypt the audiobook that you mention?

@mkb79 iirc I have a local branch somewhere with support for Android registration + MPD handling + ec+3 download (but no decryption). I can try to find it another day, hopefully it's still around on my computer.

devnoname120 commented 3 months ago

@szescxz I think you may be mixing things up. There is just one key for Widevine Content Protection in that MPD. The other one is for Microsoft PlayReady.

See here for the scheme IDs: https://dashif.org/identifiers/content_protection/

szescxz commented 3 months ago

@szescxz Were you able to decrypt the audiobook that you mention?

If you can

I can give you the decryption key.

There is just one key for Widevine Content Protection in that MPD.

Pasting the Widevine PSSH from the Zero-G manifest above

AAAAsXBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAAJESECe96GDiepAsn9qaoEPE/BESEFvn1UjnJquHS2LOtGVV88cSEBwnhnUCfvJndrBf6z7afIsaB0F1ZGlibGUiUGNpZDoNCko3M29ZT0o2a0N5ZjJwcWdROFQ4RVE9PSxXK2ZWU09jbXE0ZExZczYwWlZYenh3PT0sSENlR2RRSis4bWQyc0YvclB0cDhpdz09

to https://integration.widevine.com/diagnostics shows three key IDs. My sample only has one. Not sure where did it go wrong. I don't have access to that audiobook nor I'm willing to pay for anything so I cannot check myself.

mkb79 commented 3 months ago

@devnoname120

iirc I have a local branch somewhere with support for Android registration + MPD handling + ec+3 download (but no decryption). I can try to find it another day, hopefully it's still around on my computer.

I could now successfully register an Android device with your request body above. But I had to work on the device private key. The key, obtained after a registration does not start with -----BEGIN RSA PRIVATE KEY-----\n and end with \n-----END RSA PRIVATE KEY-----\n. Now I have to find out how the Python rsa package can read this cert. How do you have solved this?

devnoname120 commented 3 months ago

@szescxz My bad about the decryption keys then. Btw I found this as an alternative to widevine.com diagnostics: https://emarsden.github.io/pssh-box-wasm/decode/

Edit: my memory is shady but aren't those just to set up Widevine? Then the decryption key(s) are obtained from it, no?

Too bad that https://tools.axinom.com is registration-walled + requires a subscription now (but a trial is available iirc) because it has by far the best and most comprehensive tools to decode MPD/PSSH/DASH/Widevine stuff.

devnoname120 commented 3 months ago

@mkb79 I fixed this either by converting the key to the right format or changing the code of audible-cli so that it supports it (I don't remember which I did eventually). I will dig up and give you the code once I'm at my computer, probably tomorrow.

mkb79 commented 3 months ago

@mkb79 I fixed this either by converting the key to the right format or changing the code of audible-cli so that it supports it (I don't remember which I did eventually). I will dig up and give you the code once I'm at my computer, probably tomorrow.

That's great, thank you. In the meantime I'll try it myself.

devnoname120 commented 3 months ago

@mkb79 Just looked it up quickly before going to work and I have that:

import base64
from cryptography.hazmat.primitives import serialization

def base64_der_to_pkcs1(base64_key):
    der_private_key = base64.b64decode(base64_key)
    private_key = serialization.load_der_private_key(der_private_key, password=None)

    pkcs1_private_key = private_key.private_bytes(
        encoding=serialization.Encoding.PEM,
        format=serialization.PrivateFormat.TraditionalOpenSSL,
        encryption_algorithm=serialization.NoEncryption()
    )

    return pkcs1_private_key.decode('utf-8')#.replace('\n', '\\n')

base64_der_key = "MII[REDACTED VERY VERY LONG BASE64 STRING]="

# Convert to PEM format
pem_private_key = base64_der_to_pkcs1(base64_der_key)

# Print or save the PEM private key
print(pem_private_key)
devnoname120 commented 3 months ago

@mkb79 I will send the rest (authentication, etc.) tomorrow because I need to clean up personal data and right now I have to hop off to work.

szescxz commented 3 months ago

my memory is shady but aren't those just to set up Widevine?

@devnoname120 To put it in simple words, PSSH (which includes the key IDs) is included in the license request generated by the CDM, presented to license server as an identifier of the requested content. It's not encrypted in the license request, so you can see them after decoding.

As for the decryption part, quoting from https://w3c.github.io/encrypted-media/format-registry/stream/mp4.html#stream-format:

Each key is identified by a key ID and each encrypted sample is associated with the key ID of the key needed to decrypt it.

and

Streams may contain a mixture of encrypted and unencrypted samples.

devnoname120 commented 3 months ago

@szescxz OK just for info I found a screenshot of a real license request (from a device with hardware support for Dolby Atmos) that I decoded. Don't think it's useful for this part but so we have it in that thread for posterity:

image


For the decryption iirc when I looked into this I was using mp4decrypt (https://github.com/truedread/Bento4). I think it supports multiple keys but my memory is hazy (again) on that.