mkb79 / Audible

A(Sync) Interface for internal Audible API written in pure Python.
https://audible.readthedocs.io
GNU Affero General Public License v3.0
299 stars 36 forks source link

Solved: Decrypting frc cookie #42

Open mkb79 opened 3 years ago

mkb79 commented 3 years ago

I have this java script which encrypts a strigified JSON object.

Amazon apps using this string as the "frc" cookie. My app creates the frc cookie from custom bytes. Amazon still accepts this frc cookie. But I will be prepared for the future and build the frc cookie the same way as Amazon does.

I've tried to rebuild the java script to python

import base64
import hmac
import hashlib
import json
import gzip

from Crypto.Cipher import AES
from Crypto.Util.py3compat import bchr

def check_sig(pw, iv, data, sig):
    key = get_key_from_password(pw, b'HmacSHA256')
    new_sig = hmac.new(key, iv + data, hashlib.sha256).digest()
    new_sig = new_sig[:len(sig)]
    assert sig == new_sig, 'Signature check failed'

def decrypt_frc(pw, iv, data):
    key = get_key_from_password(pw, b"AES/CBC/PKCS7Padding")
    crypter = AES.new(key, AES.MODE_CBC, iv)
    decrypted_data = crypter.decrypt(data)
    return decrypted_data

def split_frc(frc):
    frc += (len(frc) % 4) * '='
    frc = base64.b64decode(frc)

    sig = frc[1:9]
    iv = frc[9:25]
    data = frc[25:]

    return sig, iv, data

def get_key_from_password(pw, salt):
    return hashlib.pbkdf2_hmac('sha1', pw, salt, 1000, 16)

def pkcs7unpad(msg, blocklen):
    assert len(msg) % blocklen == 0

    paddinglen = msg[-1]

    assert paddinglen > 0 and paddinglen <= blocklen, "Incorrect padding - Wrong key"
    assert msg[-paddinglen:] == bchr(paddinglen) * paddinglen, "Incorrect padding - Wrong key"

    return msg[:-paddinglen]

if __name__ == '__main__':
    frc_cookie = ''
    device_serial = ''
    device_serial = device_serial.encode()

    sig, iv, data = split_frc(frc_cookie)
    check_sig(device_serial, iv, data, sig)
    decrypted = decrypt_frc(device_serial, iv, data)
    decrypted = pkcs7unpad(decrypted, 16)
    decompressed = gzip.decompress(decrypted)
    decrypted_data = json.loads(decompressed)

Can someone with JAVA and Python knowledge take a look on my code if I have take a misstake?

mkb79 commented 3 years ago

Okay, I've found out how to decrypt the frc cookie myself. The 'pw' is the device serial.

mkb79 commented 3 years ago

Here is an example of the JSON data from a decrypted frc cookie

{
    "ApplicationVersion": "3.35.1",
    "DeviceOSVersion": "iOS/14.1",
    "ScreenWidthPixels": "375",
    "TimeZone": "+02:00",
    "ScreenHeightPixels": "812",
    "ApplicationName": "Audible",
    "DeviceJailbroken": false,
    "DeviceLanguage": "de-DE",
    "DeviceFingerprintTimestamp": "...",
    "ThirdPartyDeviceId": "A UUID4 string",
    "DeviceName": "iPhone",
    "Carrier": "..."
}
mkb79 commented 3 years ago

Here is my optimized code to encrypt and decrypt the frc cookie:

import base64
import gzip
import hashlib
import hmac
import json
import secrets
from io import BytesIO
from functools import partialmethod
from typing import Tuple, Union

from pyaes import AESModeOfOperationCBC, Encrypter, Decrypter

FRC_SIG_SALT: bytes = b'HmacSHA256'
FRC_AES_SALT: bytes = b'AES/CBC/PKCS7Padding'

class FrcCookieHelper:
    def __init__(self, password: str) -> None:
        self.password = password.encode()

    def _get_key(self, salt: bytes) -> bytes:
        return hashlib.pbkdf2_hmac('sha1', self.password, salt, 1000, 16)

    get_signature_key = partialmethod(_get_key, FRC_SIG_SALT)

    get_aes_key = partialmethod(_get_key, FRC_AES_SALT)

    @staticmethod
    def unpack(frc: str) -> Tuple[bytes, bytes, bytes]:
        pad = (4 - len(frc) % 4) * '='
        frc = BytesIO(base64.b64decode(frc+pad))
        frc.seek(1)  # the first byte is always 0, skip them
        return frc.read(8), frc.read(16), frc.read()  # sig, iv, data

    @staticmethod
    def pack(sig: bytes, iv: bytes, data: bytes) -> str:
        frc = b'\x00' + sig[:8] + iv[:16] + data
        frc = base64.b64encode(frc).strip(b'=')
        return frc.decode()

    def verify_signature(self, frc: str) -> bool:
        key = self.get_signature_key()
        sig, iv, data = self.unpack(frc)
        new_signature = hmac.new(key, iv + data, hashlib.sha256).digest()
        return sig == new_signature[:len(sig)]

    def decrypt(self, frc: str, verify_signature: bool = True) -> bytes:
        if verify_signature:
            self.verify_signature(frc)

        key = self.get_aes_key()
        sig, iv, data = self.unpack(frc)
        decrypter = Decrypter(AESModeOfOperationCBC(key, iv))
        decrypted = decrypter.feed(data) + decrypter.feed()
        decompressed = gzip.decompress(decrypted)
        return decompressed

    def encrypt(self, data: Union[str, dict]) -> str:
        if isinstance(data, dict):
            data = json.dumps(data, indent=2, separators=(',', ' : ')).encode()

        compressed = BytesIO()
        with gzip.GzipFile(fileobj=compressed, mode='wb', mtime=False) as f:
            f.write(data)
        compressed.seek(8)
        compressed.write(b'\x00\x13')
        compressed = compressed.getvalue()

        key = self.get_aes_key()
        iv = secrets.token_bytes(16)
        encrypter = Encrypter(AESModeOfOperationCBC(key, iv))
        encrypted = encrypter.feed(compressed) + encrypter.feed()

        key = self.get_signature_key()
        signature = hmac.new(key, iv + encrypted, hashlib.sha256).digest()

        return self.pack(signature, iv, encrypted)
eode commented 3 years ago

Hey, I don't have the knowledge of Amazon's code to verify or compare it, or to evaluate this -- but:

If both of these are true:

Or if this is:

You could use something like pyv8 or bond or (if you're using qt/webkit) evaluateJavaScript().

In any case, if you can get the native function to be callable from Python, then you'd have a pretty foolproof way of ensuring that your results match theirs.

Note: Inter-language frameworks can be a rabbit-hole. This isn't a request for you to use the options mentioned, I just thought this info might be interesting or useful.

mkb79 commented 3 years ago

Hi, thank you for your hint.

Life would much easier using Python and Java Script in one project.

But: I wrote my Python apps with Pythonista for iOS. That's the reason why I can't use Java Script in my Audible projects. And that's the same reason why I use native Python packages only as dependencies (or such package which are shipped with Pythonista).

Have Amazon changed encrypting the frc cookies?

mkb79 commented 1 year ago

Did an you ever figure out how to decrypt/encrypt the frc cookies?

Here is the working code to decrypt/encrypt a frc cookie. As far as I found out, the password is the device serial.

They seem to be outputting an frc string containing "/" I tried your code but it only produces "/" in the string. Any clue why that is?

What you mean with this? Have you an example?

mkb79 commented 1 year ago

@figamore

Here you can find a gist to register a Goodreads device using a frc cookie.

seidnerj commented 1 year ago

This is awesome work! Great job! Where did you get the original Java implementation from? I am looking at some other Amazon constructs and am looking for some hints.

Was this from the Android Audible app?

mkb79 commented 1 year ago

@seidnerj This was from another Android App which uses the Amazon login method. These App was not so much obfuscated as the Audible App. I reverse engineered the App and it gives me the code from my first post.

What you are looking for exactly? Maybe I can help.

seidnerj commented 1 year ago

Specifically there's a cookie called CES that I suspect has something to do with captcha authentication.

If it is not present/incorrect, even if the captcha is entered correctly you get an error message that's identical to the one you get when you enter an incorrect captcha.

mkb79 commented 1 year ago

Specifically there's a cookie called CES that I suspect has something to do with captcha authentication.

I've never heard of a CES cookie. Maybe it's a kind of CSRF cookie that the server sends to the client and just needs to be sent back?! In which app or Amazon service is this cookie required?

seidnerj commented 1 year ago

Sorry, I meant it is a field in the login form, not a cookie. It is in the Amazon shopping app/mobile web and only appears during a captcha challenge.

It is indeed similar to the CSRF in the sense that it is sent from the server and posted back but for some reason if you try to login programmatically it seems that the value for CES sent from the server and then reposted is incorrect.

seidnerj commented 1 year ago

It is worth noting that I noticed that sometimes, in the actual html file (not the posted value), the metadata1 field value appears in the form of ape:RUNkSVRlQ3M6<rest of base64 encoded value> and only after decoding the base64 encoded portion it results in the recognizable form of ECdITeCs:<encrypted data>

Now back to the subject at hand - here is an example of the CES value:

ape:QUNTX1RDamRCblV3ZVRuaG1jRDJmMzkvZndBQUFBQUFBQUFCYlhYbDNlZE9HT3NlOHJiQTRhWXBnS2JmU0YrNjVOYjdlTXoyc3VSUkRzQzRZQzNYZkxXVjJZQUdFN0N2SWNDZ2hkZ0U2dmhJN0NwWkhVSmFhY1VJQzk2UU4yUXNMN0RwWjRCbDJQRHp3bFFTSXJYdHh2L2JhRk93MDM1QXNaeStNUDJIcjdxd0g5NzhEci9iLzRzNU5HTUdLVDVVMTVxUkpUZHhEUXcrNk9VQk5OSUw3WVl2NVRvVFM4RVppdHQvM3NrcU02STBuOWVYSmw2QVoxempJWnBaSDkxSmZaWlllMXlHL0NLMnA5WEdoV2R4Nm9xS2pGbnFSc21jSVJENHcrWkxNUHp0MUhoZEZ2WHhqcTdBNUZOSnNscHlUTmFkaVlYR1FHVGhGbHprdkRvR0Jwc3dHY2g2QzJaM3dqTGkvbjFldzI1clJQY29wdWJpTnNKZmNDdkwzQnhXMnpiazUydz0=

The part after "ape:" is base64 encoded, after decoding it you get something that looks similar to what is in the metadata1 field (after the "ECdITeCs:" prefix):

ACS_TCjdBnUweTnhmcD2f39/fwAAAAAAAAABbXXl3edOGOse8rbA4aYpgKbfSF+65Nb7eMz2suRRDsC4YC3XfLWV2YAGE7CvIcCghdgE6vhI7CpZHUJaacUIC96QN2QsL7DpZ4Bl2PDzwlQSIrXtxv/baFOw035AsZy+MP2Hr7qwH978Dr/b/4s5NGMGKT5U15qRJTdxDQw+6OUBNNIL7YYv5ToTS8EZitt/3skqM6I0n9eXJl6AZ1zjIZpZH91JfZZYe1yG/CK2p9XGhWdx6oqKjFnqRsmcIRD4w+ZLMPzt1HhdFvXxjq7A5FNJslpyTNadiYXGQGThFlzkvDoGBpswGch6C2Z3wjLi/n1ew25rRPcopubiNsJfcCvL3BxW2zbk52w=

I tried decoding it using the same method used for decoding the encrypted potion of metadata1 but it didn't work unfortunately, failing due to "'utf-8' codec can't decode byte".

mkb79 commented 1 year ago

I doesn’t know anything about the „ape:“ prefix. But from the name I would think first on „Authenticated Permutation-Based Encryption for Lightweight Cryptography (APE)“?! But I‘m not sure.

seidnerj commented 1 year ago

I looked it up, maybe - either way I haven't been able to make progress there for time being 🤷‍♂️

mkb79 commented 1 year ago

I would also like to take a closer look. Can I also use the Amazon iOS Shopping App for this? Or does it work differently there?

seidnerj commented 1 year ago

Yes, thanks a lot! You'd need to get it to issue you with a captcha challenge though, otherwise this field isn't going to be present. I think an easier way is to get this through the mobile web version of amazon.com. For the most part, the Amazon app is simply a wrapper for the mobile website.

mkb79 commented 1 year ago

Okay. Now I have to figure out how to force a captcha prompt. I‘m think I must disable 2FA?!

seidnerj commented 1 year ago

If you start a session with no cookies at all, then type the wrong password a couple of times, you usually get captcha 🙂

seidnerj commented 1 year ago

I'm not sure about having to disable 2FA, in my case it sometimes sends an email with an OTP, so I guess the 2FA is enabled but I still get a captcha if I do what I suggested in my previous post.

seidnerj commented 1 year ago

Hey, I don't have the knowledge of Amazon's code to verify or compare it, or to evaluate this -- but:

If both of these are true:

  • amazon updates their function for authentication enough that keeping python synchronized is a pain

  • the function that they use is atomic in javascript (in the sense that it can be used as a regular function call)

Or if this is:

  • You want to test your function's output to verify it matches Amazon's output

You could use something like pyv8 or bond or (if you're using qt/webkit) evaluateJavaScript().

In any case, if you can get the native function to be callable from Python, then you'd have a pretty foolproof way of ensuring that your results match theirs.

Note: Inter-language frameworks can be a rabbit-hole. This isn't a request for you to use the options mentioned, I just thought this info might be interesting or useful.

Pyv8 is no longer maintained, luckily it was forked by Cloudare and is still being actively maintained. Under project name "Stpyv8", see here: https://github.com/cloudflare/stpyv8

mkb79 commented 1 year ago

@seidnerj When I POST to https://www.amazon.com/ap/signin there are some ape: values in the body too.

openid.return_to had the value ape:aHR0cHM6Ly93d3cuYW1hem9uLmNvbS8/X2VuY29kaW5nPVVURjgmcmVmXz1uYXZtX2hkcl9zaWduaW4=. prevRID had the value ape:NkVFS0RZNk1BS0pNMTY4MVg0NlA=.

So my Safari browser must calculate this fields or they are provided in the previous html response. If they are calculated my browser must receive a js script before to know how the must do this. So you can take a look in the js files you receive.

seidnerj commented 1 year ago

Yes, there are more form values that appear with this same prefix, thing is, in all other cases (such as the ones you've listed below), the data after that prefix its simply base64 encoded and when you decode it you get plain text that's not encrypted. For example, for the values you provided:

aHR0cHM6Ly93d3cuYW1hem9uLmNvbS8/X2VuY29kaW5nPVVURjgmcmVmXz1uYXZtX2hkcl9zaWduaW4= decodes to "https://www.amazon.com/?_encoding=UTF8&ref_=navm_hdr_signin"

Whereas NkVFS0RZNk1BS0pNMTY4MVg0NlA=decodes to "6EEKDY6MAKJM1681X46P", which if you look closely, is essentially the previous request ID.

In both cases, the "payload" is simply base64 encoded but not encrypted. In the case of the CES value, it seems the data is encrypted prior to the base64 encoding.

github-actions[bot] commented 10 months ago

This issue has not been updated for a while and will be closed soon.