Lockbox to replace existing aes_cbc:encrypt(payload) & aes_cbc:decrypt(payload)

[nodecentral]

Hi

I’ve finally found some time to focus on this, with the goal to use lockbox to replace some Lua functions/libraries I don’t have access to.

The commands are as follows.

Encrypt

local aes_cbc, err = aes:new(key, nil, aes.cipher(128, 'cbc'), { iv = iv }, nil , 1) 
local ciphertext = aes_cbc:encrypt(payload)

Decrypt

local aes_cbc, err = aes:new(key, nil, aes.cipher(128, 'cbc'), { iv = iv }, nil, 0) 
local decrypted = aes_cbc:decrypt(data)

I’m still learning Lua so this is a bit of a stretch to me, but looking at the test, would the encrypt be something like the following?.

local String = require("string");
local Array = require("lockbox.util.array");
local Stream = require("lockbox.util.stream");
local CBCMode = require("lockbox.cipher.mode.cbc");
local ZeroPadding = require("lockbox.padding.zero");
local AES128Cipher = require("lockbox.cipher.aes128");

local mypayload = "TBC"
local mykey = "????????"
local myiv = "TBC"
local myciphertext = "????????"

local testVectors = {
        cipher = CBCMode.Cipher,
        decipher = CBCMode.Decipher,
        key = Array.fromHex(mykey),
        iv = Array.fromHex(myiv),
        plaintext = Array.fromHex(mypayload),
        ciphertext = Array.fromHex(myciphertext),
        padding = ZeroPadding
    }

for _, v in pairs(testVectors) do
    local cipher = v.cipher()
            .setKey(v.key)
            .setBlockCipher(AES128Cipher)
            .setPadding(v.padding);

    local cipherOutput = cipher
                        .init()
                        .update(Stream.fromArray(v.iv))
                        .update(Stream.fromArray(v.plaintext))
                        .finish()
                        .asHex();

    local decipher = v.decipher()
            .setKey(v.key)
            .setBlockCipher(AES128Cipher)
            .setPadding(v.padding);

    local plainOutput = decipher
                        .init()
                        .update(Stream.fromArray(v.iv))
                        .update(Stream.fromArray(v.ciphertext))
                        .finish()
                        .asHex();

    assert(cipherOutput == Array.toHex(v.ciphertext)
                , String.format("cipher failed!  expected(%s) got(%s)", Array.toHex(v.ciphertext), cipherOutput));

    assert(plainOutput == Array.toHex(v.plaintext)
                , String.format("decipher failed!  expected(%s) got(%s)", Array.toHex(v.plaintext), plainOutput));

end

[somesocks]

Hi @nodecentral,

At a glance your usage looks correct, but I think you may have a small error in your test code, it looks like testVectors should be an array of test objects, not a single object. So,

local testVectors = {
        {
                cipher = CBCMode.Cipher,
                decipher = CBCMode.Decipher,
                key = Array.fromHex(mykey),
                iv = Array.fromHex(myiv),
                plaintext = Array.fromHex(mypayload),
                ciphertext = Array.fromHex(myciphertext),
                padding = ZeroPadding
        }
}

Otherwise your for loop will be iterating over the test fixture properties (cipher, decipher, key, etc...)

[nodecentral]

Many thanks @somesocks

Starting with encryption, If I was wrap this into a function, and call it with the 3 elements I have?

local ciphertext = lockbox_aes_128_cbc_encrypt(mypayload, myiv, mykey)

What exactly am I looking to return to get the ciphertext I need ?

Then when it come to doing any decryption, what do I need to change/do differently ? (Sorry if these are basic questions)

[somesocks]

To build a working block cipher you need to know 6 things:

• the block cipher (in your case, AES 128)
• the mode of operation (in your case, CBC mode)
• the key as an array of bytes. FYI, this needs to be the same size as the block size (for AES128 that's 128 bits/16 bytes).
• the initialization vector, as a stream of bytes (you may need to decode this from hex, base64, utf8, etc.)
• the plain text, as a stream of bytes (decoded from hex, base64, utf8, etc.)
• a padding algorithm (plaintext for block ciphers needs to be padded so that the length in bytes is a multiple of the block size)

So, you're missing a few things in your function definition:

local ciphertext = lockbox_aes_128_cbc_encrypt(mypayload, myiv, mykey)

You have the block cipher (AES 128) and the mode (CBC), but you're missing the padding algorithm, the input format for the key/iv/plaintext, and the output format.

If I assume you're using zero padding, and passing all the inputs and outputs as hex strings, then I'd probably write it like this:

    local String = require("string");
    local Array = require("lockbox.util.array");
    local Stream = require("lockbox.util.stream");
    local CBCMode = require("lockbox.cipher.mode.cbc");
    local ZeroPadding = require("lockbox.padding.zero");
    local AES128Cipher = require("lockbox.cipher.aes128");

    function lockbox_aes_128_cbc_zero_padding_encrypt(plaintext, iv, key)

        local cipher = CBCMode.Cipher()
            .setBlockCipher(AES128Cipher)
            .setKey(Array.fromHex(key))
            .setPadding(ZeroPadding);

        local ciphertext = cipher
            .init()
            .update(Stream.fromHex(iv))
            .update(Stream.fromHex(plaintext))
            .finish()
            .asHex();

        return ciphertext;

    end

You could also skip the function, and drop it in as a one-liner:

    local ciphertext = CBCMode.Cipher()
        .setBlockCipher(AES128Cipher)
        .setKey(Array.fromHex(mykey))
        .setPadding(ZeroPadding);
        .init()
        .update(Stream.fromHex(myiv))
        .update(Stream.fromHex(mypayload))
        .finish()
        .asHex();

For deciphering, you need all the same parameters:

• the block cipher (still AES128)
• the mode of operation (still CBC mode)
• the key
• the iv
• the cipher text
• the padding algorithm

So, the decryption function would be something like:

    local String = require("string");
    local Array = require("lockbox.util.array");
    local Stream = require("lockbox.util.stream");
    local CBCMode = require("lockbox.cipher.mode.cbc");
    local ZeroPadding = require("lockbox.padding.zero");
    local AES128Cipher = require("lockbox.cipher.aes128");

    function lockbox_aes_128_cbc_zero_padding_decrypt(ciphertext, iv, key)

        local cipher = CBCMode.Decipher()
            .setBlockCipher(AES128Cipher)
            .setKey(Array.fromHex(key))
            .setPadding(ZeroPadding);

        local plaintext = cipher
            .init()
            .update(Stream.fromHex(iv))
            .update(Stream.fromHex(ciphertext))
            .finish()
            .asHex();

        return plaintext;

    end

[nodecentral]

Wow, thanks so much ! This is great stuff :-)

My challenge now is put it into action with my TV, as found a Lua script online to help pair my Pi device with it (to then act as a remote control).

1) To start that process off, I send a request to my TV to get a visible PIN number and a challenge key Via xml - example of both below..

PIN = 1234
Challenge_Key = 6Sj5RAitzqplQ860TviWLw==

2) To get the IV, it looks like I would normally need to b64 encode the challenge key, but I guess I can do HEX instead?

IV = 36536A35524169747A71706C51383630547669574C773D3D

3) Then with the IV, I can then create the key, which it seems to suggest I do this, however does this still work with a HEX value vs b64 ?

    local iv_vals = { iv:byte(1, -1) }
    local key_vals = {}

    for i = 1, 16, 4 do
        key_vals[ i ] = bit.band(bit.bnot(iv_vals[ i + 3 ]), 0xFF)
        key_vals[ i + 1 ] = bit.band(bit.bnot(iv_vals[ i + 2 ]), 0xFF)
        key_vals[ i + 2 ] = bit.band(bit.bnot(iv_vals[ i + 1 ]), 0xFF)
        key_vals[ i + 3 ] = bit.band(bit.bnot(iv_vals[ i ]), 0xFF)
    end

    local key = string.char(unpack(key_vals))

4) Next would be the hmac_key which seems to point to doing the following..

local hmac_key_mask = binascii.unhexlify('15C95AC2B08AA7EB4E228F811E34D04FA54BA7DCAC9879FA8ACDA3FC244F3854')
    local hmac_key_mask_vals = { hmac_key_mask:byte(1, -1) }
    local hmac_vals = {}

    for i = 1, 32, 4 do
        hmac_vals[ i ] = bit.bxor(hmac_key_mask_vals[ i ], iv_vals[ bit.band(i + 1, 0xF) + 1 ])
        hmac_vals[ i + 1 ] = bit.bxor(hmac_key_mask_vals[ i + 1 ], iv_vals[ bit.band(i + 2, 0xF) + 1 ])
        hmac_vals[ i + 2 ] = bit.bxor(hmac_key_mask_vals[ i + 2 ], iv_vals[ bit.band(i - 1, 0xF) + 1 ])
        hmac_vals[ i + 3 ] = bit.bxor(hmac_key_mask_vals[ i + 3 ], iv_vals[ bit.band(i, 0xF) + 1 ])
    end

    local hmac_key = string.char(unpack(hmac_vals))

Any advise you can give on the above to help me get the required values in the necessary format, would be greatly appreciated..

[somesocks]

@nodecentral

I took a look through the forum post you linked and, and it seems like there's a lot of confusion in general about how this auth exchange actually works. I think this code from this github thread comment is a good starting point: https://github.com/florianholzapfel/panasonic-viera/issues/9#issuecomment-580232555

Running through that code:

import binascii
import base64
import hmac, hashlib
from Crypto.Cipher import AES

# Example challenge (which is our IV)
iv = base64.b64decode("mUQdS7/RyJTMsiojPz9i1Q==")

^ this is not an initialization vector, this is the encryption key. They're using the wrong term here, which adds a lot of confusion.

# Get character codes from IV bytes
iv_vals = [ord(c) for c in iv]

# Initialise key character codes array
key_vals = [0] * 16

# Derive key from IV
i = 0
while i < 16:
    key_vals[i] = ~iv_vals[i + 3] & 0xFF
    key_vals[i + 1] = ~iv_vals[i + 2] & 0xFF
    key_vals[i + 2] = ~iv_vals[i + 1] & 0xFF
    key_vals[i + 3] = ~iv_vals[i] & 0xFF
    i += 4

^ you don't "derive" a key from an initialization vector. There should never be any relationship between an IV and an encryption key, they should be two independent random numbers. And, to me, this doesn't look like a key derivation either, this looks much more like an endian conversion, converting bytes from network-order to machine-order (Though you don't use a ~ operator for endian conversions, so that might be a bug). See https://en.wikipedia.org/wiki/Endianness

# Initialise HMAC key mask (taken from libtvconnect.so)
hmac_key_mask_vals = [ord(c) for c in binascii.unhexlify("15C95AC2B08AA7EB4E228F811E34D04FA54BA7DCAC9879FA8ACDA3FC244F3854")]

# Initialise HMAC key character codes array
hmac_vals = [0] * 32

# Calculate HMAC key using HMAC key mask and IV
i = 0
while i < 32:
    hmac_vals[i] = hmac_key_mask_vals[i] ^ iv_vals[(i + 2) & 0xF]
    hmac_vals[i + 1] = hmac_key_mask_vals[i + 1] ^ iv_vals[(i + 3) & 0xF]
    hmac_vals[i + 2] = hmac_key_mask_vals[i + 2] ^ iv_vals[i & 0xF]
    hmac_vals[i + 3] = hmac_key_mask_vals[i + 3] ^ iv_vals[(i + 1) & 0xF]
    i += 4

# Convert our HMAC key character codes to bytes
hmac_key = ''.join(chr(c) for c in hmac_vals)

# This is our plaintext SOAP argument for the pin code shown on the TV
authinfo = "<X_PinCode>4410</X_PinCode>"

^ the HMAC key initialization seems fine, as far as I can tell

# First 12 bytes are randomised, let's just set them to 0 because it doesn't matter
payload = "000000000000"

# The next 4 bytes contain the plaintext (SOAP arg) length in big endian
n = len(authinfo)
payload += chr(n >> 24)
payload += chr((n >> 16) & 0xFF)
payload += chr((n >> 8) & 0xFF)
payload += chr(n & 0xFF)

^ THIS is the IV. Those bytes should be randomized for every message sent, and in general, it very much matters that the IV is generated and used correctly. A null IV weakens the security of a CBC cipher, see https://en.wikipedia.org/wiki/Initialization_vector for more info Also, the IV should be a full block-size. It looks like panasonic is cheating a little, and using the last 4 bytes of the IV as space to add the message length, which helps protect against extension attacks, but might weaken the security of the cipher a little.

# Let's encrypt it with AES-CBC! We need to make sure we pad it to a multiple of 16 bytes beforehand
aes = AES.new(key, AES.MODE_CBC, iv)
ciphertext = aes.encrypt(pad(payload))

^ We need to make sure we pad it to a multiple of 16 bytes beforehand implies they're using zero-padding, as opposed to something like PKCS7. See https://en.wikipedia.org/wiki/Padding_(cryptography) for more info

[nodecentral]

Thanks again @somesocks , i’m not sure I’ve been able to follow everything you’ve said, but it’s so good to have another pair of eyes look at this.

Unfortunately I’m only familiar with Lua, so the language used for that other script, is not as easy to follow, but if I’ve understood you correctly, the challenge key that the TV sends back, I then convert to base64, is actually the encryption key now?

And when it comes to the IV value, that’s sounds like it will derived from what I’d been referring to before as thepayload, hence going forward , my hmac_key , will be created using a new IV than before..

If you’ll indulge me, here’s a side by side with the code you linked to earlier showing what I’ve done so far, and I’m still grateful for any help.

-- import binascii
-- import base64
-- import hmac, hashlib
-- from Crypto.Cipher import AES

local bit = require("bit")
local mime = require("mime")
local binascii = require("binascii")

-- # Example challenge (which is our IV)
-- iv = base64.b64decode("mUQdS7/RyJTMsiojPz9i1Q==")

local challenge_key = "iL9XqQOMfkFWz2rvh0Xm+w=="
local challenge_Key_unb64 = mime.unb64(challenge_key)
print (challenge_Key_unb64)

-- # Get character codes from IV bytes
-- iv_vals = [ord(c) for c in iv]

-- # Initialise key character codes array
-- key_vals = [0] * 16

-- # Derive key from IV
-- i = 0
-- while i < 16:
--     key_vals[i] = ~iv_vals[i + 3] & 0xFF
--     key_vals[i + 1] = ~iv_vals[i + 2] & 0xFF
--     key_vals[i + 2] = ~iv_vals[i + 1] & 0xFF
--     key_vals[i + 3] = ~iv_vals[i] & 0xFF
--     i += 4

-- # Convert our key character codes to bytes
-- key = ''.join(chr(c) for c in key_vals)

local challenge_Key_unb64 = "ˆ¿W©Œ~AVÏjï‡Eæû"
local challengekey_vals = { challenge_Key_unb64:byte(1, -1) }
local key_vals = {}

for i = 1, 16, 4 do
    key_vals[ i ] = bit.band(bit.bnot(challengekey_vals[ i + 3 ]), 0xFF)
    key_vals[ i + 1 ] = bit.band(bit.bnot(challengekey_vals[ i + 2 ]), 0xFF)
    key_vals[ i + 2 ] = bit.band(bit.bnot(challengekey_vals[ i + 1 ]), 0xFF)
    key_vals[ i + 3 ] = bit.band(bit.bnot(challengekey_vals[ i ]), 0xFF)
end

local key = string.char(unpack(key_vals))
print(key) -- "V¨@w¾sü•0©ºx     "

-- # Initialise HMAC key mask (taken from libtvconnect.so)
-- hmac_key_mask_vals = [ord(c) for c in binascii.unhexlify("15C95AC2B08AA7EB4E228F811E34D04FA54BA7DCAC9879FA8ACDA3FC244F3854")]

-- # Initialise HMAC key character codes array
-- hmac_vals = [0] * 32

-- # Calculate HMAC key using HMAC key mask and IV
-- i = 0
-- while i < 32:
--     hmac_vals[i] = hmac_key_mask_vals[i] ^ iv_vals[(i + 2) & 0xF]
--     hmac_vals[i + 1] = hmac_key_mask_vals[i + 1] ^ iv_vals[(i + 3) & 0xF]
--     hmac_vals[i + 2] = hmac_key_mask_vals[i + 2] ^ iv_vals[i & 0xF]
--     hmac_vals[i + 3] = hmac_key_mask_vals[i + 3] ^ iv_vals[(i + 1) & 0xF]
--     i += 4

-- # Convert our HMAC key character codes to bytes
-- hmac_key = ''.join(chr(c) for c in hmac_vals)

local challenge_Key_unb64 = "ˆ¿W©Œ~AVÏjï‡Eæû"
local challengekey_vals = { challenge_Key_unb64:byte(1, -1) }

local hmac_key_mask = binascii.unhexlify('15C95AC2B08AA7EB4E228F811E34D04FA54BA7DCAC9879FA8ACDA3FC244F3854')
local hmac_key_mask_vals = { hmac_key_mask:byte(1, -1) }
local hmac_vals = {}

for i = 1, 32, 4 do
    hmac_vals[i] = bit.bxor(hmac_key_mask_vals[ i ], challengekey_vals[ bit.band(i + 1, 0xF) + 1 ])
    hmac_vals[i+1] = bit.bxor(hmac_key_mask_vals[ i + 1 ], challengekey_vals[ bit.band(i + 2, 0xF) + 1 ])
    hmac_vals[i+2] = bit.bxor(hmac_key_mask_vals[ i + 2 ], challengekey_vals[ bit.band(i - 1, 0xF) + 1 ])
    hmac_vals[i+3] = bit.bxor(hmac_key_mask_vals[ i + 3 ], challengekey_vals[ bit.band(i, 0xF) + 1 ])
end

local hmac_key = string.char(unpack(hmac_vals))
print(hmac_key) --"B`Ò}Îˤg$ÍÙNøÏWòâ/cÒÙzvà"õ3´¿""

-- # This is our plaintext SOAP argument for the pin code shown on the TV
-- authinfo = "<X_PinCode>4410</X_PinCode>"

-- # First 12 bytes are randomised, let's just set them to 0 because it doesn't matter
-- payload = "000000000000"

-- # The next 4 bytes contain the plaintext (SOAP arg) length in big endian
-- n = len(authinfo)
-- payload += chr(n >> 24)
-- payload += chr((n >> 16) & 0xFF)
-- payload += chr((n >> 8) & 0xFF)
-- payload += chr(n & 0xFF)

-- # Now we concatenate our payload, which is starting at byte 17 of the payload
-- payload += authinfo

local payload = '000000000000' -- First 12 bytes are randomised
local pincode = "<X_PinCode>1234</X_PinCode>"  -- Next 4 bytes are from the pincode prompted by the TV
n = #pincode

payload = payload .. string.char(bit.band(bit.rshift(n, 24), 0xFF))
payload = payload .. string.char(bit.band(bit.rshift(n, 16), 0xFF))
payload = payload .. string.char(bit.band(bit.rshift(n, 8), 0xFF))
payload = payload .. string.char(bit.band(n, 0xFF))
payload = payload .. pincode

local iv = payload
print(iv) -- "0000000000001234"

-- # Let's encrypt it with AES-CBC! We need to make sure we pad it to a multiple of 16 bytes beforehand
-- aes = AES.new(key, AES.MODE_CBC, iv)
-- ciphertext = aes.encrypt(pad(payload))

-- # Calculate the HMAC-SHA-256 signature of our encrypted payload
-- sig = hmac.new(hmac_key, ciphertext, hashlib.sha256).digest()

-- # Concatenate the HMAC signature to the encrypted payload and base64 encode it, and we're done!
-- encrypted_payload = base64.b64encode(ciphertext + sig)

Which then leads us into what we do with your code and the AES-CBC encryption….

[somesocks]

Here's how I'd try to complete your code. I haven't tested it at all, so there may be some bugs in here:

local Array = require("lockbox.util.array");
local Stream = require("lockbox.util.stream");
local Base64 = require("lockbox.util.base64");

local CBCMode = require("lockbox.cipher.mode.cbc");
local ZeroPadding = require("lockbox.padding.zero");
local AES128Cipher = require("lockbox.cipher.aes128");

local HMAC = require("lockbox.mac.hmac");
local SHA2_256 = require("lockbox.digest.sha2_256");

-- # This is our plaintext SOAP argument for the pin code shown on the TV
-- authinfo = "<X_PinCode>4410</X_PinCode>"

-- # First 12 bytes are randomised, let's just set them to 0 because it doesn't matter
-- payload = "000000000000"

-- # The next 4 bytes contain the plaintext (SOAP arg) length in big endian
-- n = len(authinfo)
-- payload += chr(n >> 24)
-- payload += chr((n >> 16) & 0xFF)
-- payload += chr((n >> 8) & 0xFF)
-- payload += chr(n & 0xFF)

-- # Now we concatenate our payload, which is starting at byte 17 of the payload
-- payload += authinfo

local plaintext = "<X_PinCode>1234</X_PinCode>";
local plaintext_size = #plaintext;

local iv = "000000000000" -- should be randomized
iv =
    iv
    .. string.char(bit.band(bit.rshift(plaintext_size, 24), 0xFF))
    .. string.char(bit.band(bit.rshift(plaintext_size, 16), 0xFF))
    .. string.char(bit.band(bit.rshift(plaintext_size, 8), 0xFF))
    .. string.char(bit.band(plaintext_size, 0xFF));

-- # Let's encrypt it with AES-CBC! We need to make sure we pad it to a multiple of 16 bytes beforehand
-- aes = AES.new(key, AES.MODE_CBC, iv)
-- ciphertext = aes.encrypt(pad(payload))

local challenge_key_bytes = Base64.toArray(challenge_key)
local plaintext_bytes = Array.fromString(plaintext);
local iv_bytes = Array.fromString(iv)

local cipher = CBCMode.Cipher()
    .setBlockCipher(AES128Cipher)
    .setKey(challenge_key_bytes)
    .setPadding(ZeroPadding); -- its not clear what the `pad` function does, it might be zero padding, it might be something else.  I'm guessing zero padding.

local ciphertext_bytes = cipher
    .init()
    .update(iv_bytes)
    .update(plaintext_bytes)
    .finish()
    .asBytes();

-- # Calculate the HMAC-SHA-256 signature of our encrypted payload
-- sig = hmac.new(hmac_key, ciphertext, hashlib.sha256).digest()

local hmac_key_bytes = Array.fromString(hmac_key)

local hmac = HMAC()
    .setBlockSize(32) -- the HMAC key looks to be 32 bytes, so its probably a 32-byte block size
    .setDigest(SHA2_256)
    .setKey(hmac_key_bytes);

local sig_bytes = hmac  
    .init()
    .update(ciphertext_bytes)
    .finish()
    .asBytes();

-- # Concatenate the HMAC signature to the encrypted payload and base64 encode it, and we're done!
-- encrypted_payload = base64.b64encode(ciphertext + sig)

local encrypted_payload_bytes = Array.concat(ciphertext_bytes, sig_bytes);
local encrypted_payload_b64 = Base64.fromArray(encrypted_payload_bytes);

[somesocks]

@nodecentral I'm closing this issue due to inactivity, but if you did get it to work, it'd be great if you could post the code here for others to see