Documentation on 32 byte derived private EC key. A cry for help to the Gods

[Corne173]

I wish someone could explain or refer me to documentation that explains how OpenSSL generates thats 32 byte key derived from the EC private key.

I'm trying to automate the whole process of adding new devices to the cloud, generating keys for those devices, adding them etc. I've used the python Cryptography library that's installed with the rest of the google-cloud-api libraries. Documentation found here. It generates the private and public EC keys, source documentation found here I haven't been able to find much information on how that last part works, that openssl ec -in ec_private_device1.pem -noout -text part.

I've used a test platform, found here. I am able to connect and publish when I use the key generated using openssl, but not using my implementation. There must be some required arguments to HKDF that I do not know of.

from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.hkdf import HKDF

private_key = ec.generate_private_key(ec.SECP256R1()) #P-256
private_pem = private_key.private_bytes(
    encoding=serialization.Encoding.PEM,
    format=serialization.PrivateFormat.PKCS8,
    encryption_algorithm=serialization.NoEncryption()
)
with open('../ec_private.pem', 'wb') as filekey:
    filekey.write(private_pem)
print(private_pem.decode("utf-8"))

public_key = private_key.public_key()
public_pem = public_key.public_bytes(
   encoding=serialization.Encoding.PEM,
   format=serialization.PublicFormat.SubjectPublicKeyInfo)

with open('../ec_public.pem', 'wb') as filekey:
    filekey.write(public_pem)
print(public_pem.decode("utf-8"))

shared_key = private_key.exchange(
    ec.ECDH(), private_key.public_key())
# Perform key derivation.
derived_key = HKDF(
    algorithm=hashes.SHA256(),
    length=32,
    salt=None,
    info=None,
).derive(shared_key)

shared_hex_keys = list(map(hex,list(derived_key)))
with open("../shared_hex_keys.txt", "w") as file:
    data = ",".join(shared_hex_keys)
    file.write(data)
print(f"Device key:\n {data}")

[raphael-bmec-co]

This is how we do it with Google Cloud Functions

const ECKey = require('ec-key');
const crypto = require('crypto');

        // Generate the ES256 key pair.

        // Create the key generation class.
        let ecdh = crypto.createECDH('prime256v1');

        // generate the keys.
        ecdh.generateKeys();

        // Get the private key and format.
        let privateKey = ecdh.getPrivateKey()
            .toString("hex")  // Convert to string.
            .match(/.{1,2}/g)  // Split every byte (2 characters).
            .join(":");

        // Generate the key.
        let key = new ECKey({
            privateKey: ecdh.getPrivateKey(),
            publicKey: ecdh.getPublicKey(),
            curve: 'prime256v1'
        });

        // Get the public key.
        let publicKey = key.asPublicECKey().toString()

[Corne173]

Thank you for much for you quick response! This is exactly what I'm planning on doing but using Python. I started of using Dataflow (like how every how-to-tutorial recommends) but its so cripplingly expensive. Cloud functions are awesome (and dirt cheap) but pretty hands on.

I pretty sure I can conclude that my root certs for the ESP are valid since everything works fine when I use the key generated by OpenSSL from the terminal. Just to be clear, the 32 byte EC key that I want to generate is the private key that is used by the ESP. Does your JS implementation produce that?

[raphael-bmec-co]

Hi.

Yip that is the:

        // Get the private key and format.
        let privateKey = ecdh.getPrivateKey()
            .toString("hex")  // Convert to string.
            .match(/.{1,2}/g)  // Split every byte (2 characters).
            .join(":");

I won't go into details about how we get that to the ESP for security reasons but that is the string private key used in the ESP MQTT implementation.

Hope that helps.

[Corne173]

I got it working...after a very unnecessary deep dive into the documentation. Your code snippet gave me a good clue that it was much simpler than what I thought. Thank you once again for you help and quick response.

    from cryptography.hazmat.primitives.asymmetric import ec
    from cryptography.hazmat.primitives import serialization

    # create a private EC key
    private_key = ec.generate_private_key(ec.SECP256R1()) 
    private_pem = private_key.private_bytes(
        encoding=serialization.Encoding.PEM,
        format=serialization.PrivateFormat.TraditionalOpenSSL,
        encryption_algorithm=serialization.NoEncryption()
    )
    print(private_pem.decode("utf-8"))

    # get private key value
    private_key_bytes = bytes.fromhex(format(private_key.private_numbers().private_value, '064x')) #converts int to hex
    list_of_bytes = list(map(hex, list(private_key_bytes)))     # creates list of bytes
    ESP_private_key = ",".join(list_of_bytes)                   # comma separates each byte
    print(f"Device key:\n {ESP_private_key}\n")

    # generate public EC key from private key
    public_key = private_key.public_key()
    public_pem = public_key.public_bytes(
        encoding=serialization.Encoding.PEM,
        format=serialization.PublicFormat.SubjectPublicKeyInfo)
    print(public_pem.decode("utf-8"))