Mbed-TLS / mbedtls

An open source, portable, easy to use, readable and flexible TLS library, and reference implementation of the PSA Cryptography API. Releases are on a varying cadence, typically around 3 - 6 months between releases.
https://www.trustedfirmware.org/projects/mbed-tls/
Other
5.23k stars 2.56k forks source link

ECDSA signature - key/pubkey extraction with ugly approach #7205

Open MaJerle opened 1 year ago

MaJerle commented 1 year ago

Suggested enhancement

I was researching migration to mbedTLS and got stuck with ECDSA verification API, simply because it is not really clear how to handle things between PK, ECDSA and group modules. To make things working, access to private components was often required - unless I misused the API, which then it means that documentation is to be improved.

As you will see in the code example (that may be wrong, indeed - but result of verification was always OK), public key in PEM/DER format and signature in DER format work very well together - without accessing private components of the structure.

The rest - it is not the case.

Justification

I was struggling to get signature working, then I realized that ANS.1 type is a must in mbedTLS for ECDSA secp256r1 signature. To use the s|r type of signature (P1363), a pretty dirty way of working is required. Then, there is a public key, that could be in compressed mode (65-bytes for SECP256R1, with 0x04 as start byte) or in PEM, string-based mode.

Discussion started here: https://stackoverflow.com/questions/75635019/mbedtls-ecdsa-verification-fails/75641568?noredirect=1#comment133450813_75641568

Mbed TLS needs this because user should avoid accessing private structure members, that may change over the time.

Example

This will describe 2 types of public key storage and 2 types of signature combinations -> in total there must be way to do any combination, in my opinion.

Private key:

-----BEGIN PRIVATE KEY-----
MIGHAgEAMBMGByqGSM49AgEGCCqGSM49AwEHBG0wawIBAQQgqhA28jTY51Cs27SG
8FvanN1N8+Pss0QjlLqEP7ZZOHahRANCAARVQmzipHZmb+whhjXoKWWGDuInpDFH
wZc1c8TBjPIDP5P3gghIpM5Hq34SftzzOzGhH+7/ZagrWv+3kUHtOC0A
-----END PRIVATE KEY-----

Public key in DER format

-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEVUJs4qR2Zm/sIYY16Cllhg7iJ6Qx
R8GXNXPEwYzyAz+T94IISKTOR6t+En7c8zsxoR/u/2WoK1r/t5FB7TgtAA==
-----END PUBLIC KEY-----

Public key in binary format - uncompressed, 65 bytes

0x4,0x55,0x42,0x6c,0xe2,0xa4,0x76,0x66,0x6f,0xec,0x21,0x86,0x35,0xe8,0x29,0x65,0x86,0xe,0xe2,0x27,0xa4,0x31,0x47,0xc1,0x97,0x35,0x73,0xc4,0xc1,0x8c,0xf2,0x3,0x3f,0x93,0xf7,0x82,0x8,0x48,0xa4,0xce,0x47,0xab,0x7e,0x12,0x7e,0xdc,0xf3,0x3b,0x31,0xa1,0x1f,0xee,0xff,0x65,0xa8,0x2b,0x5a,0xff,0xb7,0x91,0x41,0xed,0x38,0x2d,0x0

Input data - raw bytes

0x54,0x68,0x69,0x73,0x20,0x69,0x73,0x20,0x6d,0x79,0x20,0x69,0x6e,0x70,0x75,0x74,0x20,0x64,0x61,0x74,0x61,0x20,0x30,0x31,0x32,0x33,0x34,0x35,0x36,0x37,0x38,0x39

SHA256 of input data

0xd5,0x7b,0x51,0x75,0x85,0xde,0xc0,0xdb,0x2f,0xa5,0xd2,0x67,0xb0,0x8b,0x48,0x70,0x68,0x8b,0x7c,0xa3,0xcc,0x42,0x69,0x9a,0x60,0x51,0x3c,0xc9,0xe1,0xdc,0xc,0x7c

Signature in DER format (size can vary depends on the BIGINT signed bit)

0x30,0x46,0x2,0x21,0x0,0xb1,0x71,0x54,0x3e,0xa3,0x9c,0xf,0x92,0x18,0x5a,0x7c,0x91,0x83,0xb4,0x6f,0x8,0xb1,0xc,0xdf,0xb3,0xef,0x26,0xab,0x85,0x47,0xdb,0xe2,0x63,0x3a,0x4,0x78,0xc6,0x2,0x21,0x0,0xe9,0x91,0x58,0x2c,0x21,0x29,0x9b,0xf2,0x37,0xb5,0x45,0x97,0xba,0x53,0x5c,0x4b,0x72,0xba,0xca,0x69,0xb0,0x8c,0x31,0x60,0xe0,0x35,0xbc,0xfc,0xfd,0xce,0xa2,0xa6

Signature in P1363 format

0xd2,0x56,0x93,0x53,0xab,0x8a,0xc2,0x5c,0xad,0x83,0x35,0xe6,0x30,0xaf,0x43,0x73,0xeb,0xa0,0x6b,0xf0,0x41,0x29,0xc0,0x20,0x7b,0x4a,0x8f,0xc2,0xdb,0x3e,0x5,0x9d,0x60,0x4f,0x90,0x16,0x26,0xb3,0xa0,0xbb,0x59,0xfc,0x8d,0x7b,0x26,0x32,0x57,0xaa,0x5b,0x9,0x90,0x50,0x2d,0x4a,0xf4,0x22,0x9e,0xc7,0x9,0x1e,0xfb,0x17,0xcc,0x46

MbedTLS tests under Windows environment

#include <stdio.h>
#include <string.h>
#include "windows.h"

#include "mbedtls_config.h"
#include "mbedtls/ecdsa.h"
#include "mbedtls/aes.h"
#include "mbedtls/sha256.h"
#include "mbedtls/pk.h"

/* Raw input data*/
const uint8_t data_raw_input[] = {
#include "keys/data_raw_input_array.txt"
};

/* SHA256 of raw input data */
const uint8_t data_raw_hash_digest[] = {
#include "keys/data_raw_hash_digest_array.txt"
};

/* Signature of hash of raw data with private key */
const uint8_t signature_der[] = {
#include "keys/signature_der_array.txt"
};
const uint8_t signature_p1363[] = {
#include "keys/signature_p1363_array.txt"
};

/* ECC public key - compressed binary */
const uint8_t ecc_public_key_compressed_bin[] = {
#include "keys/ecc_public_key_compressed_array.txt"
};

/* ECC public key - uncompressed binary */
const uint8_t ecc_public_key_uncompressed_bin[] = {
#include "keys/ecc_public_key_uncompressed_array.txt"
};

/* ECC public key full text */
const uint8_t ecc_public_key_text[] = {
#include "keys/public.ec.oneline.key"
};

/* ECC private key full text */
const uint8_t ecc_private_key_text[] = {
#include "keys/private.ec.oneline.key"
};

int
main(void) {
    volatile int res;
    mbedtls_ecdsa_context ecdsa_ctx;
    mbedtls_pk_context pubkey_ctx;

    /*
     * Public key cryphography playground shows different way of parsing
     * actual public key (DER format (string) or binary format) and 
     * different types of signature representation (DER or P1363).
     */

    /*
     * Read this SO post for details:
     *
     * https://stackoverflow.com/questions/75635019/mbedtls-ecdsa-verification-fails/75641568#
     */

    /* 
     * Public key format:   DER/PEM (String based public key, --- BEGIN PUBLIC KEY --- type of message)
     * Signature format :   DER     (70, 71 or 72 bytes long)
     */
    {
        /* Parse public key */
        mbedtls_pk_init(&pubkey_ctx);
        res = mbedtls_pk_parse_public_key(&pubkey_ctx, ecc_public_key_text, sizeof(ecc_public_key_text));
        printf("mbedtls_pk_parse_public_key: %d\r\n", res);

        /* Verify with DER native format support */
        res = mbedtls_pk_verify(&pubkey_ctx, MBEDTLS_MD_SHA256, data_raw_hash_digest, sizeof(data_raw_hash_digest),
                                signature_der, sizeof(signature_der));
        printf("mbedtls_pk_verify: %d\r\n", res);
        printf("-----\r\n");

        /* Free objects */
        mbedtls_pk_free(&pubkey_ctx);
    }

    /* 
     * Public key format:   binary  (65-bytes long, 0x04 as first byte)
     * Signature format :   DER     (70, 71 or 72 bytes long)
     */
    {
        printf("Public key - ECDSA verification with binary format\r\n");

        /* Parse public key */
        mbedtls_ecdsa_init(&ecdsa_ctx);
        mbedtls_ecp_group_load(&ecdsa_ctx.private_grp, MBEDTLS_ECP_DP_SECP256R1);
        res = mbedtls_ecp_point_read_binary(&ecdsa_ctx.private_grp, &ecdsa_ctx.private_Q,
                                            ecc_public_key_uncompressed_bin, sizeof(ecc_public_key_uncompressed_bin));
        printf("mbedtls_ecp_point_read_binary: %d\r\n", res);

        /* Verify with DER native format support */
        res = mbedtls_ecdsa_read_signature(&ecdsa_ctx, data_raw_hash_digest, sizeof(data_raw_hash_digest),
                                           signature_der, sizeof(signature_der));
        printf("mbedtls_ecdsa_read_signature: %d\r\n", res);
        printf("-----\r\n");
    }

    /* 
     * Public key format:   binary  (65-bytes long, 0x04 as first byte)
     * Signature format :   P1363   (64-bytes long, r|s)
     */
    {
#define SIGNATURE_LEN (sizeof(signature_p1363))

        /* Parse public key in binary format */
        printf("Parse public key in binary format\r\n");
        mbedtls_ecdsa_init(&ecdsa_ctx);
        mbedtls_ecp_group_load(&ecdsa_ctx.private_grp, MBEDTLS_ECP_DP_SECP256R1);
        res = mbedtls_ecp_point_read_binary(&ecdsa_ctx.private_grp, &ecdsa_ctx.private_Q,
                                            ecc_public_key_uncompressed_bin, sizeof(ecc_public_key_uncompressed_bin));
        printf("mbedtls_ecp_point_read_binary: %d\r\n", res);
        printf("-----\r\n");

        /* Manually structure R and S components */
        printf("Parse signature in P1363 format to r and s big numbers\r\n");
        mbedtls_mpi r, s;
        mbedtls_mpi_init(&r);
        mbedtls_mpi_init(&s);
        mbedtls_mpi_read_binary(&r, signature_p1363, SIGNATURE_LEN / 2);
        mbedtls_mpi_read_binary(&s, signature_p1363 + SIGNATURE_LEN / 2, SIGNATURE_LEN / 2);

        /* Run verify with ecdsa */
        res = mbedtls_ecdsa_verify(&ecdsa_ctx.private_grp, data_raw_hash_digest, sizeof(data_raw_hash_digest),
                                   &ecdsa_ctx.private_Q, &r, &s);
        printf("mbedtls_ecdsa_verify: %d\r\n", res);
        printf("-----\r\n");

        /* Free objects */
        mbedtls_mpi_free(&r);
        mbedtls_mpi_free(&s);
        mbedtls_ecdsa_free(&ecdsa_ctx);
#undef SIGNATURE_LEN
    }

    /* 
     * Public key format:   DER/PEM (String based public key, --- BEGIN PUBLIC KEY --- type of message)
     * Signature format :   P1363   (64-bytes long, r|s)
     */
    {
#define SIGNATURE_LEN (sizeof(signature_p1363))
        /* Parse public key */
        mbedtls_pk_init(&pubkey_ctx);
        res = mbedtls_pk_parse_public_key(&pubkey_ctx, ecc_public_key_text, sizeof(ecc_public_key_text));
        printf("mbedtls_pk_parse_public_key: %d\r\n", res);

        /* Parse P1363 signature */
        mbedtls_mpi r, s;
        mbedtls_mpi_init(&r);
        mbedtls_mpi_init(&s);
        mbedtls_mpi_read_binary(&r, signature_p1363, SIGNATURE_LEN / 2);
        mbedtls_mpi_read_binary(&s, signature_p1363 + SIGNATURE_LEN / 2, SIGNATURE_LEN / 2);

        /* Get ECDSA verify context from pk structure */
        mbedtls_ecdsa_context* ctx = pubkey_ctx.private_pk_ctx;
        res = mbedtls_ecdsa_verify(&ctx->private_grp, data_raw_hash_digest, sizeof(data_raw_hash_digest),
                                   &ctx->private_Q, &r, &s);
        printf("mbedtls_ecdsa_verify: %d\r\n", res);
        printf("-----\r\n");

        /* Free objects */
        mbedtls_mpi_free(&r);
        mbedtls_mpi_free(&s);
        mbedtls_pk_free(&pubkey_ctx);
#undef SIGNATURE_LEN
    }
    return 0;
}

Script to generate the keys and files

import sys, random, base64, os, json
from Crypto.PublicKey import ECC
from Crypto.Cipher import AES
from Crypto.Hash import SHA256
from Crypto.Signature import DSS

# Input string - make it 16 bytes aligned for future AES test
data_raw = 'This is my input data 0123456789'
data_raw_encoded = data_raw.encode('utf-8')

# Calculate SHA-256 hash option of it and get its digest
data_raw_hash = SHA256.new(data_raw_encoded)
data_raw_hash_digest = data_raw_hash.digest()

# Generate private/public key pair for ECC
# Use various formats for one curve -> PEM, compressed, uncompressed, ...
key = ECC.generate(curve = 'secp256r1')
private_key = key.export_key(format = 'PEM', compress = False) # Export private key
public_key = key.public_key().export_key(format = 'PEM', compress = False) # Generate public key and export it
public_key_sec1_compressed = key.public_key().export_key(format = 'SEC1', compress = True) # Generate public key and export it
public_key_sec1_uncompressed = key.public_key().export_key(format = 'SEC1', compress = False) # Generate public key and export it in uncompressed mode -> used by mbedTLS

# Sign the hash of raw data with private key
# Sign in P1363 format
print('Signature with P1363 format')
signer_p1363 = DSS.new(ECC.import_key(private_key), 'fips-186-3')
signature_p1363 = signer_p1363.sign(data_raw_hash)
print('signature _p1363', ''.join(['{:02X}'.format(i) for i in signature_p1363]))
print('len _p1363', len(signature_p1363))
# Sign in DER format - default for mbedTLS
print('Signature with DER format')
signer_der = DSS.new(ECC.import_key(private_key), 'fips-186-3', encoding = 'der')
signature_der = signer_der.sign(data_raw_hash)
print('signature _der', ''.join(['{:02X}'.format(i) for i in signature_der]))
print('len _der', len(signature_der))

# Quick signature verification
try:
    DSS.new(ECC.import_key(public_key), 'fips-186-3').verify(data_raw_hash, signature_p1363)
    print('Verification is OK - P1363')
except:
    print('Signature verification failed - P1363')
try:
    DSS.new(ECC.import_key(public_key), 'fips-186-3', encoding = 'der').verify(data_raw_hash, signature_der)
    print('Verification is OK - DER')
except:
    print('Signature verification failed - DER')

# Write generated data to files
with open('keys/data_raw_input_str.txt', 'w') as f:                 f.write(data_raw)
with open('keys/data_raw_input_array.txt', 'w') as f:               f.write(','.join([hex(i) for i in data_raw_encoded]))
with open('keys/data_raw_hash_digest_array.txt', 'w') as f:         f.write(','.join([hex(i) for i in data_raw_hash_digest]))
with open('keys/private.ec.key', 'w') as f:                         f.write(private_key)
with open('keys/public.ec.key', 'w') as f:                          f.write(public_key)
with open('keys/private.ec.oneline.key', 'w') as f:                 f.write('"' + str(private_key).replace('\n', '\\r\\n') + '"')
with open('keys/public.ec.oneline.key', 'w') as f:                  f.write('"' + str(public_key).replace('\n', '\\r\\n') + '"')
with open('keys/ecc_public_key_compressed_array.txt', 'w') as f:    f.write(','.join([hex(i) for i in public_key_sec1_compressed]))
with open('keys/ecc_public_key_uncompressed_array.txt', 'w') as f:  f.write(','.join([hex(i) for i in public_key_sec1_uncompressed]))
with open('keys/signature_der_array.txt', 'w') as f:                f.write(','.join([hex(i) for i in signature_der]))      # Signature of the hash
with open('keys/signature_p1363_array.txt', 'w') as f:              f.write(','.join([hex(i) for i in signature_p1363]))    # Signature of the hash
MaJerle commented 1 year ago

New findings - actually it is possible to extract 3 out of 4 options in relatively clean way, but one remains a mistery. How to NOT use private variables if public key is in binary format (65-bytes) and signature is in DEM format.

Case 1: PubKey in binary, signature in DER

This is the only example that is accessing private components. mbedtls_ecdsa_context and mbedtls_ecp_keypair are same type. Keypair utilizes group and Q values, that are accessed in private mode.

How to fix that? Which API functions do allow us to do it properly?

    /* 
     * PublicKey format:   binary  (65-bytes long, 0x04 as first byte)
     * Signature format:   DER     (70, 71 or 72 bytes long)
     */
    {
        mbedtls_ecdsa_context ecdsa_ctx;

        printf("Public key - ECDSA verification:\r\nPublicKey format: Binary\r\nSignature format: DER\r\n\r\n");

        /* Parse public key */
        mbedtls_ecdsa_init(&ecdsa_ctx);
        mbedtls_ecp_group_load(&ecdsa_ctx.private_grp, MBEDTLS_ECP_DP_SECP256R1);
        res = mbedtls_ecp_point_read_binary(&ecdsa_ctx.private_grp, &ecdsa_ctx.private_Q,
                                            ecc_public_key_uncompressed_bin, sizeof(ecc_public_key_uncompressed_bin));
        printf("mbedtls_ecp_point_read_binary: %d\r\n", res);

        /* Verify with DER native format support */
        res = mbedtls_ecdsa_read_signature(&ecdsa_ctx, data_raw_hash_digest, sizeof(data_raw_hash_digest),
                                           signature_der, sizeof(signature_der));
        printf("mbedtls_ecdsa_read_signature: %d\r\n", res);

        /* Free objects */
        mbedtls_ecdsa_free(&ecdsa_ctx);

        /* Done */
        printf("-----\r\n");
    }

Case 2: PubKey in DER/PEM, signature in DER

    volatile int res;
    /* 
     * PublicKey format:   DER/PEM (String based public key, --- BEGIN PUBLIC KEY --- type of message)
     * Signature format:   DER     (70, 71 or 72 bytes long)
     */
    {
        mbedtls_pk_context pubkey_ctx;

        printf("Public key - ECDSA verification:\r\nPublicKey format: DER/PEM\r\nSignature format: DER\r\n\r\n");

        /* Parse public key */
        mbedtls_pk_init(&pubkey_ctx);
        res = mbedtls_pk_parse_public_key(&pubkey_ctx, ecc_public_key_text, sizeof(ecc_public_key_text));
        printf("mbedtls_pk_parse_public_key: %d\r\n", res);

        /* Verify with DER native format support */
        res = mbedtls_pk_verify(&pubkey_ctx, MBEDTLS_MD_SHA256, data_raw_hash_digest, sizeof(data_raw_hash_digest),
                                signature_der, sizeof(signature_der));
        printf("mbedtls_pk_verify: %d\r\n", res);

        /* Free objects */
        mbedtls_pk_free(&pubkey_ctx);

        /* Done */
        printf("-----\r\n");
    }

Case 3: PubKey in binary, signature in P1363

    /* 
     * PublicKey format:   binary  (65-bytes long, 0x04 as first byte)
     * Signature format:   P1363   (64-bytes long, r|s)
     */
    {
#define SIGNATURE_LEN (sizeof(signature_p1363))
        mbedtls_mpi r, s;
        mbedtls_ecp_group group;
        mbedtls_ecp_point q;

        printf("Public key - ECDSA verification:\r\nPublicKey format: Binary\r\nSignature format: P1363\r\n\r\n");

        /* Initialize all modules */
        mbedtls_mpi_init(&r);
        mbedtls_mpi_init(&s);
        mbedtls_ecp_point_init(&q);
        mbedtls_ecp_group_init(&group);

        /* Parse public key in binary format */
        mbedtls_ecp_group_load(&group, MBEDTLS_ECP_DP_SECP256R1);
        res = mbedtls_ecp_point_read_binary(&group, &q, ecc_public_key_uncompressed_bin,
                                            sizeof(ecc_public_key_uncompressed_bin));
        printf("mbedtls_ecp_point_read_binary: %d\r\n", res);

        /* Parse signature in P1363 format to r and s big numbers */
        res = mbedtls_mpi_read_binary(&r, signature_p1363, SIGNATURE_LEN / 2);
        printf("mbedtls_mpi_read_binary: %d\r\n", res);
        res = mbedtls_mpi_read_binary(&s, signature_p1363 + SIGNATURE_LEN / 2, SIGNATURE_LEN / 2);
        printf("mbedtls_mpi_read_binary: %d\r\n", res);

        /* Run verify with ecdsa */
        res = mbedtls_ecdsa_verify(&group, data_raw_hash_digest, sizeof(data_raw_hash_digest), &q, &r, &s);
        printf("mbedtls_ecdsa_verify: %d\r\n", res);

        /* Free objects */
        mbedtls_mpi_free(&r);
        mbedtls_mpi_free(&s);
        mbedtls_ecp_point_free(&q);
        mbedtls_ecp_group_free(&group);

        /* Done */
        printf("-----\r\n");
#undef SIGNATURE_LEN
    }

Case 4: PubKey in DER/PEM, signature in P1363

    /* 
     * PublicKey format:   DER/PEM (String based public key, --- BEGIN PUBLIC KEY --- type of message)
     * Signature format:   P1363   (64-bytes long, r|s)
     * 
     * Steps to follow:
     * 
     * - Parse public key with PK module
     * - Extract EC parameters -> generate key pair
     * - Extract group, D and Q values from the pair
     * - Parse P1363 format into 2 big numbers R and S
     * - Call ecdsa verification function
     */
    {
#define SIGNATURE_LEN (sizeof(signature_p1363))
        mbedtls_pk_context pubkey_ctx;
        mbedtls_ecp_keypair* pair;
        mbedtls_ecp_group group;
        mbedtls_mpi d;
        mbedtls_ecp_point q;
        mbedtls_mpi r, s;

        printf("Public key - ECDSA verification:\r\nPublicKey format: DER/PEM\r\nSignature format: P1363\r\n\r\n");

        /* Initialize all values to its default state - avoid any segmentation faults */
        mbedtls_pk_init(&pubkey_ctx);
        mbedtls_mpi_init(&r);
        mbedtls_mpi_init(&s);
        mbedtls_ecp_group_init(&group);
        mbedtls_mpi_init(&d);
        mbedtls_ecp_point_init(&q);

        /* Parse public key */
        res = mbedtls_pk_parse_public_key(&pubkey_ctx, ecc_public_key_text, sizeof(ecc_public_key_text));
        printf("mbedtls_pk_parse_public_key: %d\r\n", res);

        /* Get EC pair from parsed public key */
        pair = mbedtls_pk_ec(pubkey_ctx);
        printf("mbedtls_pk_ec: %p\r\n", (void*)pair);

        /* Export data from the pair - required for ECDSA verification purpose */
        res = mbedtls_ecp_export(pair, &group, &d, &q);
        printf("mbedtls_ecp_export: %d\r\n", res);

        /* Parse P1363 signature to 2 big nums */
        mbedtls_mpi_read_binary(&r, signature_p1363, SIGNATURE_LEN / 2);
        mbedtls_mpi_read_binary(&s, signature_p1363 + SIGNATURE_LEN / 2, SIGNATURE_LEN / 2);

        /* Get ECDSA verify context from parsed pk structure */
        res = mbedtls_ecdsa_verify(&group, data_raw_hash_digest, sizeof(data_raw_hash_digest), &q, &r, &s);
        printf("mbedtls_ecdsa_verify: %d\r\n", res);

        /* Free objects */
        mbedtls_pk_free(&pubkey_ctx);
        mbedtls_mpi_free(&r);
        mbedtls_mpi_free(&s);
        mbedtls_ecp_group_free(&group);
        mbedtls_mpi_free(&d);
        mbedtls_ecp_point_free(&q);

        /* Done */
        printf("-----\r\n");
#undef SIGNATURE_LEN
    }