Engine doesn't work while on TLS handshake

[Juice-XIJ]

Hello!

I have used this openssl-engine on my ATECC508A. As we know the private key is stored inside and we have no way to get it out.

But during TLS communication with server, if server want's to verify the device identity, device needs to send signer cert and client cert to server. Besides device needs to send a response encrypted by client cert private key to server. Server will use the CA to verify this cert chain and use the public key in client cert to verify the response from client. All those thing would be finished during TLS handshake.

Therefore I used the engine to overwrite the sign algorithm in ECDSA and compile it with ECC_DEBUG macro. However when I run my program to connect the server, no output from new sign algorithm in ECDSA and of course, the server is not able to identify the client. I think this engine doesn't work on TLS handshake but response encryption during handshake is necessary. Hence do you have any ideas to solve this problem?

My test file in client side is here:

#include <openssl/ssl.h>
#include <openssl/conf.h>
#include "openssl/eccx08_engine.h"
#include "openssl/eccx08_engine_internal.h"

int main()
{
    static ENGINE *ateccx08_engine;
    OpenSSL_add_all_algorithms();
    ERR_load_crypto_strings();

    PRINTF("ENGINE_load_dynamic");
    ENGINE_load_dynamic();

    printf("CONF_modules_load_file");
    if (!CONF_modules_load_file(NULL, NULL, CONF_MFLAGS_DEFAULT_SECTION))
    {
        printf("Config failed to load");
    }

    printf("ENGINE_by_id");
    ateccx08_engine = ENGINE_by_id("ateccx08");

    if (ateccx08_engine == NULL)
    {
        printf("Engine failed to load");
    }

    // after some initialization

    // load client-side cert and key
    SSL_CTX_use_certificate_file(m_ctx, ClientCertificateFileTest, SSL_FILETYPE_PEM);

    // no need anymore because no way to extract private key
    // SSL_CTX_use_PrivateKey_file(m_ctx, ClientPrivateKeyFileTest, SSL_FILETYPE_PEM);

    // load intermediate cert
    X509* chaincert = X509_new();
    BIO* bio_cert = BIO_new_file(SignerCertificateFileTest, "rb");
    PEM_read_bio_X509(bio_cert, &chaincert, NULL, NULL);
    SSL_CTX_add1_chain_cert(m_ctx, chaincert)

    m_ssl = SSL_new(m_ctx);

    // get_seocket is my own API
    m_sock = get_socket();

    SSL_set_fd(m_ssl, m_sock)

    // doing handshake and build connection, however no output from ECDSA sign algorithm
    auto r = SSL_connect(m_ssl);
}

[bryan-hunt]

The problem is you have to attach engine functionality to OpenSSL. It's not sufficient to just open the engine - the resulting handle has to be connected to a resource in some way.

Basically you'll need the following in your code before your ssl context creation:

ENGINE_init(ateccx08_engine);     // Obviously would be good to check the return from this function
ENGINE_set_default_ECDSA(ateccx08_engine);

See the OpenSSL documentation for more details

Alternatively you can look at using cryptoauthlib by itself as a pkcs11 provider and using the pkcs11 engine instead. See the cryptoauthlib wiki for more details on how to do this. The application will still need to know a little bit about setting up the engine but the pkcs11 engine should be fully documented in numerous examples in many applications that already use it.

[Juice-XIJ]

Thanks a lot! I would have a look of that one.

On Tue, Jan 8, 2019 at 7:25 AM Bryan Hunt notifications@github.com wrote:

The problem is you have to attach engine functionality to OpenSSL. It's not sufficient to just open the engine - the resulting handle has to be connected to a resource in some way.

Basically you'll need the following in your code before your ssl context creation:

ENGINE_init(ateccx08_engine); // Obviously would be good to check the return from this function ENGINE_set_default_ECDSA(ateccx08_engine);

See the OpenSSL documentation for more details https://www.openssl.org/docs/man1.0.2/crypto/engine.html

Alternatively you can look at using cryptoauthlib by itself as a pkcs11 provider and using the pkcs11 engine instead. See the cryptoauthlib wiki https://github.com/MicrochipTech/cryptoauthlib/wiki/PKCS11-Linux-Setup for more details on how to do this. The application will still need to know a little bit about setting up the engine but the pkcs11 engine should be fully documented in numerous examples in many applications that already us it.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/MicrochipTech/cryptoauth-openssl-engine/issues/32#issuecomment-452337993, or mute the thread https://github.com/notifications/unsubscribe-auth/ARkgQ6oaAhv-oc95DaBBUO__rvFy3Z2Pks5vBLhzgaJpZM4Z0zXA .

[Juice-XIJ]

Thank you again. I found the other problem in my code that is I comment "SSL_CTX_use_PrivateKey_file" so that OpenSSL doesn't working for chain certs at all.

I use client cert public key as input of "SSL_CTX_use_PrivateKey_file" and set default engine. Then everything works fine!!

[cjycjw]

Thank you again. I found the other problem in my code that is I comment "SSL_CTX_use_PrivateKey_file" so that OpenSSL doesn't working for chain certs at all.

I use client cert public key as input of "SSL_CTX_use_PrivateKey_file" and set default engine. Then everything works fine!!

hi, I was trying to use my own pkcs11 engine and I encountered the same problem. I saw the source code of SSL_CTX_use_PrivateKey_file, it calls PEM_read_bio_PrivateKey inside, and it return null when I passed public key to PEM_read_bio_PrivateKey. Does everything work fine really for you? Am I missing something?