TLS1.3: p11prov_DeriveKey:Host out of memory

[kshitizvars]

Hi @simo5

We are able to run ECDH key exchange operations on tls1.2, but facing some issues in tls1.3.

ERROR
2070FEF7FFFF0000:error:40800002:pkcs11:p11prov_DeriveKey:Host out of memory error:/usr/src/debug/pkcs11-provider/0.3/src/interface.gen.c:1011:Error returned by C_DeriveKey
2070FEF7FFFF0000:error:0A0C0103:SSL routines:ssl_derive:internal error:ssl/s3_lib.c:4901:
shutting down SSL

Program received signal SIGSEGV, Segmentation fault 

On further debugging, it is because of wrong client public key point (EC_POINT) and its length:-

TLS1.3 logs

(gdb) p *((P11PROV_OBJ *)0x587920).attrs
$11 = {type = 16462464871219690244, pValue = 0xeed50f6aa03e120d, ulValueLen = 14520826335356353610}
(gdb) p *((P11PROV_OBJ *)0x587920)->attrs
$12 = {type = 16462464871219690244, pValue = 0xeed50f6aa03e120d, ulValueLen = 14520826335356353610}

TLS1.2 logs

(gdb) p *(CK_ATTRIBUTE *) 0x588b20
$24 = {type = 2152681555, pValue = 0x5ca280, ulValueLen = 65}
(gdb) p ((P11PROV_OBJ *)0x587920)->attrs[0]
$25 = {type = 2152681555, pValue = 0x5ca280, ulValueLen = 65}
(gdb) p ((P11PROV_OBJ *)0x587920)->attrs[1]

Do you have any comments? Debug logs:- debug_tls1_2.log debug_tls1_3.log

A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. Openssl configuration:-

   
   [openssl_init]
   providers = provider_sect
   alg_section = algorithm_sect

List of providers to load

[provider_sect] default = default_sect pkcs11 = pkcs11_sect

[default_sect] activate = 1 [pkcs11_sect] module = /usr/lib/ossl-modules/pkcs11.so pkcs11-module-path = /usr/lib/libckteec.so.0 pkcs11-module-cache-keys = false pkcs11-module-quirks = no-operation-state pkcs11-module-block-operations = digest activate = 1

[algorithm_sect] default_properties = ?provider=pkcs11

2. **server side command:-**

`$ openssl s_server -key "pkcs11:model=OP-TEE%20TA;manufacturer=Linaro;serial=0000000000000001;token=token0;id=%01;object=ecc-key-256;type=private?pin-value=1234" -cert server.crt -accept 443`

3.  **Run openssl s_client from another machine:-
#Client side command:-**
`$ openssl s_client -connect <server ip>:443 -tls1_3 -ciphersuites 'TLS_AES_256_GCM_SHA384' -curves secp256r1`

**Expected behavior**
TLS1.3 connection should work fine with key exchange operation.

**Operating environment (please complete the following information):**
 - OS: Linux

**Does this mean, we haven't tested ECDH derive with provider in TLS1.3?**

[kshitizvars]

Hi @simo5

Are you planning to merge this commit https://github.com/simo5/pkcs11-provider/commit/75cc2c3c622e09ce03c86cd4f55a6257bbdcd47f to main branch?

[simo5]

@kshitizvars I guess I should eventually, I had forgotten I written that code ... If I have not opened a PR it must mean that code was not ready yet.

[kshitizvars]

Hi @simo5

FYI, I have tested https://github.com/simo5/pkcs11-provider/commit/75cc2c3c622e09ce03c86cd4f55a6257bbdcd47f patch after running TLS1.2 & TLS1.3. Initially, I was getting the issue mentioned in the discussion (DeriveKey: Host out of memory) but after applying https://github.com/simo5/pkcs11-provider/pull/2, no more issues and TLS1.2 & TLS1.3 connections are working fine with ECDH curve as secp256r1.