h2o / picotls

TLS 1.3 implementation in C (master supports RFC8446 as well as draft-26, -27, -28)
535 stars 140 forks source link

[evp_keyex_init] keep refcount unchanged when the function fails #490

Closed kazuho closed 10 months ago

kazuho commented 11 months ago

evp_keyex_init is called from two locations. Both of them assume that if the functions returns zero (success) the ownership of pkey is transferred to evp_pkey_init, or if the function returns non-zero (failure) the reference count of pkey remains as-is.

However, evp_keyex_init has been inconsistent in what to do upon failure; it has been retaining reference count as-is if malloc failed, but has been decrementing the reference count if EVP_PKEY_get1_tls_encodedpoint failed.

This PR address the latter, making sure that if an error is returned the reference count remains as-is.