Dtls fingerprint verification fails on signed certificates

[szcom]

Running version Version: 11.5.1.11+0~mr11.5.1.11 git-HEAD-51248ea, I am seeing DTLS fingerprint verification failures. I suspect it involves signed Server certificates from the peer.

For example this works:

1 packet used to send server cert

But this does not:

3 packets used to send server cert, appears to be a signed cert, with root CA included.

[szcom]

According to "The return value of verify_callback controls the strategy of the further verification process. If verify_callback returns 0, the verification process is immediately stopped with “verification failed” state. And this will abort the process when it sees CA root, which is to be verified first.

[rfuchs]

How to reproduce?

[szcom]

One option is to build chain with CA and signed cert and execute verify_callback code on it to verify. CA will not match fingerprint of signed cert and it should fail.

[szcom]

Actually it may be use of X509_STORE_CTX_get_current_cert when we need leaf certificate available via ‎X509_STORE_CTX_get0_cert

as in here for ex

[rfuchs]

So something like 8fba68f perhaps then?

[szcom]

lgtm