SIGSEGV with 2.8 if "consistency of a private key" check fails

[perlun]

Hi,

With osslsigncode 2.8, I get a segmentation fault on certain conditions:

❯ osslsigncode sign -certs cert.cer -key cert.key -t http://timestamp.globalsign.com/tsa/r6advanced1 -in publish/win-x86-dev/example.exe -out publish/win-x86-dev/example-signed.exe
Failed to checking the consistency of a private key: cert.key
          with a public key in any X509 certificate: cert.cer

Creating a new signature failed
Unable to prepare new signature
Segmentation fault

I collected a stacktrace with gdb and the sigsegv seems to happen inside OpenSSL:

❯ gdb --args osslsigncode sign -certs cert.cer -key cert.key -t http://timestamp.globalsign.com/tsa/r6advanced1 -in publish/win-x86-dev/example.exe -out publish/win-x86-dev/example-signed.exe
GNU gdb (Debian 13.1-3) 13.1
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from osslsigncode...
(No debugging symbols found in osslsigncode)
(gdb) run
Starting program: /usr/bin/osslsigncode sign -certs cert.cer -key cert.key -t http://timestamp.globalsign.com/tsa/r6advanced1 -in publish/win-x86-dev/example.exe -out publish/win-x86-dev/example-signed.exe
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Failed to checking the consistency of a private key: cert.key
          with a public key in any X509 certificate: cert.cer

Creating a new signature failed
Unable to prepare new signature

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b089d5 in BIO_free () from /lib/x86_64-linux-gnu/libcrypto.so.3
(gdb) bt
#0  0x00007ffff7b089d5 in BIO_free () from /lib/x86_64-linux-gnu/libcrypto.so.3
#1  0x000055555555d1e2 in ?? ()
#2  0x00007ffff778524a in __libc_start_call_main (main=main@entry=0x55555555c3a0, argc=argc@entry=12, argv=argv@entry=0x7fffffffe378) at ../sysdeps/nptl/libc_start_call_main.h:58
#3  0x00007ffff7785305 in __libc_start_main_impl (main=0x55555555c3a0, argc=12, argv=0x7fffffffe378, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe368)
    at ../csu/libc-start.c:360
#4  0x000055555555f121 in ?? ()

The interesting part is that with osslsigncode 2.5, I do not get the SIGSEGV above. :thinking:

❯ osslsigncode sign -certs cert.cer -key cert.key -t http://timestamp.globalsign.com/tsa/r6advanced1 -in publish/win-x86-dev/example.exe -out publish/win-x86-dev/example-signed.exe
Failed to checking the consistency of a private key: cert.key
          with a public key in any X509 certificate: cert.cer

Creating a new signature failed
40698413E37F0000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode.  Input type: DER, Input structure: type-specific
40698413E37F0000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode.  Input type: DER, Input structure: PrivateKeyInfo
40698413E37F0000:error:1E08010C:DECODER routines:OSSL_DECODER_from_bio:unsupported:../crypto/encode_decode/decoder_lib.c:101:No supported data to decode. Input type: DER
40698413E37F0000:error:068000A8:asn1 encoding routines:asn1_check_tlen:wrong tag:../crypto/asn1/tasn_dec.c:1188:
40698413E37F0000:error:0688010A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:../crypto/asn1/tasn_dec.c:613:
40698413E37F0000:error:068000A8:asn1 encoding routines:asn1_check_tlen:wrong tag:../crypto/asn1/tasn_dec.c:1188:
40698413E37F0000:error:0688010A:asn1 encoding routines:asn1_item_embed_d2i:nested asn1 error:../crypto/asn1/tasn_dec.c:349:Type=RSAPrivateKey
40698413E37F0000:error:068000A8:asn1 encoding routines:asn1_check_tlen:wrong tag:../crypto/asn1/tasn_dec.c:1188:
40698413E37F0000:error:0688010A:asn1 encoding routines:asn1_item_embed_d2i:nested asn1 error:../crypto/asn1/tasn_dec.c:349:Type=PKCS8_PRIV_KEY_INFO
40698413E37F0000:error:05800074:x509 certificate routines:X509_check_private_key:key values mismatch:../crypto/x509/x509_cmp.c:408:
Failed

[mtrojnar]

Can you reproduce this issue with the current master branch?

[perlun]

Yeah, sorry for not getting back to you @mtrojnar. IIRC, this was a bit of an edge case when the certificate and key weren't really associated; osslsigncode started working as intended when I was using a correct certificate (chain) & key-pair. If I get back to it sometime, I'll try reproducing with current master, but until then feel free to ignore it. It's unlikely to hit any normal user, unless they have a weird certificate/key mismatch like I did.