openssl enc - cipher decrypt error

[kinichiro]

Hi,

I checked all ciphers by test script, enctest.sh, like this. https://gist.github.com/kinichiro/d2513ac0eccfde00d80b

This test script just encrypts small text file and decrypts it, and checks if decrypted file and original file is same or not.

And I noticed some ciphers got error when decrypting. There are 3 error patterns, like below.

(1) decrypt causes "Segmentation fault".

    -aes-128-cbc-hmac-sha1
    -aes-256-cbc-hmac-sha1

(2) "bad decrypt" occurs, but decrypt file is equal to original file.

    -aes-128-gcm
    -aes-192-gcm
    -aes-256-gcm
    -id-aes128-GCM
    -id-aes256-GCM

(3) encrypt and decrypt succeed, but decrypt file is different from original file.

    -des-ede3-cfb1

I tested this on CentOS 7, and execution log is also put in Gist above.

Thanks

[4a6f656c]

FWIW OpenBSD has src/regress/usr.bin/openssl/enctest.sh, however it probably needs to be updated/improved.

The segmentation fault does not occur on OpenBSD for (1) - if you could provide further details (preferably a trace) that would be appreciated.

I can confirm that I can reproduce (2) and (3) on OpenBSD - I'll dig into these.

[bcook-r7]

The target architecture would be useful to have. Was it i386 or x86_64?

[bcook-r7]

I'll look at integrating the usr.bin/openssl tests into portable as well - thanks for the report!

[kinichiro]

It was x86_64, sorry to forget describing Arch.

I didn't know testenc script exists already. Thanks for your info.

I will try to get further trace for segmentation fault later.

BTW, I noticed mingw64 has another error pattern. Now I can't list it here, though.

[bcook-r7]

The segfault appears to be related to AESNI/SSE3 instructions used in the SHA1 offload assembly code. The issue occurs with the packaged version of OpenSSL 1.0.1f with Ubuntu 14.04 as well:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff780bbd0 in ?? () from /lib/x86_64-linux-gnu/libcrypto.so.1.0.0

Backtrace from LibreSSL:

Starting program: /usr/local/bin/openssl enc -aes-128-cbc-hmac-sha1 -d -pass pass:test-pass-1234 -in ./test/encfile.dat--aes-128-cbc-hmac-sha1.enc -out ./test/encfile.dat--aes-128-cbc-hmac-sha1.dec

Program received signal SIGSEGV, Segmentation fault.
sha1_block_data_order_ssse3 () at sha/sha1-elf-x86_64.s:2238
2238        movdqu  48(%r9),%xmm3
(gdb) bt
#0  sha1_block_data_order_ssse3 () at sha/sha1-elf-x86_64.s:2238
#1  0xca62c1d6ca62c1d6 in ?? ()
#2  0xca62c1d6ca62c1d6 in ?? ()
#3  0xca62c1d6ca62c1d6 in ?? ()
#4  0xca62c1d6ca62c1d6 in ?? ()
#5  0xca62c1d6ca62c1d6 in ?? ()
#6  0xca62c1d6ca62c1d6 in ?? ()
#7  0xca62c1d6ca62c1d6 in ?? ()
#8  0xca62c1d6ca62c1d6 in ?? ()
#9  0x000000000000003f in ?? ()
#10 0x0000000000699d64 in ?? ()
#11 0xffffffffffffffc0 in ?? ()
#12 0x00007ffff78704f5 in sha1_update (c=0xe8514e95, c@entry=0x699d64, data=data@entry=0x699dd0, len=4177660669) at evp/e_aes_cbc_hmac_sha1.c:155
#13 0x00007ffff787076e in aesni_cbc_hmac_sha1_cipher (ctx=<optimized out>, out=0x698b70 "ghijklmnopqrstuvwxyz\n", '\v' <repeats 11 times>,
    in=0x695850 "5{\331_\260Lb\020\362\240\001\217\355|\272\276\365\223_\363^\372\374\004v\253\252\217T\253<\242t\033St\240\256\023E\275\343\367\"\207\\\304\300", len=32)
    at evp/e_aes_cbc_hmac_sha1.c:296
#14 0x00007ffff78782ee in EVP_EncryptUpdate (ctx=ctx@entry=0x698ab8, out=out@entry=0x698b60 "1234567890abcdefghijklmnopqrstuvwxyz\n", '\v' <repeats 11 times>,
    outl=outl@entry=0x698aa0,
    in=in@entry=0x695850 "5{\331_\260Lb\020\362\240\001\217\355|\272\276\365\223_\363^\372\374\004v\253\252\217T\253<\242t\033St\240\256\023E\275\343\367\"\207\\\304\300", inl=48)
    at evp/evp_enc.c:321
#15 0x00007ffff787854b in EVP_DecryptUpdate (ctx=ctx@entry=0x698ab8, out=out@entry=0x698b60 "1234567890abcdefghijklmnopqrstuvwxyz\n", '\v' <repeats 11 times>,
    outl=outl@entry=0x698aa0,
    in=in@entry=0x695850 "5{\331_\260Lb\020\362\240\001\217\355|\272\276\365\223_\363^\372\374\004v\253\252\217T\253<\242t\033St\240\256\023E\275\343\367\"\207\\\304\300",
    inl=inl@entry=48) at evp/evp_enc.c:463
#16 0x00007ffff78786d4 in EVP_CipherUpdate (ctx=ctx@entry=0x698ab8, out=out@entry=0x698b60 "1234567890abcdefghijklmnopqrstuvwxyz\n", '\v' <repeats 11 times>,
    outl=outl@entry=0x698aa0,
    in=in@entry=0x695850 "5{\331_\260Lb\020\362\240\001\217\355|\272\276\365\223_\363^\372\374\004v\253\252\217T\253<\242t\033St\240\256\023E\275\343\367\"\207\\\304\300",
    inl=inl@entry=48) at evp/evp_enc.c:251
#17 0x00007ffff786b7f3 in enc_write (b=0x698a30,
    in=0x695850 "5{\331_\260Lb\020\362\240\001\217\355|\272\276\365\223_\363^\372\374\004v\253\252\217T\253<\242t\033St\240\256\023E\275\343\367\"\207\\\304\300", inl=<optimized out>)
    at evp/bio_enc.c:260
#18 0x00007ffff781b65b in BIO_write (b=b@entry=0x698a30, in=in@entry=0x695850, inl=inl@entry=48) at bio/bio_lib.c:252
#19 0x0000000000428fd5 in enc_main (argc=<optimized out>, argv=<optimized out>) at enc.c:717
#20 0x000000000042f511 in do_cmd (prog=prog@entry=0x690470, argc=9, argv=0x7fffffffe570) at openssl.c:426
#21 0x0000000000416ff9 in main (argc=9, argv=0x7fffffffe570) at openssl.c:326

[bcook-r7]

I tried the test with the Windows binaries, and noticed that the end-of-line characters swap between \n and \r\n. I probably need to wrap or adjust the BIO file open calls to open in 'binary' mode so Windows does not do crazy default inline text conversions.

The segfault may be reproducible on OpenBSD as well if you have the hardware to trigger the aesni_cbc_hmac_sha1_cipher code path.

[kinichiro]

Hi, I got further trace for segmentation fault, and it seems same.

# gdb ../apps/.libs/openssl
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-64.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /work/github/portable/apps/.libs/openssl...(no debugging symbols found)...done.
(gdb) directory ../crypto
Source directories searched: /work/github/portable/test/../crypto:$cdir:$cwd
(gdb) run enc -AES-128-CBC-HMAC-SHA1 -d -base64 -pass pass:test -in test/user1/encfile.dat--AES-128-CBC-HMAC-SHA1.enc
Starting program: /work/github/portable/test/../apps/.libs/openssl enc -AES-128-CBC-HMAC-SHA1 -d -base64 -pass pass:test -in test/user1/encfile.dat--AES-128-CBC-HMAC-SHA1.enc
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff75e8e7d in sha1_block_data_order_ssse3 () from ../crypto/.libs/libcrypto.so.33
Missing separate debuginfos, use: debuginfo-install glibc-2.17-78.el7.x86_64
(gdb) bt
#0  0x00007ffff75e8e7d in sha1_block_data_order_ssse3 () from ../crypto/.libs/libcrypto.so.33
#1  0xca62c1d6ca62c1d6 in ?? ()
#2  0xca62c1d6ca62c1d6 in ?? ()
#3  0xca62c1d6ca62c1d6 in ?? ()
#4  0xca62c1d6ca62c1d6 in ?? ()
#5  0xca62c1d6ca62c1d6 in ?? ()
#6  0xca62c1d6ca62c1d6 in ?? ()
#7  0xca62c1d6ca62c1d6 in ?? ()
#8  0xca62c1d6ca62c1d6 in ?? ()
#9  0x0000000000415440 in EC_GROUP_get_curve_GF2m@plt ()
#10 0x00007fffffffd980 in ?? ()
#11 0x0000000000000000 in ?? ()
(gdb)

sorry for late.

[bcook-r7]

I just verified it also crashes on OpenBSD as long as the CPU has AES in dmesg. I reproduced both in a VM on a recent Macbook using Virtualbox 5.0 (which supports AES-NI passthrough) and on an i7 x220 laptop:

[TEST] -aes-128-cbc-hmac-sha1
:-) success.
./enctest.sh: line 41: 27337 Segmentation fault      (core dumped) $openssl_bin enc $c -d -pass pass:$pass -in $encfile-$c.enc -out $encfile-$c.dec
:-< error occurs, exit status = [ 139 ]

[4a6f656c]

The bulk of these are simply shortcomings with the openssl(1) enc command and the cipher algorithms that you're testing - for example, attempting to use openssl(1) with any GCM algorithms is futile since there is no way to retrieve or provide the authentication tag via the command line, hence the "bad decrypt" messages. There are also similar problems trying to use AES-XTS, for example. While the aes-128-cbc-hmac-sha1 segfault should not happen and is likely a bug, there is a good chance that it is due to the same issue (though I've not yet confirmed).

The actual list of cipher commands supported is available via "openssl list-cipher-commands" (as opposed to "openssl list-cipher-algorithms") - we may want to try to reduce the algorithms listed as part of the openssl(1) enc usage to match...

[kinichiro]

Thanks for clarification of this issue.

I was trying to verify all available cipher works fine or not. Then, it seems that I did too much. My test seems went wrong way.

I will correct my test a little bit with using list-cipher-commands to obtain the right cipher names.

And, enc command should handle if it is right cipher name or not.

Many thanks.

[sergeevabc]

Years are passing by, yet the issue with tag “bug” still exists. So what’s the current status? Should we avoid using LibreSSL for Windows? Also, I see 2.5.5 is compiled, while 2.6.4 is current and not compiled.

[busterb]

LibreSSL stopped shipping precompiled Windows binaries at version 2.6.0, which is explained in the release email:

https://bsdsec.net/articles/libressl-2-5-5-2-6-0-released

[sergeevabc]

The Windows build process has improved substantially, supporting multiple compilers, environments, and ABIs. Because of slight variations between these environments, and because it is so easy to build on Windows in general, we are releasing source code only for all platforms moving forward.

Being a mere Windows user without compiler, I feel betrayed. Lots of prominent software developers give no such lame excuses and remain committed to the digital revolution, which means not to confuse, but to empower others with ready-to-use tools. Depeche Mode asks the rest: “Where’s the revolution?”

[tamersaadeh]

I disagree: a library isn't exactly something that's designed for "users without a compiler" to install directly. They are meant for developers to use and integrate into their applications, and install on systems if missing.

[sergeevabc]

@tamersaadeh, arghhh, it’s not about library, but about openssl.exe.

[tamersaadeh]

But the command line exec (openssl.exe) is used for testing purposes only to debug functionality, so it shouldn't be used to implement/use functionality provided.

[sergeevabc]

@tamersaadeh, openssl.exe is the most accessible cross-platform tool used as a precompiled binary for encryption and keys generation by myriads for years. Once a poet said: “I have spread my dreams under your feet, tread softly”, likewise people, myself included, relied on you to raise the libre banner of SSL. Now you require that everyone become a programmer to enjoy the updates.

[busterb]

If I posted openssl.exe in the Windows App store, and charged $0.99, would you buy it?

Don't answer. We're way off-topic on this issue.

[busterb]

Follow-up on the secondary Windows build issue: Microsoft now gives away compilers for free (just a call to 'winget' away). It now also owns github so those same environments are being used for regression tests. We'll probably be able to at least automate pushing the tagged builds to releases as well (see #772 ). At any rate, considering this original issue closed.