Child pid killed by signal 11

tomlabaude commented 4 years ago

Upgrading sslsplit inside Debookee from 0.5.4 to 0.5.5, sslsplit stops with signal 11 as soon as it sees first traffic.

Debug says :

Connecting to [108.177.126.109]:993
tcp 192.168.43.211 51907 108.177.126.109 993
TCP connected to [108.177.126.109]:993
TCP connected from [192.168.43.211]:51907
Checking for a client hello
Peek found ClientHello
Child pid 25352 killed by signal 11

Configuration: On macOS 10.14.6, non-root Email traffic is redirected to interface lo0 and then port 6971 through redirections and NAT rules:

pass out on en0 route-to lo0 inet proto tcp from any to any port = 993 user != 0 flags S/SA keep state
pass out on en0 route-to lo0 inet6 proto tcp from any to any port = 993 user != 0 flags S/SA keep state
rdr pass on lo0 inet proto tcp from any to any port = 993 -> 127.0.0.1 port 6971
rdr pass on lo0 inet6 proto tcp from any to any port = 993 -> ::1 port 6971

For bug reports, please supply:

Output of sslsplit -V

./sslsplit -V
SSLsplit 0.5.5 (built 2020-08-27)
Copyright (c) 2009-2019, Daniel Roethlisberger <daniel@roe.ch>
https://www.roe.ch/SSLsplit
Build info: OSX:10.14.6 XNU:4903.241.1:fallback:4903.278.28 V:FILE HDIFF:0 N:83c4edf
Features: -DHAVE_PF
NAT engines: pf*
Local process info support: no
compiled against OpenSSL 1.0.2 22 Jan 2015 (1000200f)
rtlinked against OpenSSL 1.0.2 22 Jan 2015 (1000200f)
OpenSSL has support for TLS extensions
TLS Server Name Indication (SNI) supported
OpenSSL is thread-safe with THREADID
OpenSSL has engine support
Using SSL_MODE_RELEASE_BUFFERS
SSL/TLS protocol availability: ssl3 tls10 tls11 tls12 
SSL/TLS algorithm availability: SHA0 RSA DSA ECDSA DH ECDH EC
OpenSSL option availability: SSL_OP_NO_COMPRESSION SSL_OP_NO_TICKET SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION SSL_OP_TLS_ROLLBACK_BUG
compiled against libevent 2.1.11-stable
rtlinked against libevent 2.1.11-stable
compiled against libnet 1.1.6
rtlinked against libnet 1.1.6
compiled against libpcap n/a
rtlinked against libpcap 1.9.1
4 CPU cores detected

Output of uname -a

Darwin Pep 18.7.0 Darwin Kernel Version 18.7.0: Mon Feb 10 21:08:45 PST 2020; root:xnu-4903.278.28~1/RELEASE_X86_64 x86_64

Exact command line arguments used to run sslsplit sudo ./sslsplit -k ~/mitmproxy-ca.pem -L ~/logAllData.log -l ~/logConnections.log autossl 127.0.0.1 6971
List of failing unit tests in make test output

make test
100%: Checks: 145, Failures: 0, Errors: 0

make sudotest
shell-init: error retrieving current directory: getcwd: cannot access parent directories: Permission denied
job-working-directory: error retrieving current directory: getcwd: cannot access parent directories: Permission denied
100%: Checks: 147, Failures: 0, Errors: 0

sonertari commented 4 years ago

Can you disable (comment out) the call to the function bufferevent_openssl_set_allow_dirty_shutdown() in pxy_conn_autossl_peek_and_upgrade() in the file pxyconn.c, and then recompile and try if it is going to crash still? That seems to be one of the main differences in autossl between 0.5.4 and 0.5.5.

Btw, I guess the command line you have provided is not the actual command line you have used, because it is missing the -c option. Otherwise, sslsplit would quit with an error like "no CA cert specified (-c)". And I am surprised that the version of OpenSSL on osx is so old.

faperea commented 2 years ago

Same problem here. I commented out the line and the problem persists!

faperea commented 2 years ago

I tried with "ssl" and it stops also:

Initialized 16 connection handling threads
Started 16 connection handling threads
Starting main event loop.
SNI peek: [n/a] [complete]
Child pid 12268 killed by signal 11

sonertari commented 2 years ago

Can you try the underlying-bevs-issue#303 branch and report back please? It has a couple of fixes for autossl at least.

Otherwise, enabling the DEBUG_PROXY switch in GNUmakefile can probably give us more info.

faperea commented 2 years ago

Already tested the branch underlying-bevs-issue#303 and the failure continues. This is with "ssl":

Initialized 16 connection handling threads
Started 16 connection handling threads
Starting main event loop.
Garbage collecting caches started.
Garbage collecting caches done.
SNI peek: [n/a] [complete]
Child pid 2792 killed by signal 11

And this with "autossl":

Initialized 16 connection handling threads
Started 16 connection handling threads
Starting main event loop.
Connecting to [10.233.18.25]:12697
tcp 192.168.23.99 40975 10.233.18.25 12697
TCP connected to [10.233.18.25]:12697
TCP connected from [192.168.23.99]:40975
Checking for a client hello
Peek found ClientHello
Child pid 2851 killed by signal 11

faperea commented 2 years ago

This is with DEBUG_PROXY switch in GNUmakefile enabled. Let me know if you require more info:

Initialized 16 connection handling threads
Started 16 connection handling threads
Starting main event loop.
0x55ae931a0200             pxy_conn_ctx_new
            0x55ae931a0470 pxy_bufferevent_setup
Connecting to [10.233.18.25]:12697
0x55ae931a0200 0x55ae931a0470 eventcb dst connected
            0x7fa81c000cd0 pxy_bufferevent_setup
tcp 192.168.23.99 50021 10.233.18.25 12697
TCP connected to [10.233.18.25]:12697
TCP connected from [192.168.23.99]:50021
0x55ae931a0200 0x7fa81c000cd0 src readcb
Checking for a client hello
Peek found ClientHello
Child pid 4101 killed by signal 11

sonertari commented 2 years ago

Unfortunately, those debug logs do not help either (I guess you have started sslsplit with the -D option, right?). The debug logs in sslsplit are not verbose.

So the other option are:

Use gdb: You can generate a dump file to use with gdb, and obtain a backtrace on the same system that sslsplit crashes. This is supposed to give us the exact location in the source code where the crash happens.
Try SSLproxy: SSLproxy supports the split mode of operation similar to sslsplit with the same proxyspec syntax, and provides very verbose debug logs, so if it crashes too we may get more info. Plus, if sslproxy does not crash, so we know it is really sslsplit. And other possibilities.
macOS: I should gain access to an osx test machine, reproduce those crashes, and try to fix myself (these crashes happen on an osx, right?). But this seems unlikely.

sonertari commented 2 years ago

Btw, I don't know your xnu version, but another possibility is that this may be about the header files under xnu. Please see the xnu folder in the sources.

sonertari commented 2 years ago

If this issue is on macOS Mojave 10.14.6 (which is the system the OP was using), can you please try the xnu-4903.270.47 branch? Since I don't have a macOS machine, this is a stab in the dark.

zentheripper commented 2 years ago

Hi,

1.) I have reproduced this issue on Kali Linux VM. When I use Ubuntu 20.04 VM and the exact same software and setup, sslsplit does not crash.

Scenario: sslsplit listen on port 4042 for SSL connection and forward to 4041. When I start application which is configured to connect to 4042 sslsplit crashes - this only happens on Kali, on Ubuntu with exact same software it doesn't happen.

Command: ─# sslsplit -c CA-cert.pem -k CA-key.pem ssl 192.168.203.134 4042 192.168.203.132 4041 -X first_agent_connect_kali.pcap

I have attached output of sslsplit -V and neofetch for both VMs in Kali_System.txt and Ubuntu_System.txt

2.) I have tried to use ssl-proxy but it doesn't work for my scenario, and I also need to dump decrypted traffic to pcap file which ssl-proxy I think is not capable of

I have attached output of ssl-proxy and log of application used to connect to it in ssl-proxy.txt. Connecting app log says "SSL certificate verification failed: unsupported certificate purpose" / SSL-PROXY says "TLS handshake error from 192.168.203.134:47962: local error: tls: bad record MAC"

3.) I've used gdb but when I issued bt - there was no backtrace. Although I was able to find a coredump and have used coredumpctl to dump it. Attached coredump sslsplit_coredump and also gdb output and coredumpctl output in gdb_coredump.txt

You will find all attachments in the .zip file attachments.zip

Please tell me if any additional info is needed.

P.S. - I have updated and upgraded both VMs and rebooted before repeating test

zentheripper commented 2 years ago

Update: I've cloned github repository for sslsplit and now it's working fine in Kali Linux Output of sslsplit -V for the working scenario:

# /home/kali/Downloads/sslsplit/sslsplit -V
SSLsplit 0.5.5-12-ge17de84 (built 2022-07-22)
Copyright (c) 2009-2019, Daniel Roethlisberger <daniel@roe.ch>
https://www.roe.ch/SSLsplit
Build info: V:GIT
Features: -DHAVE_NETFILTER
NAT engines: netfilter* tproxy
netfilter: IP_TRANSPARENT IP6T_SO_ORIGINAL_DST
Local process info support: no
compiled against OpenSSL 3.0.3 3 May 2022 (30000030)
rtlinked against OpenSSL 3.0.3 3 May 2022 (30000030)
OpenSSL has support for TLS extensions
TLS Server Name Indication (SNI) supported
OpenSSL is thread-safe with THREADID
OpenSSL has engine support
Using SSL_MODE_RELEASE_BUFFERS
SSL/TLS protocol availability: tls10 tls11 tls12 
SSL/TLS algorithm availability: !SHA0 RSA DSA ECDSA DH ECDH EC
OpenSSL option availability: SSL_OP_NO_COMPRESSION SSL_OP_NO_TICKET SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION SSL_OP_TLS_ROLLBACK_BUG
compiled against libevent 2.1.12-stable
rtlinked against libevent 2.1.12-stable
compiled against libnet 1.1.6
rtlinked against libnet 1.1.6
compiled against libpcap n/a
rtlinked against libpcap 1.10.1 (with TPACKET_V3)

Output for the non-working one:

┌──(root㉿kali)-[/etc/sslsplit]
└─# sslsplit -V
SSLsplit 0.5.5 (built 2021-12-26)
Copyright (c) 2009-2019, Daniel Roethlisberger <daniel@roe.ch>
https://www.roe.ch/SSLsplit
Build info: V:FILE HDIFF:1 N:83c4edf
Features: -DHAVE_NETFILTER
NAT engines: netfilter* tproxy
netfilter: IP_TRANSPARENT IP6T_SO_ORIGINAL_DST
Local process info support: no
compiled against OpenSSL 1.1.1n  15 Mar 2022 (101010ef)
rtlinked against OpenSSL 1.1.1o  3 May 2022 (101010ff)
OpenSSL has support for TLS extensions
TLS Server Name Indication (SNI) supported
OpenSSL is thread-safe with THREADID
OpenSSL has engine support
Using SSL_MODE_RELEASE_BUFFERS
SSL/TLS protocol availability: tls10 tls11 tls12 
SSL/TLS algorithm availability: !SHA0 RSA DSA ECDSA DH ECDH EC
OpenSSL option availability: SSL_OP_NO_COMPRESSION SSL_OP_NO_TICKET SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION SSL_OP_TLS_ROLLBACK_BUG
compiled against libevent 2.1.12-stable
rtlinked against libevent 2.1.12-stable
compiled against libnet 1.1.6
rtlinked against libnet 1.1.6
compiled against libpcap n/a
rtlinked against libpcap 1.10.1 (with TPACKET_V3)
6 CPU cores detected

Seems like I was using a pretty old version and wasn't aware of it.

But on Ubuntu VM I am also using old build and it's surprisingly working:

root@ubuntu:/home/nxlog# sslsplit -V
SSLsplit 0.5.5 (built 2019-08-31)
Copyright (c) 2009-2019, Daniel Roethlisberger <daniel@roe.ch>
https://www.roe.ch/SSLsplit
Build info: V:FILE HDIFF:0 N:83c4edf
Features: -DHAVE_NETFILTER
NAT engines: netfilter* tproxy
netfilter: IP_TRANSPARENT IP6T_SO_ORIGINAL_DST
Local process info support: no
compiled against OpenSSL 1.1.1c  28 May 2019 (1010103f)
rtlinked against OpenSSL 1.1.1f  31 Mar 2020 (1010106f)
OpenSSL has support for TLS extensions
TLS Server Name Indication (SNI) supported
OpenSSL is thread-safe with THREADID
OpenSSL has engine support
Using SSL_MODE_RELEASE_BUFFERS
SSL/TLS protocol availability: tls10 tls11 tls12 
SSL/TLS algorithm availability: !SHA0 RSA DSA ECDSA DH ECDH EC
OpenSSL option availability: SSL_OP_NO_COMPRESSION SSL_OP_NO_TICKET SSL_OP_ALLOW_UNSAFE_LEGACY_RENEGOTIATION SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS SSL_OP_NO_SESSION_RESUMPTION_ON_RENEGOTIATION SSL_OP_TLS_ROLLBACK_BUG
compiled against libevent 2.1.11-stable
rtlinked against libevent 2.1.11-stable
compiled against libnet 1.1.6
rtlinked against libnet 1.1.6
compiled against libpcap n/a
rtlinked against libpcap 1.9.1 (with TPACKET_V3)
12 CPU cores detected

droe / sslsplit

Child pid killed by signal 11 #278