search

aapooksman / certmitm

A tool for testing for certificate validation vulnerabilities of TLS connections made by a client device or an application.
MIT License
454 stars 41 forks source link

Working on building unit tests #6

Open andyacer opened 3 months ago

andyacer commented 3 months ago

Hey @aapooksman, just wanted to let you know that a colleague and I are working on adding some unit tests to this project to try to ensure each of the test cases is working as intended post-install. You don't have something like in the works already, do you?

We've attempted to abstract this tool to where client devices (eg: phones) connect over Wireguard to a central sever and then we enable or disable certmitm on all port 443 traffic on that connection. We're not getting the behavior we expect though. For example, on a jailbroken device with SSL Kill Switch enabled, almost all connections should be intercepted. But that's not the case, many are not and deemed "secure." I'm hoping that unit testing can help us figure out what's going on.

aapooksman commented 3 months ago

I do not have unit tests in the works so feel free to add some.

One easy way to test that certmitm is running well, is to run curl from a Linux machine that has it's traffic intercepted by certmitm.

First running curl normally against a domain should result in different kinds of errors. Then running curl with the "insecure" flag should result in the the traffic being intercepted. See the example outputs from curl and certmitm below.

You could try this out and then replicate the tests as unit tests. Note that the TLS alerts are sent by the victim application and thus the result of the certmitm tests will differ between client TLS implementation. You could on the other hand check the curl error code from the victim side to ensure that the tests are running properly.

test@certmitm$ python3 certmitm.py -v -l 9900
INFO - 10.0.0.140: 216.58.211.238:443:google.com for test self_signed = [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:992)
INFO - 10.0.0.140: 216.58.211.238:443:google.com for test replaced_key = [SSL: TLSV1_ALERT_DECRYPT_ERROR] tlsv1 alert decrypt error (_ssl.c:992)
INFO - 10.0.0.140: 216.58.211.238:443:google.com for test real_cert_letsencrypt = Nothing received
INFO - 10.0.0.140: 216.58.211.238:443:google.com for test real_cert_CA_letsencrypt = [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:992)
CRITICAL - 10.0.0.140: 216.58.209.164:443:www.google.com for test self_signed = data intercepted!
CRITICAL - 10.0.0.140: 216.58.209.164:443:www.google.com for test replaced_key = data intercepted!
CRITICAL - 10.0.0.140: 216.58.209.164:443:www.google.com for test real_cert_letsencrypt = data intercepted!
CRITICAL - 10.0.0.140: 216.58.209.164:443:www.google.com for test real_cert_CA_letsencrypt = data intercepted!
test@victim:~$ curl https://google.com
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
test@victim:~$ curl https://google.com
curl: (35) error:0407008A:rsa routines:RSA_padding_check_PKCS1_type_1:invalid padding
test@victim:~$ curl https://google.com
curl: (60) SSL: no alternative certificate subject name matches target host name 'google.com'
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
test@victim:~$ curl https://google.com
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
test@victim:~$ curl https://google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="https://www.google.com/">here</A>.
</BODY></HTML>
test@victim:~$ curl --insecure https://www.google.com
curl: (52) Empty reply from server
test@victim:~$ curl --insecure https://www.google.com
curl: (52) Empty reply from server
test@victim:~$ curl --insecure https://www.google.com
curl: (52) Empty reply from server
test@victim:~$ curl --insecure https://www.google.com
curl: (52) Empty reply from server
test@victim:~$ curl --insecure https://www.google.com
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="fi"><head><meta content="text/html;

We're not getting the behavior we expect though. For example, on a jailbroken device with SSL Kill Switch enabled, almost all connections should be intercepted. But that's not the case, many are not and deemed "secure." I'm hoping that unit testing can help us figure out what's going on.

@andyacer are you getting the same results with other intercepting tools such as Burp or mitmproxy? I would guess that many applications do not respect the system TLS handling and do some kind of certificate pinning that "SSL Kill Switch" cannot disable. If other tools can intercept the connections but certmitm does not, then please submit an issue about that with some more details.

andyacer commented 3 months ago

Hey thanks for the fast reply! Re: SSL Kill Switch and potential cert pinning, we don't think we're running into that scenario. It seems like certmitm is inconsistent even with Safari. So, traffic from Safari should always be intercepted (or always not) but that's not what we see happening.

One of our testers brought up that Safari prefers TLS 1.3 and perhaps TLS 1.3 handling could have some issue in certmitm, whereas TLS 1.2 is good to go. But it's unclear how that could factor into this in a way that's intermittent. If Safari always preferred TLS 1.3 and certmitm always handled it incorrectly we ought to see consistent FPs or false negatives.

We'll keep looking at it and hopefully be able to submit a PR with a test suite at some point 🤞 Tests directly with Python have all worked as expected so far.

aapooksman commented 3 months ago

What kind of error messages are you getting when the traffic is not being intercepted when it should? Do you get a TLS error sent from the client or does the client simply not send data?

If the client does not send data, it could have accepted the certificate and done some kind of probing (check something from the certificate or TLS parameters) before the actual connection. These kind of connections are marked as failed tests with certmitm as they cannot be distinguished from similar connections where the certificates are accepted on TLS level but checked on application level.

You could try to run all tests multiple times with the "--retrytests x" flag to see if this is the case. If this kind of client behavior is common, then certmitm could include the ability to rerun tests for certain cases (client accepts certificate for generated certificates but did not send anything)

It would be helpful to see the certmitm logs from this happening (-v is enough). You can also send the logs to me privately to look at if you are not comfortable on posting them to the issue.

Additionally, I've seen some weird behavior from browsers when looking at their TLS traffic. Weird in terms of the TLS standard but quite understandable things such as anti-fingerprinting and sending multiple TLS connections out for a one HTTPS connection. I've not been tampering with TLS clients or browsers to make them accept certificates that they normally would not i.e. use certmitm for traffic analysis so I've not tried to tests browsers with certmitm. They have their own test suites for TLS and I expect them to work well.

However, these kind of issues might create false negatives in other clients as well and would be good to figure out what happens here to see if certmitm could be improved.