Bypassing com.android.okhttp.internal.io.RealConnection->connectTls (automatic exception patch)

[windowshopr]

Came across your article on how to defeat pinning with Frida and I'm trying to work my way through it as I'm a bit of newb, but I'm tryin!

For some context, I'm running Windows 10, an Android Pixel XL emulator, and OWASP ZAP as my proxy. I've installed the OWASP certificate onto the device, and I can now parse HTTPS traffic from the Chrome app. I'm wanting to now intercept traffic via apps!

I'm using the dating app Bumble as my first "target", so I've installed the APK file onto the device (which does NOT have Google Play Store as I've read that's important). I have the Frida server running in one terminal window as root (running frida-server-15.1.22-android-x86 which I don't know for sure if that's the correct server to be running, maybe x86_64 should be run instead? Anyway...)

So I fire up the App, where it just hangs on the main loading screen and I do not see any traffic from it in OWASP (again, even though it IS proxying internet requests fine, so that part IS working):

...and in a separate terminal window, I run the command:

frida --no-pause -U -l ./frida-script.js -f com.bumble.app

...and this is my resulting output (beware, it's long, and the trailing end just infinitely prints until the frida server dies):

frida --no-pause -U -l ./frida-script.js -f com.bumble.app
     ____
    / _  |   Frida 15.1.22 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://frida.re/docs/home/
   . . . .
   . . . .   Connected to Android Emulator 5554 (id=emulator-5554)
Spawned `com.bumble.app`. Resuming main thread!
[Android Emulator 5554::com.bumble.app ]-> ---
Unpinning Android app...
[+] SSLPeerUnverifiedException auto-patcher
[+] HttpsURLConnection (setDefaultHostnameVerifier)
[+] HttpsURLConnection (setSSLSocketFactory)
[+] HttpsURLConnection (setHostnameVerifier)
[+] SSLContext
[+] TrustManagerImpl
[ ] OkHTTPv3 (list)
[ ] OkHTTPv3 (cert)
[ ] OkHTTPv3 (cert array)
[ ] OkHTTPv3 ($okhttp)
[ ] Trustkit OkHostnameVerifier(SSLSession)
[ ] Trustkit OkHostnameVerifier(cert)
[ ] Trustkit PinningTrustManager
[ ] Appcelerator PinningTrustManager
[ ] OpenSSLSocketImpl Conscrypt
[ ] OpenSSLEngineSocketImpl Conscrypt
[ ] OpenSSLSocketImpl Apache Harmony
[ ] PhoneGap sslCertificateChecker
[ ] IBM MobileFirst pinTrustedCertificatePublicKey (string)
[ ] IBM MobileFirst pinTrustedCertificatePublicKey (string array)
[ ] IBM WorkLight HostNameVerifierWithCertificatePinning (SSLSocket)
[ ] IBM WorkLight HostNameVerifierWithCertificatePinning (cert)
[ ] IBM WorkLight HostNameVerifierWithCertificatePinning (string string)
[ ] IBM WorkLight HostNameVerifierWithCertificatePinning (SSLSession)
[ ] Conscrypt CertPinManager
[ ] CWAC-Netsecurity CertPinManager
[ ] Worklight Androidgap WLCertificatePinningPlugin
[ ] Netty FingerprintTrustManagerFactory
[ ] Squareup CertificatePinner (cert)
[ ] Squareup CertificatePinner (list)
[ ] Squareup OkHostnameVerifier (cert)
[ ] Squareup OkHostnameVerifier (SSLSession)
[+] Android WebViewClient (SslErrorHandler)
[ ] Android WebViewClient (WebResourceError)
[ ] Apache Cordova WebViewClient
[ ] Boye AbstractVerifier
Unpinning setup completed
---
  --> Bypassing Trustmanager (Android < 7) request
  --> Unexpected SSL verification failure, adding dynamic patch...
      Thrown by android.net.SSLCertificateSocketFactory->verifyHostname
      Attempting to patch automatically...
      [+] android.net.SSLCertificateSocketFactory->verifyHostname (automatic exception patch)
  --> Bypassing TrustManagerImpl checkTrusted
  --> Bypassing Trustmanager (Android < 7) request
  --> Bypassing android.net.SSLCertificateSocketFactory->verifyHostname (automatic exception patch)
  --> Bypassing TrustManagerImpl checkTrusted
  --> Bypassing TrustManagerImpl checkTrusted
  --> Unexpected SSL verification failure, adding dynamic patch...
  --> Bypassing TrustManagerImpl checkTrusted
  --> Unexpected SSL verification failure, adding dynamic patch...
      Thrown by com.android.okhttp.internal.io.RealConnection->connectTls
      Attempting to patch automatically...
      [+] com.android.okhttp.internal.io.RealConnection->connectTls (automatic exception patch)
      Thrown by com.android.okhttp.internal.io.RealConnection->connectTls
  --> Bypassing com.android.okhttp.internal.io.RealConnection->connectTls (automatic exception patch)
  --> Bypassing com.android.okhttp.internal.io.RealConnection->connectTls (automatic exception patch)
  --> Bypassing com.android.okhttp.internal.io.RealConnection->connectTls (automatic exception patch)
  --> Bypassing com.android.okhttp.internal.io.RealConnection->connectTls (automatic exception patch)
  --> Bypassing com.android.okhttp.internal.io.RealConnection->connectTls (automatic exception patch)
  --> Bypassing com.android.okhttp.internal.io.RealConnection->connectTls (automatic exception patch)
...........................

SO, what can a guy try from here? Would be cool to get some insight from the community on this one so I can get started on the app traffic track :P Thanks!!!

[pimterry]

I haven't used OWASP ZAP much - does it show any info for rejected TLS connections?

Your script is clearly running and connected to the app in Frida OK, so the possibilities are:

• The script is breaking something in the app, which errors before it makes the TLS connection at all.
• The app makes the TLS connection, but the script isn't successfully unpinning, and so the app rejects the certificate/throws an error after doing so regardless.
• The app isn't actually using your proxy settings for some reason, so it's making HTTPS connections elsewhere and not using your proxy.

If ZAP shows rejected TLS connections, then you can spot whether the 2nd case is happening. If there are rejected TLS connections, it's definitely case 2 - the connections are being made, and they are going to the proxy, but unpinning isn't working. If ZAP can't show that obviously I'm biased, but I'd suggest trying HTTP Toolkit instead. If ZAP can show that, but there are no failing connections shown, then it's one of the other two possibilities.

I'd suggest watching adb logcat -T1 while your app runs - you might be able to see errors shown there in the output which may provide some clues. If it turns out this is an error caused by the unpinning script itself, you can take a look through https://httptoolkit.tech/blog/android-reverse-engineering/ which is a good primer on extending Frida scripts to cover more advanced cases and dig into unusual issues like this.

[pimterry]

Ok, the automatic exception patch caught my eye so I dug a bit deeper. The output here means that the Frida script is patching a method not because it was a recognized method, but because it threw a suspicious SSL error, so the script is trying to automatically disable the method to stop that happening in future.

The relevant bit of script is here.

This is a backstop last-ditch fix, not a proper known-good fix for a recognized API, so I did a quick bit of digging to find the method listed: com.android.okhttp.internal.io.RealConnection->connectTls. I think this is some v2 version of OkHttp, looks like the method is here.

All the Frida script does in this case is look for the construction of SSLPeerUnverifiedException errors and try to disable that method entirely for future executions, assuming it's a "check certificate" method that we can just skip. In this case, it's not - connectTls does the actual connection setup, and disabling it like this will disable TLS connection setup entirely, which may be why you don't see any TLS connections (i.e. after one initial failing connection, this becomes case 1 above).

On the other hand though, this method was previously rejecting your certificate, so it does need some kind of fix. It looks like the specific code that's rejecting your cert is here. Interestingly, this is not certificate pinning (that's the next block). This code is just checking that the hostname for the given certificate matches the hostname the app is expecting. So if it requests example.com, it gets a certificate for example.com. That's what's failing in this case, not due to certificate pinning, which is why the script can't fix it.

It's hard to know 100% what this means, but it sounds a lot like this is a setup issue with your ZAP proxy, or an actual ZAP bug. For whatever reason, ZAP is not providing a matching certificate for the requested hostname. If you look at the exception (either in adb logcat or by logging it in the Frida script) you might be able to see more info in the exception message, which appears to helpfully log some of these details.

To fix this, you either need to reconfigure your proxy so that it generates the correct certificate, or you need to add a check in the script to force address.getHostnameVerifier().verify(...) to always return true regardless of the cert's hostname.

[windowshopr]

Thanks for looking into this!

Seems like it's likely an OWASP thing, so I'm going to spend some time (after finals are done :P) getting HTTP Toolkit and trying it on there to traceback any TLS errors. I know that OWASP has a TLS Debug addin so I'll install that too and compare it with HTTP Toolkit to see if they both offer the info I need or if one is better than the other. I like Burp Suite, but I'm just a hobbyist and don't really want to spend the money on the pro version for persistence, so the free alternatives appeal to me :P

I'll also read through the areas of code you cited to get a better understanding of what's happening and try setting that flag to True/tinker with it from there. I can close the issue now, and if I get any decent insights I'll post my findings here. Thanks!!

[windowshopr]

Just a quick update, using HTTP Toolkit, I see there is a string of 400 Bad Request's that happen when the app is loaded. This is what I'm seeing (again, the app just sits on that main screen while a few of these requests are happening a few seconds apart):

I'm assuming this could be due to pinning, or the fact that I'm using an emulator, or a combination of the two! I'm not sure yet. I'll keep playing with the unpinning js package as you suggested and look into the code a bit when I get a chance.

[pimterry]

Hmm, not sure! That is normally what you'd see if HTTP Toolkit receives non-HTTP traffic.

To explain a little: when HTTP Toolkit receives a connection, it keeps doing fake TLS handshakes (unwrapping any levels of proxying and direct HTTPS setup) until it receives something that's not TLS, and then it assumes that it's reached plain-text HTTP and tries to parse it as an HTTP request.

If that last step fails, you see something like this, where HTTP Toolkit gets something that should be an HTTP request, but it's complete gibberish that it can't parse so it just sends an error response.

It's hard to know what could cause this. It could be a bug caused by certificate unpinning, or it could be that the Bumble app is using a non-HTTP protocol (wrapped in TLS - you can tell because there's an https shown in the URL and the domain name from the requested certificate is shown). It's a bit unusual to use anything but HTTP in mobile apps like this, but not unheard of.

Notably though, if it's not HTTP then you can probably avoid intercepting this. In addition to setting the HTTP proxy on the device, which captures all well-behaved HTTP(S), HTTP Toolkit's Android VPN also captures all TCP traffic on a few target ports. That lets you capture traffic that ignores proxy settings, but can capture unrelated traffic sometimes. The default ports used are defined here and there's a button to configure the ports in the Android app when the app is connected. If this isn't HTTP at all, then it should be ignoring the proxy settings, so if you remove the ports redirection there then these broken requests should stop appearing.

If they keep appearing, then this is coming from a client that is actively using the HTTP proxy settings. It they disappear, then it's something that's ignoring the HTTP proxy settings, which pretty strongly suggests it's not HTTP at all.

Does that make sense?

[windowshopr]

It does! Thanks for your continued advice, but I'm still hitting a wall.

I've tried shuffling some ports around in an attempt to get ALL traffic, and the output of the .js script looks different, but is performing similarly as before, but this is the newest output:

     ____
    / _  |   Frida 15.1.22 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://frida.re/docs/home/
   . . . .
   . . . .   Connected to Android Emulator 5554 (id=emulator-5554)
Spawned `com.bumble.app`. Resuming main thread!
[Android Emulator 5554::com.bumble.app ]-> ---
Unpinning Android app...
[+] SSLPeerUnverifiedException auto-patcher
[+] HttpsURLConnection (setDefaultHostnameVerifier)
[+] HttpsURLConnection (setSSLSocketFactory)
[+] HttpsURLConnection (setHostnameVerifier)
[+] SSLContext
[+] TrustManagerImpl
[ ] OkHTTPv3 (list)
[ ] OkHTTPv3 (cert)
[ ] OkHTTPv3 (cert array)
[ ] OkHTTPv3 ($okhttp)
[ ] Trustkit OkHostnameVerifier(SSLSession)
[ ] Trustkit OkHostnameVerifier(cert)
[ ] Trustkit PinningTrustManager
[ ] Appcelerator PinningTrustManager
[ ] OpenSSLSocketImpl Conscrypt
[ ] OpenSSLEngineSocketImpl Conscrypt
[ ] OpenSSLSocketImpl Apache Harmony
[ ] PhoneGap sslCertificateChecker
[ ] IBM MobileFirst pinTrustedCertificatePublicKey (string)
[ ] IBM MobileFirst pinTrustedCertificatePublicKey (string array)
[ ] IBM WorkLight HostNameVerifierWithCertificatePinning (SSLSocket)
[ ] IBM WorkLight HostNameVerifierWithCertificatePinning (cert)
[ ] IBM WorkLight HostNameVerifierWithCertificatePinning (string string)
[ ] IBM WorkLight HostNameVerifierWithCertificatePinning (SSLSession)
[ ] Conscrypt CertPinManager
[ ] CWAC-Netsecurity CertPinManager
[ ] Worklight Androidgap WLCertificatePinningPlugin
[ ] Netty FingerprintTrustManagerFactory
[ ] Squareup CertificatePinner (cert)
[ ] Squareup CertificatePinner (list)
[ ] Squareup OkHostnameVerifier (cert)
[ ] Squareup OkHostnameVerifier (SSLSession)
[+] Android WebViewClient (SslErrorHandler)
[ ] Android WebViewClient (WebResourceError)
[ ] Apache Cordova WebViewClient
[ ] Boye AbstractVerifier
Unpinning setup completed
---
  --> Bypassing Trustmanager (Android < 7) request
  --> Bypassing Trustmanager (Android < 7) request
  --> Bypassing Trustmanager (Android < 7) request
  --> Bypassing Trustmanager (Android < 7) request
  --> Bypassing TrustManagerImpl checkTrusted
  --> Bypassing TrustManagerImpl checkTrusted
  --> Bypassing TrustManagerImpl checkTrusted
  --> Bypassing Trustmanager (Android < 7) request
  --> Bypassing Trustmanager (Android < 7) request
  --> Bypassing Trustmanager (Android < 7) request

...with the last bits appearing whenever the app attempts one of those unparseable requests. Almost seems like I should be proxying through WireShark to see what's going on here, but for a newbie, I've exhausted what I know how to do so far haha. Need a pro to look into it and tell me what's going on here, like you have been! :P

Thanks for your help! If I figure this out, I'll post again.

[pimterry]

Hard to tell any more from that, I'm afraid. Trying with Wireshark to look lower-level is probably a good move, although you'll probably find everything involved is encrypted such that Wireshark can't look at it, unfortunately.

I think my money is on this being non-HTTP traffic sent via TLS. In that case, there's no way to view this with HTTP Toolkit - it's just not HTTP at all. The certificate unpinning is firing for the TLS setup, so this non-HTTP traffic does trust your certificate and you can see tiny bits of its garbled messages in HTTP Toolkit, but not much more.

No idea what this non-HTTP protocol could be, but I'd be very interested if you do manage to find out! I think the next step would have to be building your own TLS proxy, manually examining the raw packets here, trying to work out what protocol it is, and then from there trying to handle that manually. If it's something common then you can probably use an off-the-shelf implementation to accept the connection and reply or forward messages from there. No idea myself though, it's nothing obviously recognizable.

Alternatively you might be able to run HTTP Toolkit's internals manually, and then modify Mockttp to let you look and handle the raw packet data it receives, here: https://github.com/httptoolkit/mockttp/blob/5c82c16f898ebc19edb65ccfc54169140ccc4b40/src/server/mockttp-server.ts#L653-L658. That would let you use all the rest of HTTP Toolkit's tools, but it'd take a little bit of setup to get it all working. None of these options are easy I'm afraid, sorry!