httptoolkit / frida-interception-and-unpinning

Frida scripts to directly MitM all HTTPS traffic from a target mobile application
https://httptoolkit.com/android/
GNU Affero General Public License v3.0
871 stars 176 forks source link

com.hulu.plus failure #55

Closed 3052 closed 6 months ago

3052 commented 6 months ago

using this command:

frida -U `
-l config.js `
-l android/android-certificate-unpinning.js `
-f com.hulu.plus

after entering password, if you click LOG IN you get this:

Hmm. Something’s up. Please check your internet settings and try again. If all’s fine on your end, visit our Help Center.

If I disable proxy and try again, it works as expected. I can share account if need be. result:

     ____
    / _  |   Frida 16.1.4 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://frida.re/docs/home/
   . . . .
   . . . .   Connected to Android Emulator 5554 (id=emulator-5554)
Spawning `com.hulu.plus`...

*** Starting scripts ***
Spawned `com.hulu.plus`. Resuming main thread!
[Android Emulator 5554::com.hulu.plus ]->
    === Disabling all recognized unpinning libraries ===
[+] javax.net.ssl.HttpsURLConnection setDefaultHostnameVerifier
[+] javax.net.ssl.HttpsURLConnection setSSLSocketFactory
[+] javax.net.ssl.HttpsURLConnection setHostnameVerifier
[+] javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)
[ ] com.android.org.conscrypt.CertPinManager isChainValid
[+] com.android.org.conscrypt.CertPinManager checkChainPinning
[+] android.security.net.config.NetworkSecurityConfig $init(*) (0)
[+] android.security.net.config.NetworkSecurityConfig $init(*) (1)
[+] com.android.okhttp.Address $init
[ ] okhttp3.CertificatePinner check(String, List)
[ ] okhttp3.CertificatePinner check(String, Certificate)
[ ] okhttp3.CertificatePinner check(String, Certificate;[])
[ ] okhttp3.CertificatePinner check$okhttp
[!] Matched class okhttp3.CertificatePinner but could not patch any methods
[ ] com.squareup.okhttp.CertificatePinner *
[ ] com.datatheorem.android.trustkit.pinning.PinningTrustManager *
[ ] appcelerator.https.PinningTrustManager *
[ ] nl.xservices.plugins.sslCertificateChecker *
[ ] com.worklight.wlclient.api.WLClient *
[ ] com.worklight.wlclient.certificatepinning.HostNameVerifierWithCertificatePinning *
[ ] com.worklight.androidgap.plugin.WLCertificatePinningPlugin *
[ ] com.commonsware.cwac.netsecurity.conscrypt.CertPinManager *
[ ] io.netty.handler.ssl.util.FingerprintTrustManagerFactory *
[ ] com.silkimen.cordovahttp.CordovaServerTrust *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyHostnameVerifier *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyTrustManager *
== Certificate unpinning completed ==
*** Scripts completed ***

 => android.security.net.config.NetworkSecurityConfig $init(*) (0)
 => com.android.okhttp.Address $init
 => com.android.okhttp.Address $init
 => javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)
pimterry commented 6 months ago

I don't have a Hulu account, so it's going to be difficult to reproduce this if it requires login.

Is there anything shown in the ADB logs, which might provide a clue about where this is being rejected? A stack trace would be super helpful.

I'd suggest also trying the fallback unpinning script, and see if that comes up with any clues - that'll log any code that throws certificate errors, which may point you in the right direction.

[!] Matched class okhttp3.CertificatePinner but could not patch any methods

This is very suspicious - that implies that OkHttp3 is included unobfuscated in the app, and so presumably being used for networking, but with some version that we can't patch automatically. If that's what's rejecting the connection here (can't tell from this output alone) then that would explain this.

Can you look in the APK, see what methods that class has, and compare it to the overloads we try to patch here? If there's just a signature missing somewhere then we could easily add that.

3052 commented 6 months ago

I don't have a Hulu account, so it's going to be difficult to reproduce this if it requires login.

I just signed up for the free trial. I can share the details if it helps. however I did notice that if you cancel during the trial (which I did), you immediately lose access. so if you do need to borrow an account, I might need to create a new one.

Is there anything shown in the ADB logs, which might provide a clue about where this is being rejected? A stack trace would be super helpful.

below is just the capture from entering password, clicking login and the error message appearing. I can try to capture more if need be:

--------- beginning of main
11-07 17:26:40.573  1454  1476 W audio_hw_generic: Not supplying enough data to HAL, expected position 771435 , only wrote 619200
--------- beginning of system
11-07 17:26:43.648  1698  1748 W ActivityManager: Unable to start service Intent { act=com.google.android.gms.drive.ApiService.RESET_AFTER_BOOT flg=0x4 cmp=com.google.android.gms/.drive.api.ApiService (has extras) } U=0: not found
11-07 17:26:43.669  2149  4303 W System  : ClassLoader referenced unknown path: /system/priv-app/PrebuiltGmsCore/lib/x86
11-07 17:26:43.725  2149  2642 I TelephonySpam: TelephonySpamChimeraService - Running Telephony Spam Chimera Service
11-07 17:26:44.083  1454  1475 W audio_hw_generic: Not supplying enough data to HAL, expected position 619274 , only wrote 619200
11-07 17:26:44.186  3992  4038 W TransportRuntime: Error scheduling event Log tag "TRuntime.SQLiteEventStore" exceeds limit of 23 characters
11-07 17:26:44.282  3992  4038 W TransportRuntime: Error scheduling event Log tag "TRuntime.SQLiteEventStore" exceeds limit of 23 characters

I'd suggest also trying the fallback unpinning script, and see if that comes up with any clues - that'll log any code that throws certificate errors, which may point you in the right direction.

here is the result. note I did a "sort unique" on the last section because a lot of duplicates. I can provide full log if its helpful:

     ____
    / _  |   Frida 16.1.4 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://frida.re/docs/home/
   . . . .
   . . . .   Connected to Android Emulator 5554 (id=emulator-5554)
Spawning `com.hulu.plus`...

*** Starting scripts ***
Spawned `com.hulu.plus`. Resuming main thread!
[Android Emulator 5554::com.hulu.plus ]->
    === Disabling all recognized unpinning libraries ===
[+] javax.net.ssl.HttpsURLConnection setDefaultHostnameVerifier
[+] javax.net.ssl.HttpsURLConnection setSSLSocketFactory
[+] javax.net.ssl.HttpsURLConnection setHostnameVerifier
[+] javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)
[+] com.android.org.conscrypt.CertPinManager isChainValid
[ ] com.android.org.conscrypt.CertPinManager checkChainPinning
[+] android.security.net.config.NetworkSecurityConfig $init(*) (0)
[+] android.security.net.config.NetworkSecurityConfig $init(*) (1)
[ ] com.android.okhttp.Address $init(String, int, Dns, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
[+] com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
[ ] okhttp3.CertificatePinner check(String, List)
[ ] okhttp3.CertificatePinner check(String, Certificate)
[ ] okhttp3.CertificatePinner check(String, Certificate;[])
[ ] okhttp3.CertificatePinner check$okhttp
[!] Matched class okhttp3.CertificatePinner but could not patch any methods
[ ] com.squareup.okhttp.CertificatePinner *
[ ] com.datatheorem.android.trustkit.pinning.PinningTrustManager *
[ ] appcelerator.https.PinningTrustManager *
[ ] nl.xservices.plugins.sslCertificateChecker *
[ ] com.worklight.wlclient.api.WLClient *
[ ] com.worklight.wlclient.certificatepinning.HostNameVerifierWithCertificatePinning *
[ ] com.worklight.androidgap.plugin.WLCertificatePinningPlugin *
[ ] com.commonsware.cwac.netsecurity.conscrypt.CertPinManager *
[ ] io.netty.handler.ssl.util.FingerprintTrustManagerFactory *
[ ] com.silkimen.cordovahttp.CordovaServerTrust *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyHostnameVerifier *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyTrustManager *
== Certificate unpinning completed ==
[+] TLS error auto-patcher
*** Scripts completed ***

 => android.security.net.config.NetworkSecurityConfig $init(*) (1)
 => android.security.net.config.NetworkSecurityConfig $init(*) (0)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)

      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      Thrown by java.security.cert.CertificateParsingException-><init>
      Thrown by okhttp3.CertificatePinner->b
      Thrown by okhttp3.internal.platform.android.AndroidCertificateChainCleaner->a
      [ ] Failed to automatically patch failure
      [ ] Unrecognized TLS error - this must be patched manually
      [+] okhttp3.CertificatePinner->b (fallback OkHttp patch)
 !!! --- Unexpected TLS failure --- !!!
 => Fallback OkHttp patch
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.org.conscrypt.CertPinManager isChainValid
TypeError: cannot read property 'overloads' of undefined
TypeError: not a function

Can you look in the APK, see what methods that class has,

OK in the file:

sources\okhttp3\CertificatePinner.java

I see these:

public /* synthetic */ CertificatePinner(Set set, CertificateChainCleaner certificateChainCleaner, int i10, DefaultConstructorMarker defaultConstructorMarker) {
public CertificatePinner(Set<Pin> set, CertificateChainCleaner certificateChainCleaner) {
public boolean equals(Object obj) {
public final CertificateChainCleaner d() {
public final CertificatePinner e(CertificateChainCleaner certificateChainCleaner) {
public final List<Pin> c(String str) {
public final void a(final String str, final List<? extends Certificate> list) throws SSLPeerUnverifiedException {
public final void b(String str, Function0<? extends List<? extends X509Certificate>> function0) {
public int hashCode() {

and compare it to the overloads we try to patch here? If there's just a signature missing somewhere then we could easily add that.

not seeing anything matching check

3052 commented 6 months ago

this worked a couple of times:

+++ b/android/android-certificate-unpinning.js
@@ -223,7 +223,7 @@ const PINNING_FIXES = {

     'okhttp3.CertificatePinner': [
         {
-            methodName: 'check',
+            methodName: 'a',
             overload: ['java.lang.String', 'java.util.List'],
             replacement: () => NO_OP
         },

but it seems to be a race condition or something, as it only works sometimes. like it might fail the first time, but then if I restart the app it will work. not sure.

pimterry commented 6 months ago

There's a lot of overlap between this log and the errors in https://github.com/httptoolkit/frida-interception-and-unpinning/issues/56, so I would look into the configuration setup part related to that first.

I haven't bothered with the full free trial account setup as I'm a bit short on time (if you could share the account details that would help) but up until that point everything intercepts correctly for me out of the box (though yes, I understand that might not fully reproduce the issue).

The method obfuscation is interesting, and yes using the obfuscated name should ensure that patch does get correctly applied.

The fallback script can actually handle this automatically for you though - that's what the [+] okhttp3.CertificatePinner->b (fallback OkHttp patch) line in the output above is doing. That method has thrown a certificate error, so the script has checked the shape of it (ignoring the names, just looking at the exception method and types of parameters etc), recognized that it's an obfuscated version of a method it knows, and patched it so that all future errors (after that first failure) are avoided. Failing once is generally not an issue, as apps retry automatically to deal with unreliable networks etc.

That means that case should probably handle itself if you just use the fallback script.

I think the best option is to fix #56 first and then once that's working, try this again with the fallback script too, and share any more issues you run into with that.

3052 commented 6 months ago

I haven't bothered with the full free trial account setup as I'm a bit short on time (if you could share the account details that would help)

OK send your public key, or email or something

but up until that point everything intercepts correctly for me out of the box (though yes, I understand that might not fully reproduce the issue).

yeah thats a big caveat. the problem happens after you enter the password, so thats an important part of the issue

That means that case should probably handle itself if you just use the fallback script.

I am confused. I did use the fallback script, after your suggestion. the result is in this comment:

https://github.com/httptoolkit/frida-interception-and-unpinning/issues/55#issuecomment-1800430369

pimterry commented 6 months ago

I am confused. I did use the fallback script,

Yes - I'm saying that when you did so, it probably handled this correctly, so that your patch isn't necessary, and any remaining problems have some other cause.

The original concern was the [!] Matched class okhttp3.CertificatePinner but could not patch any methods message - the [+] okhttp3.CertificatePinner->b (fallback OkHttp patch) output when you used the fallback script means we can (probably) now ignore that as a problem.

Get #56 working, come back to this with a working setup for that case, and then we can look at any remaining issues here knowing that the core setup works correctly.

3052 commented 6 months ago

OK this seems to be another issue surrounding MITM Proxy. using this command:

> frida -U `
-l config.js `
-l android/android-certificate-unpinning.js `
-l android/android-certificate-unpinning-fallback.js `
-f com.hulu.plus

I ran it twice with HTTP Toolkit and it works as expected. it would be cool if someone could figure out how to get this working with MITM Proxy, but I guess for now I will just stick to HTTP Toolkit. I will close this for now, pending any comments on the similar NBC issue. thanks for the help.