com.nbcuni.nbc: failure with fallback script

[3052]

using this command:

frida -U `
-l config.js `
-l android/android-certificate-unpinning.js `
-l android/android-certificate-unpinning-fallback.js `
-f com.nbcuni.nbc

with Android 7, I get this result:

     ____
    / _  |   Frida 16.1.4 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://frida.re/docs/home/
   . . . .
   . . . .   Connected to Android Emulator 5554 (id=emulator-5554)
Spawning `com.nbcuni.nbc`...

*** Starting scripts ***
Spawned `com.nbcuni.nbc`. Resuming main thread!
[Android Emulator 5554::com.nbcuni.nbc ]->
    === Disabling all recognized unpinning libraries ===
[+] javax.net.ssl.HttpsURLConnection setDefaultHostnameVerifier
[+] javax.net.ssl.HttpsURLConnection setSSLSocketFactory
[+] javax.net.ssl.HttpsURLConnection setHostnameVerifier
[+] javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)
[+] com.android.org.conscrypt.CertPinManager isChainValid
[ ] com.android.org.conscrypt.CertPinManager checkChainPinning
[+] android.security.net.config.NetworkSecurityConfig $init(*) (0)
[+] android.security.net.config.NetworkSecurityConfig $init(*) (1)
[ ] com.android.okhttp.Address $init(String, int, Dns, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
[+] com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
[ ] okhttp3.CertificatePinner *
[ ] com.squareup.okhttp.CertificatePinner *
[ ] com.datatheorem.android.trustkit.pinning.PinningTrustManager *
[ ] appcelerator.https.PinningTrustManager *
[ ] nl.xservices.plugins.sslCertificateChecker *
[ ] com.worklight.wlclient.api.WLClient *
[ ] com.worklight.wlclient.certificatepinning.HostNameVerifierWithCertificatePinning *
[ ] com.worklight.androidgap.plugin.WLCertificatePinningPlugin *
[ ] com.commonsware.cwac.netsecurity.conscrypt.CertPinManager *
[ ] io.netty.handler.ssl.util.FingerprintTrustManagerFactory *
[ ] com.silkimen.cordovahttp.CordovaServerTrust *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyHostnameVerifier *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyTrustManager *
== Certificate unpinning completed ==
[+] TLS error auto-patcher
*** Scripts completed ***

 => android.security.net.config.NetworkSecurityConfig $init(*) (1)
 => android.security.net.config.NetworkSecurityConfig $init(*) (0)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)
 => javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)

 !!! --- Unexpected TLS failure --- !!!

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function

 !!! --- Unexpected TLS failure --- !!!

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => javax.net.ssl.HttpsURLConnection setDefaultHostnameVerifier

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.org.conscrypt.CertPinManager isChainValid

 !!! --- Unexpected TLS failure --- !!!
      Thrown by java.security.cert.CertificateParsingException-><init>
      [ ] Failed to automatically patch failure
TypeError: cannot read property 'overloads' of undefined

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function

 !!! --- Unexpected TLS failure --- !!!
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
 => com.android.org.conscrypt.CertPinManager isChainValid

 !!! --- Unexpected TLS failure --- !!!
      Thrown by java.security.cert.CertificateParsingException-><init>
      [ ] Failed to automatically patch failure
TypeError: cannot read property 'overloads' of undefined

 !!! --- Unexpected TLS failure --- !!!
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
      Thrown by com.android.org.conscrypt.TrustManagerImpl->checkTrustedRecursive
      [ ] Failed to automatically patch failure
TypeError: not a function

 !!! --- Unexpected TLS failure --- !!!
      Thrown by okhttp3.internal.platform.android.d->a
      [ ] Unrecognized TLS error - this must be patched manually

[pimterry]

TypeError: not a function

This was due to a bug in the fallback script, now fixed.

Thrown by okhttp3.internal.platform.android.d->a

I strongly suspect this is just due to OkHttp catching & rethrowing existing errors so shouldn't happen if other errors are resolved.

TypeError: cannot read property 'overloads' of undefined

I haven't managed to reproduce this. That said, I think there's a strong clue here in the CertificateParsingException - is it possible that your certificate is invalid? If so, that would cause all of this.

In my environment, using either HTTP Toolkit ADB cert injection etc + the two unpinning scripts here, or using just the full set of Frida scripts here with no other device configuration/VPN, I can intercept this NBC app successfully and see everything.

The one case where I managed to reproduce that not a function error was when I disabled interception entirely (so the traffic was being sent directly) but then still used the unpinning scripts. In that case, the certificate being rejected is NBC's real certificate, which is being received unexpectedly where we're expecting the MitM certificate, and so is being rejected.

If your certificate in config.js is invalid, so that the traffic is intercepted but the cert used doesn't match your MitM certificate, that would be similar - and I think that could plausibly cause everything you're seeing here. Any chance that's what's happening?

In any case, I can happily intercept all traffic from this app on my machine with the scripts unmodified, so this must be something related to your setup.

[3052]

I haven't managed to reproduce this. That said, I think there's a strong clue here in the CertificateParsingException - is it possible that your certificate is invalid? If so, that would cause all of this.

I am using this:

https://github.com/httptoolkit/frida-interception-and-unpinning/files/13302673/mitmproxy-ca-cert.txt

which I install as a "user certificate". I have also tried installing it as a "system certificate", using this:

https://github.com/3052/blog/blob/main/2023-09/mitmproxy-cert/mitmproxy-cert.go

If your certificate in config.js is invalid, so that the traffic is intercepted but the cert used doesn't match your MitM certificate, that would be similar - and I think that could plausibly cause everything you're seeing here. Any chance that's what's happening?

here is that line in my code:

const CERT_PEM = `-----BEGIN CERTIFICATE-----
MIIDNTCCAh2gAwIBAgIUTFPBYqAZ0sRNwuWSQnZagnaXW14wDQYJKoZIhvcNAQEL
BQAwKDESMBAGA1UEAwwJbWl0bXByb3h5MRIwEAYDVQQKDAltaXRtcHJveHkwHhcN
MjIwNTA1MDgzNjI0WhcNMzIwNTA0MDgzNjI0WjAoMRIwEAYDVQQDDAltaXRtcHJv
eHkxEjAQBgNVBAoMCW1pdG1wcm94eTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBALo9aNKDgg8YILUJCuk/UE5sW54tDa2iRPIoyr015UiockWLARQLIHXl
i3suOJqAf+d1xV7KyfMT4a9szlIB7EOUlxhb0dxQguKMiEBuuLB5kWu6rjKbfPEN
P7zamyNzKCKF/1iGRCaZrHvC0Kcum/2+bybagd/n9BVsuwDBQpinwvrLBX+qPF9M
YiBtZKZZXYX7ls9HpBFiTCYECrUKsis6SvonPr7sujMIlgEK8g0H8KTd/sLyHhq8
p94guzD4XlDXLWgwwLW5dazfYL2Qv4nnOWiB+heFH3PAJrJ1uFIdCQteWdTLXgr0
I8Yg51GLTFUBjCWArKddy06gITqCSvkCAwEAAaNXMFUwDwYDVR0TAQH/BAUwAwEB
/zATBgNVHSUEDDAKBggrBgEFBQcDATAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYE
FNr/5OMzIXrfKNXDEdGeX8VZkljtMA0GCSqGSIb3DQEBCwUAA4IBAQBHN13akooZ
DonZhA4rWgoNk1BVzqDLN5Ye2XNMEGh/aY310FYvVXAVdPSZ9W4tGT/hKQUBJmQp
l9jz84ICjCPZHr65fs6FQE9zHNaXc7xlyxNWi9CHkvyiqJ6Sw27FllcjHGf8fWpC
9vWLMy+WE+1w0zhjiZN9ALqrcktghAxgKa6dW6Q5T2K0l7gCbIXfrI3hsLgGW9YW
pW8CwqdKdHQjYYzPm0x/B7kIBMWDTJpK9jHz14ZWImkLaHhRSnU/qXMu4RMZPtJh
93RDMIm6w26JYjefPJo64F/KbGqH1gAKCOQrZyAn7zvF/omUWjnml7Zeydw+zefE
INhklp7JmApa
-----END CERTIFICATE-----`;

In any case, I can happily intercept all traffic from this app on my machine with the scripts unmodified, so this must be something related to your setup.

I am using Android Studio with MITM Proxy. I have used this setup in the past with this:

https://github.com/httptoolkit/frida-interception-and-unpinning/blob/main/frida-script.js

but not had much luck with the new scripts:

https://github.com/httptoolkit/frida-interception-and-unpinning/tree/main/android

but to be fair, the version of NBC I am trying to intercept now is using stronger tactics, so that could be a factor.

[pimterry]

here is that line in my code:

That config setup looks correct.

Just FYI, I'm not sure where your CA cert has come from, but it's not actually a fully valid CA certificate - it doesn't include a country in the subject (required) and it (incorrectly) has extended key usage options. It might work in some cases, but in many others it will be rejected as invalid. That could plausibly cause the parsing error you're seeing here maybe, although I can't be sure.

You can validate whether certificates fulfil standard requirements with https://crt.sh/lintcert.

which I install as a "user certificate". I have also tried installing it as a "system certificate"

A user certificate will definitely not be sufficient for intercepting 3rd party apps like this, you'll need to get full system interception working as a prerequisite.

Your script might work, but not for any devices using Android 14 or the latest Conscrypt APEX module (see https://httptoolkit.com/blog/android-14-install-system-ca-certificate/). I've seen those changes apply in rooted custom ROMs installs of slightly older Android versions, which are presumably pulling in the latest Conscrypt changes independently of the core OS version.

the version of NBC I am trying to intercept now is using stronger tactics, so that could be a factor.

Can you share the version number? I've tested against 9.4.1, and I can intercept everything there.

---

I'm afraid I can't really help with the full details of your interception setup, but you need to sort that out before investigating the scripts further. I'd suggest putting your current config aside, resetting the device by removing proxy settings and certificates (or rebooting to drop tmpfs mounts), and then either:

• Using all scripts, as in the README, to fully manage interception that way, and confirming that that works
• Or, using HTTP Toolkit for setup with ADB, confirming that that installs system certificates successfully (it'll show you in the Android app), and confirming that doing that plus the two unpinning scripts works correctly (bear in mind HTTP Toolkit will generate its own certificate, so you'll need to update the config).

In my testing yesterday, both of those options worked immediately. If the former doesn't work, I suspect that means it is indeed a problem with the CA cert, and you might want to try the latter instead, but that's at least a clear pointer to the problem.

Once you have something that does work, you can start trying to work out what the difference is between that and your current setup that doesn't, and get that fixed.

[3052]

Just FYI, I'm not sure where your CA cert has come from, but it's not actually a fully valid CA certificate - it doesn't include a country in the subject (required) and it (incorrectly) has extended key usage options. It might work in some cases, but in many others it will be rejected as invalid. That could plausibly cause the parsing error you're seeing here maybe, although I can't be sure.

OK, so where is the HTTP Tookit certificate? I can try swapping mine for that one. I dont know that much about this stuff, but I assume if I put the HTTP Toolkit certificate in the same path that MITM Proxy is normally looking, then it will try to use it. I found these but not sure if they are the correct ones:

https://github.com/httptoolkit/httptoolkit-ui/tree/main/test/fixtures

Can you share the version number? I've tested against 9.4.1, and I can intercept everything there.

same

• Using all scripts, as in the README, to fully manage interception that way, and confirming that that works

that seems to be even worse result. using this command:

frida -U `
-l config.js `
-l native-connect-hook.js `
-l android/android-proxy-override.js `
-l android/android-system-certificate-injection.js `
-l android/android-certificate-unpinning.js `
-l android/android-certificate-unpinning-fallback.js `
-f com.nbcuni.nbc

the app crashes immediately with:

Unfortunately, NBC has stopped.

log with debug=true:

     ____
    / _  |   Frida 16.1.5 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://frida.re/docs/home/
   . . . .
   . . . .   Connected to Android Emulator 5554 (id=emulator-5554)
Spawning `com.nbcuni.nbc`...

*** Starting scripts ***
== Redirecting all TCP connections to 127.0.0.1:8000 ==
Spawned `com.nbcuni.nbc`. Resuming main thread!
[Android Emulator 5554::com.nbcuni.nbc ]-> == Proxy system configuration overridden to 127.0.0.1:8000 ==
Process terminated

[pimterry]

OK, so where is the HTTP Tookit certificate?

The CA certificates for intercepting traffic with HTTP Toolkit (and everything else similar, unless they're making a huge security mistake) are unique to each user. They get generated on your computer on the first run, and regenerated (usually about once a year) when they expire.

The location depends on your OS and config. By default it's:

• Linux: /home/<username>/.config/httptoolkit/
• Windows: C:\Users\<username>\AppData\Local\httptoolkit\Config\
• Mac: /Users/<username>/Library/Preferences/httptoolkit/

The ca.pem file is the certificate, ca.key is the private key (you'll need both if you want to use that certificate for interception in a different tool). You can also save a copy of the cert directly from the UI: click Intercept, then Anything, then "Export CA Certificate".

that seems to be even worse result

Wow, that's fascinating!

Given the output, this must be failing within android-system-certificate-injection.js. That sort-of supports the argument that is due to a bad certificate, but I've just tested myself with your cert code above, and it doesn't obviously fail for me... Could be differences in Android version or who knows what else though.

I've just added some extra logging and error handling in here, which might provide some clues. Can you give that a quick test, and see what output that provides?

Unless that gives a clear answer, you might need to debug within that script yourself I'm afraid, which really means either running the same code yourself in the REPL or sprinkling console.logs. It must be running that code (it's listed as the next script in your command) but for whatever reason it never reaches the == System certificate trust injected == message at the end - somewhere en route it all goes wrong. If you can even just work out which line is failing that would be a very useful clue.

It's just about plausible that this is actually a Frida bug now I think about it. Which version of Frida server and CLI are you using?

[3052]

• Windows: C:\Users\<username>\AppData\Local\httptoolkit\Config\

cool thanks

I've just added some extra logging and error handling in here, which might provide some clues. Can you give that a quick test, and see what output that provides?

same error in the app, and this in the console:

     ____
    / _  |   Frida 16.1.5 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://frida.re/docs/home/
   . . . .
   . . . .   Connected to Android Emulator 5554 (id=emulator-5554)
Spawning `com.nbcuni.nbc`...

*** Starting scripts ***
== Redirecting all TCP connections to 127.0.0.1:8000 ==
Spawned `com.nbcuni.nbc`. Resuming main thread!
[Android Emulator 5554::com.nbcuni.nbc ]-> == Proxy system configuration overridden to 127.0.0.1:8000 ==
Process terminated

Which version of Frida server and CLI are you using?

16.1.5 for both. note I also broke down finally and tried HTTP Toolkit instead of MITM Proxy. with this command:

frida -U `
-l config.js `
-l android/android-certificate-unpinning.js `
-f com.nbcuni.nbc

and it works perfectly as you said. also, using HTTP Toolkit, with MITM Proxy certificate works as well:

const CERT_PEM_MITM_PROXY = `-----BEGIN CERTIFICATE-----
MIIDNTCCAh2gAwIBAgIUTFPBYqAZ0sRNwuWSQnZagnaXW14wDQYJKoZIhvcNAQEL
BQAwKDESMBAGA1UEAwwJbWl0bXByb3h5MRIwEAYDVQQKDAltaXRtcHJveHkwHhcN
MjIwNTA1MDgzNjI0WhcNMzIwNTA0MDgzNjI0WjAoMRIwEAYDVQQDDAltaXRtcHJv
eHkxEjAQBgNVBAoMCW1pdG1wcm94eTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCC
AQoCggEBALo9aNKDgg8YILUJCuk/UE5sW54tDa2iRPIoyr015UiockWLARQLIHXl
i3suOJqAf+d1xV7KyfMT4a9szlIB7EOUlxhb0dxQguKMiEBuuLB5kWu6rjKbfPEN
P7zamyNzKCKF/1iGRCaZrHvC0Kcum/2+bybagd/n9BVsuwDBQpinwvrLBX+qPF9M
YiBtZKZZXYX7ls9HpBFiTCYECrUKsis6SvonPr7sujMIlgEK8g0H8KTd/sLyHhq8
p94guzD4XlDXLWgwwLW5dazfYL2Qv4nnOWiB+heFH3PAJrJ1uFIdCQteWdTLXgr0
I8Yg51GLTFUBjCWArKddy06gITqCSvkCAwEAAaNXMFUwDwYDVR0TAQH/BAUwAwEB
/zATBgNVHSUEDDAKBggrBgEFBQcDATAOBgNVHQ8BAf8EBAMCAQYwHQYDVR0OBBYE
FNr/5OMzIXrfKNXDEdGeX8VZkljtMA0GCSqGSIb3DQEBCwUAA4IBAQBHN13akooZ
DonZhA4rWgoNk1BVzqDLN5Ye2XNMEGh/aY310FYvVXAVdPSZ9W4tGT/hKQUBJmQp
l9jz84ICjCPZHr65fs6FQE9zHNaXc7xlyxNWi9CHkvyiqJ6Sw27FllcjHGf8fWpC
9vWLMy+WE+1w0zhjiZN9ALqrcktghAxgKa6dW6Q5T2K0l7gCbIXfrI3hsLgGW9YW
pW8CwqdKdHQjYYzPm0x/B7kIBMWDTJpK9jHz14ZWImkLaHhRSnU/qXMu4RMZPtJh
93RDMIm6w26JYjefPJo64F/KbGqH1gAKCOQrZyAn7zvF/omUWjnml7Zeydw+zefE
INhklp7JmApa
-----END CERTIFICATE-----`;

const CERT_PEM = CERT_PEM_MITM_PROXY;

so I think the MITM Proxy certificate is OK. also, instead of my own script I tried this instead:

https://github.com/httptoolkit/httptoolkit-server/blob/8577f6a/src/interceptors/android/adb-commands.ts#L271-L294

my device does not have /apex, which seems to simplify the process. but even still, MITM Proxy fails. I guess at this point its a MITM Proxy issue. I did notice that I was using an older MITM Proxy version 9, but trying version 10 and even 8 didn't resolve the problem either. note I also tried using HTTP Toolkit certificate with MITM Proxy with no luck. also I posted an issue to the MITM Proxy tracker, but any further feedback would be appreciated.

[3052]

here is something. I dont know if it helps. I had idea to try older Frida, so I tried again with Frida 15.2.2. still failed, but a different error:

javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification
path not found.
        at com.android.org.conscrypt.OpenSSLSocketImpl.startHandshake(OpenSSLSocketImpl.java:328)
        at okhttp3.internal.connection.f.j(RealConnection.kt:73)
        at okhttp3.internal.connection.f.n(RealConnection.kt:53)
        at okhttp3.internal.connection.f.g(RealConnection.kt:197)
        at okhttp3.internal.connection.d.b(ExchangeFinder.kt:274)
        at okhttp3.internal.connection.d.c(ExchangeFinder.kt:1)
        at okhttp3.internal.connection.d.a(ExchangeFinder.kt:48)
        at okhttp3.internal.connection.e.u(RealCall.kt:32)
        at okhttp3.internal.connection.a.intercept(ConnectInterceptor.kt:12)
        at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167)
        at okhttp3.internal.cache.a.intercept(CacheInterceptor.kt:192)
        at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167)
        at okhttp3.internal.http.a.intercept(BridgeInterceptor.kt:171)
        at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167)
        at okhttp3.internal.http.j.intercept(RetryAndFollowUpInterceptor.kt:35)
        at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167)
        at com.nbc.utils.a.intercept(HttpLoggingInterceptor.java:15)
        at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167)
        at okhttp3.internal.connection.e.t(RealCall.kt:118)
        at okhttp3.internal.connection.e.execute(RealCall.kt:28)
        at retrofit2.o.execute(OkHttpCall.java:21)
        at com.jakewharton.retrofit2.adapter.rxjava2.b.J(CallObservable.java:17)
        at io.reactivex.o.b(Observable.java:15)
        at com.jakewharton.retrofit2.adapter.rxjava2.a.J(BodyObservable.java:8)
        at io.reactivex.o.b(Observable.java:15)
        at io.reactivex.internal.operators.observable.u.x(ObservableSingleSingle.java:10)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.f.x(SingleDoOnSubscribe.java:10)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.g.x(SingleDoOnSuccess.java:8)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.m.x(SingleMap.java:10)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.g.x(SingleDoOnSuccess.java:8)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.e.x(SingleDoOnError.java:8)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.m.x(SingleMap.java:10)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.p$a.onError(SingleResumeNext.java:22)
        at io.reactivex.internal.operators.single.e$a.onError(SingleDoOnError.java:30)
        at io.reactivex.internal.operators.single.g$a.onError(SingleDoOnSuccess.java:3)
        at io.reactivex.internal.operators.single.k.x(SingleFromCallable.java:47)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.g.x(SingleDoOnSuccess.java:8)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.e.x(SingleDoOnError.java:8)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.p.x(SingleResumeNext.java:10)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.i.x(SingleFlatMap.java:10)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.f.x(SingleDoOnSubscribe.java:10)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.i.x(SingleFlatMap.java:10)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.g.x(SingleDoOnSuccess.java:8)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.e.x(SingleDoOnError.java:8)
        at io.reactivex.s.a(Single.java:15)
        at io.reactivex.internal.operators.single.p$a.onError(SingleResumeNext.java:22)
        at io.reactivex.internal.operators.single.f$a.onError(SingleDoOnSubscribe.java:11)
        at io.reactivex.internal.operators.single.e$a.onError(SingleDoOnError.java:30)
        at io.reactivex.internal.operators.single.g$a.onError(SingleDoOnSuccess.java:3)
        at io.reactivex.internal.operators.single.k.x(S

[pimterry]

javax.net.ssl.SSLHandshakeException: java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.

That error means that for whatever reason the CA isn't trusted to sign this cert. That's interesting but not super useful, it's functionally the same issue as the checkTrustedRecursive errors, and it might just be a different symptom of the same problem, appearing in a new way due to the different Frida setups.

same error in the app, and this in the console

This is very interesting - it is definitely failing within that android/android-system-certificate-injection.js script. Can you try adding some log lines there at various points, and see if you can work out where it's crashing when running all the scripts together like this?

Given that the process terminates entirely when running with all scripts, it'd be really helpful to know which line is actually failing, to try and fix that "running all scripts" failure properly and stop anybody else running into this.

using HTTP Toolkit, with MITM Proxy certificate works as well

Interesting! Ok, so in the "unpinning scripts only" case, setup does seem to be working OK throughout, it's just that interception is failing later on somewhere in mitmproxy's interception process. Hard to know what without debugging into mitmproxy itself, but I'd certainly be interested in any answers there. Obviously mitmproxy isn't my focus, but I'd like these scripts to be usable by everybody, regardless of the proxy itself.

If you do manage to work out why the interception is being rejected in this case, let me know. It's likely there will be a way to extend the hooks to handle things like this more flexibly, but I'd need to know exactly why it's failing when HTTP Toolkit works first.

[3052]

OK:

NBC has stopped

log:

> frida -U `
>> -l config.js `
>> -l native-connect-hook.js `
>> -l android/android-proxy-override.js `
>> -l android/android-system-certificate-injection.js `
>> -l android/android-certificate-unpinning.js `
>> -l android/android-certificate-unpinning-fallback.js `
>> -f com.nbcuni.nbc
     ____
    / _  |   Frida 16.1.5 - A world-class dynamic instrumentation toolkit
   | (_| |
    > _  |   Commands:
   /_/ |_|       help      -> Displays the help system
   . . . .       object?   -> Display information about 'object'
   . . . .       exit/quit -> Exit
   . . . .
   . . . .   More info at https://frida.re/docs/home/
   . . . .
   . . . .   Connected to Android Emulator 5554 (id=emulator-5554)
Spawning `com.nbcuni.nbc`...

*** Starting scripts ***
== Redirecting all TCP connections to 127.0.0.1:8080 ==
Spawned `com.nbcuni.nbc`. Resuming main thread!
[Android Emulator 5554::com.nbcuni.nbc ]-> == Proxy system configuration overridden to 127.0.0.1:8080 ==
Rewriting <class: java.net.ProxySelector>
Rewriting <class: sun.net.spi.DefaultProxySelector>
== Proxy configuration overridden to 127.0.0.1:8080 ==
1 #########################################################
2 ############################################################
3 ############################################################
4 ############################################################
5 ############################################################
6 ############################################################
7 ############################################################
8 ############################################################
9 ############################################################
9 ############################################################
13 ############################################################
14 ############################################################
14 ############################################################
[+] Injected cert into com.android.org.conscrypt.TrustedCertificateIndex
[ ] Skipped cert injection for org.conscrypt.TrustedCertificateIndex (not present)
[ ] Skipped cert injection for org.apache.harmony.xnet.provider.jsse.TrustedCertificateIndex (not present)
== System certificate trust injected ==

    === Disabling all recognized unpinning libraries ===
[+] javax.net.ssl.HttpsURLConnection setDefaultHostnameVerifier
[+] javax.net.ssl.HttpsURLConnection setSSLSocketFactory
[+] javax.net.ssl.HttpsURLConnection setHostnameVerifier
[+] javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom)
[+] com.android.org.conscrypt.CertPinManager isChainValid
[ ] com.android.org.conscrypt.CertPinManager checkChainPinning
[+] android.security.net.config.NetworkSecurityConfig $init(*) (0)
[+] android.security.net.config.NetworkSecurityConfig $init(*) (1)
10 ############################################################
11 ############################################################
12 ############################################################
10 ############################################################
11 ############################################################
12 ############################################################
[+] com.android.okhttp.internal.tls.OkHostnameVerifier verify(String, SSLSession)
[ ] com.android.okhttp.Address $init(String, int, Dns, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
[+] com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
[ ] okhttp3.CertificatePinner *
[ ] com.squareup.okhttp.CertificatePinner *
[ ] com.datatheorem.android.trustkit.pinning.PinningTrustManager *
[ ] appcelerator.https.PinningTrustManager *
[ ] nl.xservices.plugins.sslCertificateChecker *
[ ] com.worklight.wlclient.api.WLClient *
[ ] com.worklight.wlclient.certificatepinning.HostNameVerifierWithCertificatePinning *
[ ] com.worklight.androidgap.plugin.WLCertificatePinningPlugin *
[ ] com.commonsware.cwac.netsecurity.conscrypt.CertPinManager *
[ ] io.netty.handler.ssl.util.FingerprintTrustManagerFactory *
[ ] com.silkimen.cordovahttp.CordovaServerTrust *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyHostnameVerifier *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor *
[ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyTrustManager *
== Certificate unpinning completed ==
10 ############################################################
11 ############################################################
12 ############################################################
10 ############################################################
11 ############################################################
12 ############################################################
== Unpinning fallback auto-patcher installed ==
*** Scripts completed ***

Ignoring attempt to clear http.proxyHost system property
Ignoring attempt to clear https.proxyHost system property
Ignoring attempt to clear http.proxyPort system property
Ignoring attempt to clear https.proxyPort system property
Ignoring attempt to clear http.nonProxyHosts system property
Ignoring attempt to clear https.nonProxyHosts system property
 => android.security.net.config.NetworkSecurityConfig $init(*) (1)
 => android.security.net.config.NetworkSecurityConfig $init(*) (0)
 => com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector)
Ignoring unix:stream connection

 !!! --- Intercepted tcp6 connection 40 failed when redirected to proxy 127.0.0.1:8080 --- !!!
         Is your proxy configured correctly?

Ignoring unix:stream connection
Manually intercepting connection to [0:0:0:0:0:0:0:0:0:0:ff:ff:8e:fa:73:5f]:443
Ignoring unix:stream connection

note originally the proxy error was showing port 8000, but I changed it for the log you see here to 8080 because that is what MITM Proxy uses. not sure why I got much more logs this time, maybe I wasnt using all the scripts before. I included the command I used above if its helpful. At any rate, even with the better result, the app still crashes as see at the top of my comment here. I can use a different combination of scripts, or make further edits if its helpful.

[pimterry]

Huh, that's super weird, it's now running all the steps that previously failed immediately! That doesn't tell us much about why it's not working.

The "NBC has stopped" means that the app has thrown an exception though. In that case, you should be able to get info from the ADB logs (adb logcat -T1, then watch it while this crashes) which will tell you exactly what's gone wrong and where.

Hopefully that provides a clue, because to be honest I'm stumped. If you have any idea what's wrong here, and why this works in some cases but others I'd love to hear it. I'll do some testing of my own with mitmproxy soon to explore that setup and share any clues I find myself here too.

[3052]

Huh, that's super weird, it's now running all the steps that previously failed immediately! That doesn't tell us much about why it's not working.

OK. so regarding that question, again it seems to be due to the Android version. I didn't test all, but with Android API 23, it dies immediately, then with Android API 24-27, it dies during the initial load screen. however with Android API 28-30, it still dies during the initial load screen, but it also gives more detail to the problem:

~~~ ____ / _ | Frida 16.1.7 - A world-class dynamic instrumentation toolkit | (_| | > _ | Commands: /_/ |_| help -> Displays the help system . . . . object? -> Display information about 'object' . . . . exit/quit -> Exit . . . . . . . . More info at https://frida.re/docs/home/ . . . . . . . . Connected to Android Emulator 5554 (id=emulator-5554) Spawning `com.nbcuni.nbc`... *** Starting scripts *** == Redirecting all TCP connections to 127.0.0.1:8080 == Spawned `com.nbcuni.nbc`. Resuming main thread! [Android Emulator 5554::com.nbcuni.nbc ]-> == Proxy system configuration overridden to 127.0.0.1:8080 == Rewriting Rewriting Rewriting == Proxy configuration overridden to 127.0.0.1:8080 == [+] Injected cert into com.android.org.conscrypt.TrustedCertificateIndex [ ] Skipped cert injection for org.conscrypt.TrustedCertificateIndex (not present) [ ] Skipped cert injection for org.apache.harmony.xnet.provider.jsse.TrustedCertificateIndex (not present) == System certificate trust injected == === Disabling all recognized unpinning libraries === [+] javax.net.ssl.HttpsURLConnection setDefaultHostnameVerifier [+] javax.net.ssl.HttpsURLConnection setSSLSocketFactory [+] javax.net.ssl.HttpsURLConnection setHostnameVerifier [+] javax.net.ssl.SSLContext init(KeyManager;[], TrustManager;[], SecureRandom) [ ] com.android.org.conscrypt.CertPinManager isChainValid [+] com.android.org.conscrypt.CertPinManager checkChainPinning [+] android.security.net.config.NetworkSecurityConfig $init(*) (0) [+] android.security.net.config.NetworkSecurityConfig $init(*) (1) [+] com.android.okhttp.internal.tls.OkHostnameVerifier verify(String, SSLSession) [+] com.android.okhttp.Address $init(String, int, Dns, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector) [ ] com.android.okhttp.Address $init(String, int, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector) [ ] okhttp3.CertificatePinner * [ ] com.squareup.okhttp.CertificatePinner * [ ] com.datatheorem.android.trustkit.pinning.PinningTrustManager * [ ] appcelerator.https.PinningTrustManager * [ ] nl.xservices.plugins.sslCertificateChecker * [ ] com.worklight.wlclient.api.WLClient * [ ] com.worklight.wlclient.certificatepinning.HostNameVerifierWithCertificatePinning * [ ] com.worklight.androidgap.plugin.WLCertificatePinningPlugin * [ ] com.commonsware.cwac.netsecurity.conscrypt.CertPinManager * [ ] io.netty.handler.ssl.util.FingerprintTrustManagerFactory * [ ] com.silkimen.cordovahttp.CordovaServerTrust * [ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyHostnameVerifier * [ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyInterceptor * [ ] com.appmattus.certificatetransparency.internal.verifier.CertificateTransparencyTrustManager * == Certificate unpinning completed == == Unpinning fallback auto-patcher installed == *** Scripts completed *** Ignoring attempt to clear http.proxyHost system property Ignoring attempt to clear https.proxyHost system property Ignoring attempt to clear http.proxyPort system property Ignoring attempt to clear https.proxyPort system property Ignoring attempt to clear http.nonProxyHosts system property Ignoring attempt to clear https.nonProxyHosts system property => android.security.net.config.NetworkSecurityConfig $init(*) (1) => android.security.net.config.NetworkSecurityConfig $init(*) (0) => android.security.net.config.NetworkSecurityConfig $init(*) (1) => android.security.net.config.NetworkSecurityConfig $init(*) (0) => com.android.okhttp.Address $init(String, int, Dns, SocketFactory, SSLSocketFactory, HostnameVerifier, CertificatePinner, Authenticator, Proxy, List, List, ProxySelector) THESE FOUR LINES REPEATED A BUNCH: Ignoring unix:stream connection Ignoring unix:stream connection Connected tcp6 fd 56 to null (-1) Manually intercepting connection to [0:0:0:0:0:0:0:0:0:0:ff:ff:8e:fa:72:5f]:443 Process crashed: java.net.ConnectException: Failed to connect to /127.0.0.1:8080 *** FATAL EXCEPTION: main Process: com.nbcuni.nbc, PID: 3352 java.net.ConnectException: Failed to connect to /127.0.0.1:8080 at okhttp3.internal.connection.f.i(RealConnection.kt:147) at okhttp3.internal.connection.f.k(RealConnection.kt:14) at okhttp3.internal.connection.f.g(RealConnection.kt:175) at okhttp3.internal.connection.d.b(ExchangeFinder.kt:274) at okhttp3.internal.connection.d.c(ExchangeFinder.kt:1) at okhttp3.internal.connection.d.a(ExchangeFinder.kt:48) at okhttp3.internal.connection.e.u(RealCall.kt:32) at okhttp3.internal.connection.a.intercept(ConnectInterceptor.kt:12) at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167) at okhttp3.internal.cache.a.intercept(CacheInterceptor.kt:192) at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167) at okhttp3.internal.http.a.intercept(BridgeInterceptor.kt:171) at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167) at okhttp3.internal.http.j.intercept(RetryAndFollowUpInterceptor.kt:35) at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167) at com.nbc.utils.a.intercept(HttpLoggingInterceptor.java:15) at okhttp3.internal.http.g.c(RealInterceptorChain.kt:167) at okhttp3.internal.connection.e.t(RealCall.kt:118) at okhttp3.internal.connection.e.execute(RealCall.kt:28) at retrofit2.o.execute(OkHttpCall.java:21) at com.jakewharton.retrofit2.adapter.rxjava2.b.J(CallObservable.java:17) at io.reactivex.o.b(Observable.java:15) at com.jakewharton.retrofit2.adapter.rxjava2.a.J(BodyObservable.java:8) at io.reactivex.o.b(Observable.java:15) at io.reactivex.internal.operators.observable.u.x(ObservableSingleSingle.java:10) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.f.x(SingleDoOnSubscribe.java:10) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.g.x(SingleDoOnSuccess.java:8) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.m.x(SingleMap.java:10) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.g.x(SingleDoOnSuccess.java:8) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.e.x(SingleDoOnError.java:8) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.m.x(SingleMap.java:10) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.p$a.onError(SingleResumeNext.java:22) at io.reactivex.internal.operators.single.e$a.onError(SingleDoOnError.java:30) at io.reactivex.internal.operators.single.g$a.onError(SingleDoOnSuccess.java:3) at io.reactivex.internal.operators.single.k.x(SingleFromCallable.java:47) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.g.x(SingleDoOnSuccess.java:8) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.e.x(SingleDoOnError.java:8) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.p.x(SingleResumeNext.java:10) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.i.x(SingleFlatMap.java:10) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.f.x(SingleDoOnSubscribe.java:10) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.i.x(SingleFlatMap.java:10) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.g.x(SingleDoOnSuccess.java:8) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.e.x(SingleDoOnError.java:8) at io.reactivex.s.a(Single.java:15) at io.reactivex.internal.operators.single.p$a.onError(SingleResumeNext.java:22) at io.reactivex.internal.operators.single.f$a.onError(SingleDoOnSubscribe.java:11) at io.reactivex.internal.operators.single.e$a.onError(SingleDoOnError.java:30) at io.reactivex.internal.operators.single.g$a.onError(SingleDoOnSuccess.java:3) at io.reactivex.internal.operators.single.k.x(SingleFromCallable.java:47) at io.reactivex.s.a(Single.java:15) ~~~

since this is now the second time I am running into issues with the Frida script here due to Android version, I think the script itself should notify the user LOUDLY if they are using an untested version. so for example if you are only testing against Android 31 or something, any older Androids should either fail immediately with an error, or warn the user they are using an untested Android in regards to the Frida script here.

[pimterry]

since this is now the second time I am running into issues with the Frida script here due to Android version, I think the script itself should notify the user LOUDLY if they are using an untested version

v23 shouldn't be an unsupported version. I'm open to adding warnings for versions that don't work, but I'm not sure what the lower bound is right now. As far as I'm aware, almost every optional API is used very cautiously, and so the scripts should still work - only applying to existing APIs - back a very long way through old Android releases.

You're welcome to test old versions if you're interested, and if you can confirm where any unsupported Android APIs are used without fallbacks, we can either fix those or add warnings for those cases.

Testing this reliably is harder than it sounds though, as even the same OS version on different devices/emulators can behave differently, there's also a wide variety of Frida versions with different OS version support, and individual apps' own implementations play a part in testing.

I've done some more digging on your issue, and I think this may be due to the proxy script (./android/android-proxy-override.js). Can you try intercepting with every script except that one? The native hook script should mean that connections are still redirected (although less delicately).

On my machine, using Frida 16.1.4 on Android 23, intercepting processes crashes as you describe in that script where it runs Java.enumerateMethods('*!select(java.net.URI): java.util.List/s'). This isn't due to the script not supporting that Android version - this appears to be a Frida bug (this is a fairly simple Frida command - just scanning for methods matching that signature). If you run Frida on Android 23 against a target app without any scripts at all, and then call that Frida API in the REPL, the app process seems to crash every time. We can add workarounds for this on Android <23 later on, but for now you can just skip that script to continue your testing.

Remaining issues on later Android versions are less clear I'm afraid. It would be helpful if you could test different combinations of scripts, play with extra logging, and see if you can find the relevant line that's crashing the app in those cases too.

[3052]

v23 shouldn't be an unsupported version.

right but if you remember here:

https://github.com/httptoolkit/frida-interception-and-unpinning/issues/52#issuecomment-1797068437

the problem was that the script was assuming users were on Oreo (API 26) or higher. so while v23 might be "supported", if its not being actively tested against, then I think its being a good neighbor to at least loudly warn users of that fact, either in the README or the script itself. something like:

NOTE THIS IS ONLY TESTED ON ANDROID API 31. IT SHOULD WORK ON OLDER VERSIONS, PROCEED AT YOUR OWN RISK.

that way people can use the known good version if they want, or they can use older versions, but with the knowledge that its a greater chance of not working.

I'm open to adding warnings for versions that don't work, but I'm not sure what the lower bound is right now.

I would say depending on what you are actively testing, probably the current minimum should be reported as Android 9 (API 28).

You're welcome to test old versions if you're interested, and if you can confirm where any unsupported Android APIs are used without fallbacks, we can either fix those or add warnings for those cases.

In general I would like to avoid being the guinea pig here. I normally use API 23 just because with less secure apps you dont even have to worry about installing a system certificate. but if I know some item (like this script) requires a higher version, then I will use that.

I've done some more digging on your issue, and I think this may be due to the proxy script (./android/android-proxy-override.js). Can you try intercepting with every script except that one? The native hook script should mean that connections are still redirected (although less delicately).

looks like same result as my previous comment. here is full log if it helps:

https://github.com/httptoolkit/frida-interception-and-unpinning/files/13420295/2023-11-20.txt

We can add workarounds for this on Android <23 later on, but for now you can just skip that script to continue your testing.

again I am not locked into any specific version. my preference is API 23, but I am comfortable using even up to API 30. so if you say "API 29 works great", then I will use that. I just want something that works with MITM Proxy if possible. I dont wish to undermine HTTP Toolkit, but as you said I think its good if we have a robust script that works with different tools.