httptoolkit / frida-interception-and-unpinning

Frida scripts to directly MitM all HTTPS traffic from a target mobile application
https://httptoolkit.com/android/
GNU Affero General Public License v3.0
909 stars 179 forks source link

Cert rejected in-app with Roblox #50

Open yaeleiger opened 8 months ago

yaeleiger commented 8 months ago

After using this great tutorial, I can confirm twitter's certificate un-pinning and can see traffic from twitter. Unfortunately, when I try to do the same with Roblox, I still can't get passed their cert pining. In my mitmdump I see: "Client TLS handshake failed. The client does not trust the proxy's certificate for *.roblox.com"

And in app, I see the roblox alert "Connection error. Unable to contact server."

I am using mitmproxy and http toolkit 'connect through adb'. The phone is rooted and I have su access. HTTPtoolkit gives all green checkmarks.

I love all these tools and use the cert pinning for other apps, but Roblox is awfully stubborn! fwiw, when I do a tcpdump of playing Roblox on macos (instead of on android) I see the RakNet protocol for all the Roblox traffic (encrypted).

Thanks for all wisdom! And thanks for all the great docs and open source code in the past!

pimterry commented 8 months ago

I've got a guide for cases that don't work out of the box that you might find helpful here: https://httptoolkit.com/blog/android-reverse-engineering/.

In general, the hard part is working out where the actual pinning is happening. It's a game of digging through logs, searching the source, and using Frida to monitor calls to every kind of exception throwing or relevant APIs that you can think of.

Something within Roblox must be trying to make a request that's then rejecting the connection. If you can find the code that's actually doing the check and failing, share it here and I'm happy to help write up a patch to work around it (and potentially integrate that here, if possible).

Is there anything shown in the ADB logs, or any errors listed in the Frida script output that might provide some clues? A stacktrace would be perfect, but any kinds of hints as to where exactly in the code this is being rejected would be helpful.

yaeleiger commented 8 months ago

Thanks for the repsponse! Yeah! When I logcat it looks to be this TlsVerificationFail: 10-24 16:59:08.184 19039 19150 I Roblox : 2023-10-24T23:59:07.886Z,4.886150,da2aacc0,12 [DFLog::HttpTraceError] HttpResponse(#41 0xb400007c86f82148) time:84.1ms (net:80.2ms callback:0.0ms timeInRetryQueue:0.0ms) error:11 message:HttpError: TlsVerificationFail url:{ "https://apis.roblox.com/upsellcard/type" } ip:127.0.0.1 external:0 numberOfTimesRetried:0 proxy: "127.0.0.1"

which I can't find exactly (maybe I am doing something wrong?) when I search through the JADX decompiled apk but I do see things like:

import java.security.cert.Certificate;
import java.security.cert.CertificateEncodingException;
import java.security.cert.CertificateException;
import java.security.cert.CertificateFactory;

import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;

   public List<Certificate> a(List<Certificate> param1List, String param1String) throws SSLPeerUnverifiedException {
     try {
       X509Certificate[] arrayOfX509Certificate = param1List.<X509Certificate>toArray(new X509Certificate[param1List.size()]);
       return (List)this.b.invoke(this.a, new Object[] { arrayOfX509Certificate, "RSA", param1String });
     } catch (InvocationTargetException invocationTargetException) {
       SSLPeerUnverifiedException sSLPeerUnverifiedException = new SSLPeerUnverifiedException(invocationTargetException.getMessage());
       sSLPeerUnverifiedException.initCause(invocationTargetException);
       throw sSLPeerUnverifiedException;
     } catch (IllegalAccessException illegalAccessException) {
       throw new AssertionError(illegalAccessException);
     }
   }
yaeleiger commented 8 months ago

Here's that full file if its helpful, seems like there's lots of booleans I can set to true like in your tutorial for reverse engineering ?

package ob;

import android.os.Build;
import android.util.Log;
import gb.e0;
import hb.e;
import java.io.IOException;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.nio.charset.StandardCharsets;
import java.security.NoSuchAlgorithmException;
import java.security.cert.Certificate;
import java.security.cert.TrustAnchor;
import java.security.cert.X509Certificate;
import java.util.List;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLPeerUnverifiedException;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.X509TrustManager;
import qb.e;

class b extends f {
  private final Class<?> c;

  private final Class<?> d;

  private final Method e;

  private final Method f;

  private final Method g;

  private final Method h;

  private final b i = b.b();

  b(Class<?> paramClass1, Class<?> paramClass2, Method paramMethod1, Method paramMethod2, Method paramMethod3, Method paramMethod4) {
    this.c = paramClass1;
    this.d = paramClass2;
    this.e = paramMethod1;
    this.f = paramMethod2;
    this.g = paramMethod3;
    this.h = paramMethod4;
  }

  private boolean v(String paramString, Class<?> paramClass, Object paramObject) throws InvocationTargetException, IllegalAccessException {
    try {
      return ((Boolean)paramClass.getMethod("isCleartextTrafficPermitted", new Class[0]).invoke(paramObject, new Object[0])).booleanValue();
    } catch (NoSuchMethodException noSuchMethodException) {
      return super.r(paramString);
    } 
  }

  private boolean w(String paramString, Class<?> paramClass, Object paramObject) throws InvocationTargetException, IllegalAccessException {
    try {
      return ((Boolean)paramClass.getMethod("isCleartextTrafficPermitted", new Class[] { String.class }).invoke(paramObject, new Object[] { paramString })).booleanValue();
    } catch (NoSuchMethodException noSuchMethodException) {
      return v(paramString, paramClass, paramObject);
    } 
  }

  public static f x() {
    if (!f.q())
      return null; 
    try {
      Class<?> clazz1 = Class.forName("com.android.org.conscrypt.SSLParametersImpl");
      Class<?> clazz2 = Class.forName("com.android.org.conscrypt.OpenSSLSocketImpl");
      try {
        return new b(clazz1, clazz2, clazz2.getDeclaredMethod("setUseSessionTickets", new Class[] { boolean.class }), clazz2.getMethod("setHostname", new Class[] { String.class }), clazz2.getMethod("getAlpnSelectedProtocol", new Class[0]), clazz2.getMethod("setAlpnProtocols", new Class[] { byte[].class }));
      } catch (NoSuchMethodException noSuchMethodException) {}
      StringBuilder stringBuilder = new StringBuilder();
      stringBuilder.append("Expected Android API level 21+ but was ");
      stringBuilder.append(Build.VERSION.SDK_INT);
      throw new IllegalStateException(stringBuilder.toString());
    } catch (ClassNotFoundException classNotFoundException) {
      return null;
    } 
  }

  static int y() {
    try {
      return Build.VERSION.SDK_INT;
    } catch (NoClassDefFoundError noClassDefFoundError) {
      return 0;
    } 
  }

  public qb.c c(X509TrustManager paramX509TrustManager) {
    try {
      Class<?> clazz = Class.forName("android.net.http.X509TrustManagerExtensions");
      return new a(clazz.getConstructor(new Class[] { X509TrustManager.class }, ).newInstance(new Object[] { paramX509TrustManager }, ), clazz.getMethod("checkServerTrusted", new Class[] { X509Certificate[].class, String.class, String.class }));
    } catch (Exception exception) {
      return super.c(paramX509TrustManager);
    } 
  }

  public e d(X509TrustManager paramX509TrustManager) {
    try {
      Method method = paramX509TrustManager.getClass().getDeclaredMethod("findTrustAnchorByIssuerAndSignature", new Class[] { X509Certificate.class });
      method.setAccessible(true);
      return new c(paramX509TrustManager, method);
    } catch (NoSuchMethodException noSuchMethodException) {
      return super.d(paramX509TrustManager);
    } 
  }

  public void g(SSLSocket paramSSLSocket, String paramString, List<e0> paramList) throws IOException {
    if (!this.d.isInstance(paramSSLSocket))
      return; 
    if (paramString != null) {
      try {
        this.e.invoke(paramSSLSocket, new Object[] { Boolean.TRUE });
        this.f.invoke(paramSSLSocket, new Object[] { paramString });
        this.h.invoke(paramSSLSocket, new Object[] { f.e(paramList) });
        return;
      } catch (IllegalAccessException illegalAccessException) {

      } catch (InvocationTargetException invocationTargetException) {}
      throw new AssertionError(invocationTargetException);
    } 
    this.h.invoke(invocationTargetException, new Object[] { f.e(paramList) });
  }

  public void h(Socket paramSocket, InetSocketAddress paramInetSocketAddress, int paramInt) throws IOException {
    try {
      paramSocket.connect(paramInetSocketAddress, paramInt);
      return;
    } catch (AssertionError assertionError) {
      if (e.A(assertionError))
        throw new IOException(assertionError); 
      throw assertionError;
    } catch (ClassCastException classCastException) {
      if (Build.VERSION.SDK_INT == 26)
        throw new IOException("Exception in connect", classCastException); 
      throw classCastException;
    } 
  }

  public SSLContext n() {
    boolean bool = true;
    try {
      int i = Build.VERSION.SDK_INT;
      if (i >= 22)
        bool = false; 
    } catch (NoClassDefFoundError noClassDefFoundError) {}
    if (bool)
      try {
        return SSLContext.getInstance("TLSv1.2");
      } catch (NoSuchAlgorithmException noSuchAlgorithmException) {} 
    try {
      return SSLContext.getInstance("TLS");
    } catch (NoSuchAlgorithmException noSuchAlgorithmException) {
      throw new IllegalStateException("No TLS provider", noSuchAlgorithmException);
    } 
  }

  public String o(SSLSocket paramSSLSocket) {
    boolean bool = this.d.isInstance(paramSSLSocket);
    SSLSocket sSLSocket = null;
    if (!bool)
      return null; 
    try {
      String str;
      byte[] arrayOfByte = (byte[])this.g.invoke(paramSSLSocket, new Object[0]);
      paramSSLSocket = sSLSocket;
      if (arrayOfByte != null)
        str = new String(arrayOfByte, StandardCharsets.UTF_8); 
      return str;
    } catch (IllegalAccessException illegalAccessException) {

    } catch (InvocationTargetException invocationTargetException) {}
    throw new AssertionError(invocationTargetException);
  }

  public Object p(String paramString) {
    return this.i.a(paramString);
  }

  public boolean r(String paramString) {
    try {
      Class<?> clazz = Class.forName("android.security.NetworkSecurityPolicy");
      return w(paramString, clazz, clazz.getMethod("getInstance", new Class[0]).invoke(null, new Object[0]));
    } catch (ClassNotFoundException|NoSuchMethodException classNotFoundException) {
      return super.r(paramString);
    } catch (IllegalAccessException illegalAccessException) {
      throw new AssertionError("unable to determine cleartext support", illegalAccessException);
    } catch (IllegalArgumentException illegalArgumentException) {
      throw new AssertionError("unable to determine cleartext support", illegalArgumentException);
    } catch (InvocationTargetException invocationTargetException) {
      throw new AssertionError("unable to determine cleartext support", invocationTargetException);
    } 
  }

  public void t(int paramInt, String paramString, Throwable paramThrowable) {
    byte b1 = 5;
    if (paramInt != 5)
      b1 = 3; 
    String str = paramString;
    if (paramThrowable != null) {
      StringBuilder stringBuilder = new StringBuilder();
      stringBuilder.append(paramString);
      stringBuilder.append('\n');
      stringBuilder.append(Log.getStackTraceString(paramThrowable));
      str = stringBuilder.toString();
    } 
    paramInt = 0;
    int i = str.length();
    label23: while (paramInt < i) {
      int j = str.indexOf('\n', paramInt);
      if (j == -1)
        j = i; 
      while (true) {
        int k = Math.min(j, paramInt + 4000);
        Log.println(b1, "OkHttp", str.substring(paramInt, k));
        if (k >= j) {
          paramInt = k + 1;
          continue label23;
        } 
        paramInt = k;
      } 
    } 
  }

  public void u(String paramString, Object paramObject) {
    if (!this.i.c(paramObject))
      t(5, paramString, null); 
  }

  static final class a extends qb.c {
    private final Object a;

    private final Method b;

    a(Object param1Object, Method param1Method) {
      this.a = param1Object;
      this.b = param1Method;
    }

    public List<Certificate> a(List<Certificate> param1List, String param1String) throws SSLPeerUnverifiedException {
      try {
        X509Certificate[] arrayOfX509Certificate = param1List.<X509Certificate>toArray(new X509Certificate[param1List.size()]);
        return (List)this.b.invoke(this.a, new Object[] { arrayOfX509Certificate, "RSA", param1String });
      } catch (InvocationTargetException invocationTargetException) {
        SSLPeerUnverifiedException sSLPeerUnverifiedException = new SSLPeerUnverifiedException(invocationTargetException.getMessage());
        sSLPeerUnverifiedException.initCause(invocationTargetException);
        throw sSLPeerUnverifiedException;
      } catch (IllegalAccessException illegalAccessException) {
        throw new AssertionError(illegalAccessException);
      } 
    }

    public boolean equals(Object param1Object) {
      return param1Object instanceof a;
    }

    public int hashCode() {
      return 0;
    }
  }

  static final class b {
    private final Method a;

    private final Method b;

    private final Method c;

    b(Method param1Method1, Method param1Method2, Method param1Method3) {
      this.a = param1Method1;
      this.b = param1Method2;
      this.c = param1Method3;
    }

    static b b() {
      Exception exception2;
      Method method = null;
      try {
        Class<?> clazz = Class.forName("dalvik.system.CloseGuard");
        Method method3 = clazz.getMethod("get", new Class[0]);
        Method method2 = clazz.getMethod("open", new Class[] { String.class });
        Method method1 = clazz.getMethod("warnIfOpen", new Class[0]);
        method = method3;
      } catch (Exception exception1) {
        exception1 = null;
        exception2 = exception1;
      } 
      return new b(method, (Method)exception2, (Method)exception1);
    }

    Object a(String param1String) {
      Method method = this.a;
      if (method != null)
        try {
          Object object = method.invoke(null, new Object[0]);
          this.b.invoke(object, new Object[] { param1String });
          return object;
        } catch (Exception exception) {
          return null;
        }  
      return null;
    }

    boolean c(Object param1Object) {
      boolean bool = false;
      if (param1Object != null)
        try {
          this.c.invoke(param1Object, new Object[0]);
          return true;
        } catch (Exception exception) {
          return false;
        }  
      return bool;
    }
  }

  static final class c implements e {
    private final X509TrustManager a;

    private final Method b;

    c(X509TrustManager param1X509TrustManager, Method param1Method) {
      this.b = param1Method;
      this.a = param1X509TrustManager;
    }

    public X509Certificate a(X509Certificate param1X509Certificate) {
      X509Certificate x509Certificate = null;
      try {
        TrustAnchor trustAnchor = (TrustAnchor)this.b.invoke(this.a, new Object[] { param1X509Certificate });
        param1X509Certificate = x509Certificate;
        if (trustAnchor != null)
          param1X509Certificate = trustAnchor.getTrustedCert(); 
        return param1X509Certificate;
      } catch (IllegalAccessException illegalAccessException) {
        throw new AssertionError("unable to get issues and signature", illegalAccessException);
      } catch (InvocationTargetException invocationTargetException) {
        return null;
      } 
    }

    public boolean equals(Object param1Object) {
      if (param1Object == this)
        return true; 
      if (!(param1Object instanceof c))
        return false; 
      param1Object = param1Object;
      return (this.a.equals(((c)param1Object).a) && this.b.equals(((c)param1Object).b));
    }

    public int hashCode() {
      return this.a.hashCode() + this.b.hashCode() * 31;
    }
  }
}
yaeleiger commented 8 months ago

Oh! and when I spawn Frida with the current codebase, I see: Screenshot 2023-10-26 at 12 24 29 PM

pimterry commented 8 months ago

I don't think that code posted above is relevant - that does throw SSLPeerUnverifiedException but only after a InvocationTargetException which I suspect is related to some lower level or parsing error on the certificate (which shouldn't happen in these cases) rather than certificate pinning.

The logs are very interesting though - that TlsVerificationFail definitely sounds like the issue, and like a unique error we should be able to follow.

Unfortunately though, searching the APK, that doesn't appear literally in the code - only inside the binary native file in resources/lib/$ARCHITECTURE/libroblox.so.

I suspect that means Roblox is shipping its own native TLS library, bundled inside that SO file. This could be for cross-platform compatibility & development benefits, since it means they can build once, compile natively, and then run the same code everywhere. It's inconvenient for us though, since this means that modifying this logic will require reverse engineering that, and that is definitely going to be challenging (and beyond what I'm familiar with I'm afraid - I'm more comfortable on the Java side here).

One more promising clue though is resources/assets/cacert.pem. That file in the APK contains a long list of certificates. I strongly suspect that's a limited set of CAs that the app trusts! If you modify that, then rebuild the APK and install it, that might well work for you.

pimterry commented 8 months ago

There's also resources/assets/fingerprint.txt, which could very likely be a certificate fingerprint used for pinning.

yaeleiger commented 8 months ago

Awesome, will definitely try to alter the pem file and fingerprint.txt. Thanks so much for the quick response!!

antiworkink commented 6 months ago

hey did you find a fix?