SSL Handshake failure in Android 10

[nildeka]

Android 10 throws exception in SSL Handshaking both in emulators and Pixel devices. The same code works for Android 9 and before. Steps to reproduce:

1. Create a Private Public Key Pair. KeyPairGenerator kpg = KeyPairGenerator.getInstance( KeyProperties.KEY_ALGORITHM_RSA);

       kpg.initialize(new KeyGenParameterSpec.Builder(
               CLIENT_KEY_ALIAS,
               KeyProperties.PURPOSE_SIGN | KeyProperties.PURPOSE_VERIFY)
               .setDigests(KeyProperties.DIGEST_NONE)
               .setSignaturePaddings(KeyProperties.SIGNATURE_PADDING_RSA_PKCS1)
               .build());
   
       KeyPair keyPair = kpg.generateKeyPair();
       return keyPair;

2. Send public key to server and receives client certiricates, and store it in Android Keystore. As well as store Root Server certificate.

       CertificateFactory certificateFactory = CertificateFactory.getInstance("X.509");
   
       X509Certificate[] clientCertArray = new X509Certificate[clientCertArrayFinal.length];
       i = 0;
   
       for (String indCert : clientCertArrayFinal) {
           X509Certificate clientCertFromApi = (X509Certificate) certificateFactory.generateCertificate(new
           ByteArrayInputStream(indCert.getBytes()));
           clientCertArray[i++] = clientCertFromApi;
       }
   
       PrivateKey privateKeyClient = iPrivateKeyClient;
   
       InputStream inServer = iContext.getResources().openRawResource(R.raw.mrca);
       X509Certificate serverCert = (X509Certificate) certificateFactory.generateCertificate(inServer);
   
       KeyStore androidKeyStore = KeyStore.getInstance(CertificateManager.ANDROID_KEYSTORE);
       androidKeyStore.load(null);
   
       androidKeyStore.setCertificateEntry(CertificateManager.SERVER_CERT_ALIAS, serverCert);  //Server cert
       androidKeyStore.setKeyEntry(CertificateManager.CLIENT_KEY_ALIAS, privateKeyClient, null, clientCertArray); //Client cert, can be opened using private key

3. Using Trustmananger to create SSLContext

       KeyStore androidKeyStore = KeyStore.getInstance(CertificateManager.ANDROID_KEYSTORE);
       androidKeyStore.load(null);
   
       TrustManagerFactory trustmanagerfactory =
      TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
       trustmanagerfactory.init(androidKeyStore);
       TrustManager[] trustManagers = trustmanagerfactory.getTrustManagers();
       KeyManagerFactory kmf = KeyManagerFactory
               .getInstance(KeyManagerFactory.getDefaultAlgorithm());
       kmf.init(androidKeyStore, "".toCharArray());
   
       final SSLContext sslContext = SSLContext.getInstance("TLS");
       sslContext.init(kmf.getKeyManagers(), trustManagers, null);

4. Start handshake with server: String server = "185.58.87.115"; SSLSocketFactory ssf = sslContext.getSocketFactory();

                   InetSocketAddress inetSocketAddress = new InetSocketAddress(server, 443);
   
                   SSLSocket socket = (SSLSocket) ssf.createSocket();
                   socket.connect(inetSocketAddress);
                   socket.setKeepAlive(true);
   
                   socket.addHandshakeCompletedListener(new HandshakeCompletedListener() {
                       @Override
                       public void handshakeCompleted(HandshakeCompletedEvent event) {
                           try {
                               String principal = event.getPeerPrincipal().getName();
                               Log.i(TAG, principal);
                               Log.i(TAG, event.getCipherSuite().toString());
   
                           } catch (SSLPeerUnverifiedException e) {
                               e.printStackTrace();
                           }
   
                       }
                   });
                   Thread.sleep(1000);
                   socket.startHandshake();      

The handshake failed with below exception:

`W/CryptoUpcalls: Preferred provider doesn't support key: java.security.InvalidKeyException: Keystore operation failed at android.security.KeyStore.getInvalidKeyException(KeyStore.java:1362) at android.security.KeyStore.getInvalidKeyException(KeyStore.java:1402) at android.security.keystore.KeyStoreCryptoOperationUtils.getInvalidKeyExceptionForInit(KeyStoreCryptoOperationUtils.java:54) at android.security.keystore.KeyStoreCryptoOperationUtils.getExceptionForCipherInit(KeyStoreCryptoOperationUtils.java:89) at android.security.keystore.AndroidKeyStoreCipherSpiBase.ensureKeystoreOperationInitialized(AndroidKeyStoreCipherSpiBase.java:265) at android.security.keystore.AndroidKeyStoreCipherSpiBase.engineInit(AndroidKeyStoreCipherSpiBase.java:109) at javax.crypto.Cipher.tryTransformWithProvider(Cipher.java:2984) at javax.crypto.Cipher.tryCombinations(Cipher.java:2891) at javax.crypto.Cipher$SpiAndProviderUpdater.updateAndGetSpiAndProvider(Cipher.java:2796) at javax.crypto.Cipher.chooseProvider(Cipher.java:773) at javax.crypto.Cipher.init(Cipher.java:1143) at javax.crypto.Cipher.init(Cipher.java:1084) at com.android.org.conscrypt.CryptoUpcalls.rsaOpWithPrivateKey(CryptoUpcalls.java:173) at com.android.org.conscrypt.CryptoUpcalls.rsaSignDigestWithPrivateKey(CryptoUpcalls.java:132) at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method) at com.android.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:387) at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:226) at asynctls.mimecast.com.asynctlsapp.MainActivity$1.run(MainActivity.java:352) at java.lang.Thread.run(Thread.java:919) Caused by: android.security.KeyStoreException: Incompatible padding mode at android.security.KeyStore.getKeyStoreException(KeyStore.java:1292) at android.security.KeyStore.getInvalidKeyException(KeyStore.java:1402) at android.security.keystore.KeyStoreCryptoOperationUtils.getInvalidKeyExceptionForInit(KeyStoreCryptoOperationUtils.java:54) at android.security.keystore.KeyStoreCryptoOperationUtils.getExceptionForCipherInit(KeyStoreCryptoOperationUtils.java:89) at android.security.keystore.AndroidKeyStoreCipherSpiBase.ensureKeystoreOperationInitialized(AndroidKeyStoreCipherSpiBase.java:265) at android.security.keystore.AndroidKeyStoreCipherSpiBase.engineInit(AndroidKeyStoreCipherSpiBase.java:109) at javax.crypto.Cipher.tryTransformWithProvider(Cipher.java:2984) at javax.crypto.Cipher.tryCombinations(Cipher.java:2891) at javax.crypto.Cipher$SpiAndProviderUpdater.updateAndGetSpiAndProvider(Cipher.java:2796) at javax.crypto.Cipher.chooseProvider(Cipher.java:773) at javax.crypto.Cipher.init(Cipher.java:1143) at javax.crypto.Cipher.init(Cipher.java:1084) at com.android.org.conscrypt.CryptoUpcalls.rsaOpWithPrivateKey(CryptoUpcalls.java:173) at com.android.org.conscrypt.CryptoUpcalls.rsaSignDigestWithPrivateKey(CryptoUpcalls.java:132) at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method) at com.android.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:387) at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:226) at asynctls.mimecast.com.asynctlsapp.MainActivity$1.run(MainActivity.java:352) at java.lang.Thread.run(Thread.java:919)

W/CryptoUpcalls: Could not find provider for algorithm: RSA/ECB/NoPadding W/System.err: javax.net.ssl.SSLHandshakeException: Handshake failed W/System.err: at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:288) W/System.err: at asynctls.mimecast.com.asynctlsapp.MainActivity$1.run(MainActivity.java:351) W/System.err: at java.lang.Thread.run(Thread.java:919) W/System.err: Caused by: javax.net.ssl.SSLProtocolException: SSL handshake aborted: ssl=0x7dbcec0850c8: Failure in SSL library, usually a protocol error W/System.err: error:04000044:RSA routines:OPENSSL_internal:internal error (external/conscrypt/common/src/jni/main/cpp/conscrypt/native_crypto.cc:740 0x7dbce6155e73:0x00000000) W/System.err: at com.android.org.conscrypt.NativeCrypto.SSL_do_handshake(Native Method) W/System.err: at com.android.org.conscrypt.NativeSsl.doHandshake(NativeSsl.java:387) W/System.err: at com.android.org.conscrypt.ConscryptFileDescriptorSocket.startHandshake(ConscryptFileDescriptorSocket.java:226) W/System.err: ... 2 more`

The app works fine with Pre Android 10 phone. Could you please look, this will block our app's functionality completely.

Thanks,

[flooey]

Just to clarify what's going on, it looks like you're doing client certificate authentication, and it's failing while trying to sign with the client cert's private key? Can you supply a complete working example of the failure? What you've posted involves talking to a server we don't have access to that produces the client cert. Just an example client cert and private key being installed in Android Keystore and then used should work.

[nildeka]

Yes we initially install the client cert and private key in the Android KeyStore and trust it using the TrustManager, and we create a SSLContext. Using the SSLContext we try to initiate a handshake. So it fails during the handshaking. (even while trying asynchronously using java.nio it failed during unwrap event). It works fine in all OS before 10. (Currently we are checking with our organization's security team if I can provide you the working example). The server here is specific to our organization.

[nildeka]

We are able to solve the problem. Adding setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_NONE) during RSA Key-pair generation solved the problem.

kpg.initialize(new KeyGenParameterSpec.Builder( CLIENT_KEY_ALIAS, KeyProperties.PURPOSE_SIGN | KeyProperties.PURPOSE_VERIFY) .setDigests(KeyProperties.DIGEST_NONE, KeyProperties.DIGEST_SHA256, KeyProperties.DIGEST_SHA512) .setEncryptionPaddings(KeyProperties.ENCRYPTION_PADDING_NONE) .setSignaturePaddings(KeyProperties.SIGNATURE_PADDING_RSA_PKCS1) .build());

Looks like conscrypt has changed internal implementation of Signing handshake data. It worked in Pre-Android 10, because it only does Signing with private key. But from Android 10, it encrypts data with private key. In our case it encrypts with RSA/ECB/NoPadding. After adding encryption padding during key pair genration solved the problem.

Thanks

[flooey]

Ah, yes, that's true, we did do that (in order to support PSS signatures). Apologies for not realizing that would require this kind of change.

[alfdev]

Hi,

i've the same issue while trying to sign in a Client Authentication flow. Everything works pre android 10, now i receive this error:

CryptoUpcalls: Could not find provider for algorithm: RSA/ECB/PKCS1Padding cr_AndroidKeyStore: Exception while signing message with SHA256withRSA and RSA private key (com.mypackageCustomPrivateKey): java.security.SignatureException: java.lang.RuntimeException: error:04000044:RSA routines:OPENSSL_internal:internal error

[farble1670]

Ah, yes, that's true, we did do that (in order to support PSS signatures). Apologies for not realizing that would require this kind of change.

@alfdev Would it be possible to get more information about this? A link to the pull request perhaps?

[alfdev]

Hi @farble1670,

i was able to fix this issue creating a Signature for SHA256withRSA and registering it in my custom Provider.

[satur9nine]

@farble1670 This appears to be the commit that broke us: https://github.com/google/conscrypt/commit/80469b4a9eae1f51d1c7af42963c452eb2d35fcd

Our custom keystore only allowed Signature and not Cipher so after this change conscrypt cannot use our keystore to perform mutual TLS. Ouch.