SSL error when porting from node:tls with a self sign certificate (SSLV3_ALERT_HANDSHAKE_FAILURE or CertPathValidatorException)

[vricosti]

Hi,

I am trying to port a library androidtv-remote that was originally designed to run in a node environment to be able to run inside a react-native app. For the ssl part it was using node:tls and it works fine with it.

The android device I am connecting to is using a self signed certificate and the app is using a self signed certificate too.

Here is the server ssl characteristics:

openssl s_client -state -connect 192.168.1.102:6467
CONNECTED(00000003)
SSL_connect:before SSL initialization
SSL_connect:SSLv3/TLS write client hello
SSL_connect:SSLv3/TLS write client hello
Can't use SSL_get_servername
SSL_connect:SSLv3/TLS read server hello
depth=0 dnQualifier = full_fbx6lcv2/fbx6lcv2/Freebox Player Mini v2, CN = atvremote/863600S191201703
verify error:num=18:self-signed certificate
verify return:1
depth=0 dnQualifier = full_fbx6lcv2/fbx6lcv2/Freebox Player Mini v2, CN = atvremote/863600S191201703
verify return:1
SSL_connect:SSLv3/TLS read server certificate
SSL_connect:SSLv3/TLS read server key exchange
SSL_connect:SSLv3/TLS read server certificate request
SSL_connect:SSLv3/TLS read server done
SSL_connect:SSLv3/TLS write client certificate
SSL_connect:SSLv3/TLS write client key exchange
SSL_connect:SSLv3/TLS write change cipher spec
SSL_connect:SSLv3/TLS write finished
SSL3 alert read:fatal:handshake failure
SSL_connect:error in error
40277A58637E0000:error:0A000410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:../ssl/record/rec_layer_s3.c:1584:SSL alert number 40
---
Certificate chain
 0 s:dnQualifier = full_fbx6lcv2/fbx6lcv2/Freebox Player Mini v2, CN = atvremote/863600S191201703
   i:dnQualifier = full_fbx6lcv2/fbx6lcv2/Freebox Player Mini v2, CN = atvremote/863600S191201703
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Jan 25 11:00:00 2021 GMT; NotAfter: Jan 19 03:14:07 2038 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIICNzCCAiCgAwIBAgIGAXc6px/tMA0GCSqGSIb3DQEBCwUAMF0xNjA0BgNVBC4M
LWZ1bGxfZmJ4NmxjdjIvZmJ4NmxjdjIvRnJlZWJveCBQbGF5ZXIgTWluaSB2MjEj
MCEGA1UEAxMaYXR2cmVtb3RlLzg2MzYwMFMxOTEyMDE3MDMwHhcNMjEwMTI1MTEw
MDAwWhcNMzgwMTE5MDMxNDA3WjBdMTYwNAYDVQQuDC1mdWxsX2ZieDZsY3YyL2Zi
eDZsY3YyL0ZyZWVib3ggUGxheWVyIE1pbmkgdjIxIzAhBgNVBAMTGmF0dnJlbW90
ZS84NjM2MDBTMTkxMjAxNzAzMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
AQEA1hmo54HHzsk8i0+/hNax+zVtZR4WDP6F/3JXT1Y3YtYVP7HjW5THXSXvn+4i
VPgOwwFrg5fFd2N6hnk5As56Ks2h+r/r3UzuHvMiYnkkpbW89b0VoAhY740wShky
xY0TJx/Ipj1YlU65oB6AxWWM78MAHIAtNJuB+/DFvKTpZoGj32Eq2lHyEhfnHCYU
rkT27jkTp9TyWj6LIYkZFfcykgPKu90LDNqFtxrBCvA9d354TyrbZdBYbW3Ca4Vh
0jpfuxDPzOyRnSGWi4IwFZdIuv0vLu1TMOyp3aFzIbR4L+EBvvNKoSwDWFPnVRns
UAP+50dCjk6/zSNu4Fou4EwikwIDAQABMA0GCSqGSIb3DQEBCwUAAwIAAA==
-----END CERTIFICATE-----
subject=dnQualifier = full_fbx6lcv2/fbx6lcv2/Freebox Player Mini v2, CN = atvremote/863600S191201703
issuer=dnQualifier = full_fbx6lcv2/fbx6lcv2/Freebox Player Mini v2, CN = atvremote/863600S191201703
---
No client certificate CA names sent
Client Certificate Types: RSA sign, ECDSA sign
Requested Signature Algorithms: RSA+SHA512:ECDSA+SHA512:RSA+SHA384:ECDSA+SHA384:RSA+SHA256:ECDSA+SHA256:RSA+SHA224:ECDSA+SHA224:RSA+SHA1:ECDSA+SHA1
Shared Requested Signature Algorithms: RSA+SHA512:ECDSA+SHA512:RSA+SHA384:ECDSA+SHA384:RSA+SHA256:ECDSA+SHA256:RSA+SHA224:ECDSA+SHA224
Peer signing digest: SHA256
Peer signature type: RSA
Server Temp Key: ECDH, prime256v1, 256 bits
---
SSL handshake has read 1072 bytes and written 423 bytes
Verification error: self-signed certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-CHACHA20-POLY1305
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-CHACHA20-POLY1305
    Session-ID: E5935763D407F7CE037E1EE9F10B714E5347182F06929B4C07A17FCA936C37AB
    Session-ID-ctx: 
    Master-Key: 1F6A12558802A1F2DE12478C3E1146A6ECCB4F1A08CF657B14C196CA7FC9DDBD873C83EFC3DE1DBE0F72D138815E65F1
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1716205017
    Timeout   : 7200 (sec)
    Verify return code: 18 (self-signed certificate)
    Extended master secret: yes
---

The application I am using is available here: https://github.com/vricosti/TestAndroidTVRemoteApp

At the beginning a self signed certificate is generated inside packages/androidtv-remote/src/certificate/CertificateGenerator.js

this.cert = await CertificateGenerator.generateFull(
                this.service_name,
                'CNT',
                'ST',
                'LOC',
                'O',
                'OU'
            );

this.cert object holds 2 fields, key and cert holding private key and certificate in pem

and then this certificate was used to start the tls connection:

async start() {
        return new Promise((resolve, reject) => {
            let options = {
                port: this.port,
                host : this.host,
                key: this.certs.key,
                cert: this.certs.cert,
                rejectUnauthorized: false,
            };

            if (jsEnv.isNodeOrDeno) {
                console.debug('set options to use node:tls');

                this.client = tls.connect(options, () => {
                    console.debug(this.host + " Pairing connected");
                });

            } else if (jsEnv.isReactNative) {
                console.debug('set options to use react-native-tcp-socket');
                options.tls = true;
                options.tlsCheckValidity = false;
                options.cert = this.certs.cert;

                this.client = TcpSockets.connectTLS(options, () => {
                    console.debug(this.host + " Pairing connected");
                });
            }

            this.client.pairingManager = this;

            //const connectEventName = jsEnv.isNodeOrDeno ? "secureConnect" : "connect";
            this.client.on("secureConnect", () => {
                console.debug(this.host + " Pairing secure connected ");
                this.client.write(this.pairingMessageManager.createPairingRequest(this.service_name));
            });

            this.client.on('data', (data) => {
                let buffer = Buffer.from(data);
                this.chunks = Buffer.concat([this.chunks, buffer]);

                if(this.chunks.length > 0 && this.chunks.readInt8(0) === this.chunks.length - 1){

                    let message = this.pairingMessageManager.parse(this.chunks);

                    console.debug("Receive : " + Array.from(this.chunks));
                    console.debug("Receive : " + JSON.stringify(message.toJSON()));

                    if (message.status !== this.pairingMessageManager.Status.STATUS_OK){
                        this.client.destroy(new Error(message.status));
                    }
                    else {
                        if(message.pairingRequestAck){
                            this.client.write(this.pairingMessageManager.createPairingOption());
                        }
                        else if(message.pairingOption){
                            this.client.write(this.pairingMessageManager.createPairingConfiguration());
                        }
                        else if(message.pairingConfigurationAck){
                            this.emit('secret');
                        }
                        else if(message.pairingSecretAck){
                            console.debug(this.host + " Paired!");
                            this.client.destroy();
                        }
                        else {
                            console.debug(this.host + " What Else ?")
                        }
                    }
                    this.chunks = Buffer.from([]);
                }
            });

            this.client.on('close', (hasError) => {
                console.debug(this.host + " Pairing Connection closed", hasError);
                if(hasError){
                    reject(false);
                }
                else{
                    resolve(true);
                }
            });

            this.client.on('error', (error) => {
                console.error(error);
            });
        });
    }

I have the following error:

Entering useEffect()
 LOG  Before instantiating AndroidRemote
 LOG  AndroidRemote.constructor
 LOG  After instantiating AndroidRemote
 LOG  Before start()
 LOG  AndroidRemote.start
 LOG  before CertificateGenerator.generateFull
 LOG  Entering generateFull
 LOG  after generateKeyPair with keys:  {"privateKey": {"d": {"data": [Array], "s": 0, "t": 79}, "dP": {"data": [Array], "s": 0, "t": 40}, "dQ": {"data": [Array], "s": 0, "t": 40}, "decrypt": [Function anonymous], "e": {"data": [Array], "s": 0, "t": 1}, "n": {"data": [Array], "s": 0, "t": 79}, "p": {"data": [Array], "s": 0, "t": 40}, "q": {"data": [Array], "s": 0, "t": 40}, "qInv": {"data": [Array], "s": 0, "t": 40}, "sign": [Function anonymous]}, "publicKey": {"e": {"data": [Array], "s": 0, "t": 1}, "encrypt": [Function anonymous], "n": {"data": [Array], "s": 0, "t": 79}, "verify": [Function anonymous]}}
 LOG  after createCertificate with cert:  {"extensions": [], "generateSubjectKeyIdentifier": [Function anonymous], "getExtension": [Function anonymous], "isIssuer": [Function anonymous], "issued": [Function anonymous], "issuer": {"addField": [Function anonymous], "attributes": [], "getField": [Function anonymous], "hash": null}, "md": null, "publicKey": null, "serialNumber": "00", "setExtensions": [Function anonymous], "setIssuer": [Function anonymous], "setSubject": [Function anonymous], "siginfo": {"algorithmOid": null}, "sign": [Function anonymous], "signature": null, "signatureOid": null, "subject": {"addField": [Function anonymous], "attributes": [], "getField": [Function anonymous], "hash": null}, "validity": {"notAfter": 2024-05-17T08:13:05.387Z, "notBefore": 2024-05-17T08:13:05.387Z}, "verify": [Function anonymous], "verifySubjectKeyIdentifier": [Function anonymous], "version": 2}
 LOG  after CertificateGenerator.generateFull
 LOG  before new PairingManager
 LOG  PairingMessageManager.constructor
 LOG  after new PairingManager
 DEBUG  set options to use react-native-tcp-socket
 Read error: ssl=0x756ae993d8: Failure in SSL library, usually a protocol error
error:10000410:SSL routines:OPENSSL_internal:SSLV3_ALERT_HANDSHAKE_FAILURE (external/boringssl/src/ssl/tls_record.cc:592 0x751aef9020:0x00000003)

if use rejectUnauthorized: true

 ERROR  java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.

So in both case it fails...

I tried to add the server certificate and pass it but same error:

let options = {
                port: this.port,
                host : this.host,
                ca: require('../../../../server-cert.pem'),
                key: this.certs.key,
                cert: this.certs.cert,
                //rejectUnauthorized: false,
            };

and I even check that inside the android code it receives the passed certificate...

thanks in advance

[vricosti]

Ok I found the fix and basically it implies to save the cert and the private key inside a keystore, I forked it.

[ManuelLatorre98]

Ok I found the fix and basically it implies to save the cert and the private key inside a keystore, I forked it.

you mean the private key of client or server?

[vricosti]

Ok I found the fix and basically it implies to save the cert and the private key inside a keystore, I forked it.

you mean the private key of client or server?

private key of client In my branch I have also allowed the possibility to load key/cert/ca as a string: https://github.com/Rapsssito/react-native-tcp-socket/compare/master...vricosti:react-native-tcp-socket:dev/more-node-tls-compliant

Now I will try to implement getCertificate and getPeerCertificate, on android it should be easy but no idea about ios.