search

hivemq / hivemq-mqtt-client

HiveMQ MQTT Client is an MQTT 5.0 and MQTT 3.1.1 compatible and feature-rich high-performance Java client library with different API flavours and backpressure support
https://hivemq.github.io/hivemq-mqtt-client/
Apache License 2.0
847 stars 158 forks source link

SSL failing on KitKat (Android 4.4) #511

Closed whyameye closed 2 years ago

whyameye commented 2 years ago

Expected behavior

connects successfully without error

Actual behavior

com.hivemq.client.mqtt.exceptions.ConnectionFailedException: javax.net.ssl.SSLProtocolException: Unexpected message type has been received: 21

To Reproduce

SSL works great on Android 11. On Android 4.4 with the same code we see:

2021-11-15 19:03:46.998 3682-3682/com.example.myapplication E/dalvikvm: Could not find class 'org.slf4j.spi.LocationAwareLogger', referenced from method io.netty.util.internal.logging.Slf4JLoggerFactory.wrapLogger
2021-11-15 19:03:46.998 3682-3682/com.example.myapplication E/dalvikvm: Could not find class 'io.netty.util.internal.logging.Log4J2Logger', referenced from method io.netty.util.internal.logging.Log4J2LoggerFactory.newInstance
2021-11-15 19:03:47.088 3682-3682/com.example.myapplication E/dalvikvm: Could not find class 'io.netty.util.internal.LongAdderCounter', referenced from method io.netty.util.internal.PlatformDependent.newLongCounter
2021-11-15 19:03:47.418 3682-3701/com.example.myapplication E/dalvikvm: Could not find class 'io.netty.internal.tcnative.SSL', referenced from method io.netty.handler.ssl.OpenSsl.loadTcNative
2021-11-15 19:03:47.668 3682-3716/com.example.myapplication E/MQTT: FAIL
2021-11-15 19:03:47.668 3682-3716/com.example.myapplication E/MQTT: com.hivemq.client.mqtt.exceptions.ConnectionFailedException: javax.net.ssl.SSLProtocolException: Unexpected message type has been received: 21

Steps

install the code below. Unfortunately we need to support API 19 (Android 4.4 KitKat) or there would be no problem. :)

Reproducer code

MainActivity.java:

package com.example.myapplication;

import android.content.res.AssetFileDescriptor;
import android.content.res.AssetManager;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.util.Log;

import com.hivemq.client.mqtt.MqttClient;
import com.hivemq.client.mqtt.mqtt3.Mqtt3AsyncClient;

import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileDescriptor;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.security.KeyStore;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.UnrecoverableKeyException;
import java.security.cert.CertificateException;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;

import javax.net.ssl.KeyManagerFactory;
import javax.net.ssl.TrustManagerFactory;

public class MainActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        Log.i("MQTT","RUNNING");

        KeyStore trustStore = null;
        try {
            trustStore = KeyStore.getInstance("BKS");
        } catch (KeyStoreException e) {
            e.printStackTrace();
        }

        AssetManager assetManager = getAssets();
        InputStream iis = null;

        try {
            iis = assetManager.open("try2.bks");
        } catch (IOException e) {
            e.printStackTrace();
        }

        try {
            trustStore.load(iis, MYPASS.toCharArray());
        } catch (CertificateException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        }

        TrustManagerFactory tmf = null;
        try {
            tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        } catch (NoSuchAlgorithmException e) {
            e.printStackTrace();
        }
        try {
            tmf.init(trustStore);
        } catch (KeyStoreException e) {
            e.printStackTrace();
        }

        Mqtt3AsyncClient client = MqttClient.builder()
                .useMqttVersion3()
                .identifier(MYID)
                .serverHost(MYHOST)
                .serverPort(MYPORT)
                .sslConfig()
                .trustManagerFactory(tmf)
                .applySslConfig()
                .buildAsync();
        client.connect()
                .whenComplete((connAck, throwable) -> {
                    if (throwable != null) {
                        Log.e("MQTT", "FAIL");
                        Log.e("MQTT", String.valueOf(throwable));
                    } else {
                        Log.i("MQTT", "SUCCESS");
                    }
                });
    }
}

build.gradle:

buildscript {
    repositories {
        google()
        gradlePluginPortal()
        jcenter()
    }
    dependencies {
        classpath 'gradle.plugin.com.github.sgtsilvio.gradle:android-retrofix:0.4.1'
    }
}

plugins {
    id 'com.android.application'
}

android {
    compileSdk 30

    defaultConfig {
        applicationId "com.example.myapplication"
        minSdk 19
        targetSdk 19
        versionCode 1
        versionName "1.0"
        packagingOptions {
            exclude 'META-INF/INDEX.LIST'
            exclude 'META-INF/io.netty.versions.properties'
        }
        testInstrumentationRunner "android.support.test.runner.AndroidJUnitRunner"
    }

    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android-optimize.txt'), 'proguard-rules.pro'
        }
    }
    compileOptions {
        sourceCompatibility JavaVersion.VERSION_1_8
        targetCompatibility JavaVersion.VERSION_1_8
    }
}

dependencies {
    implementation("com.hivemq:hivemq-mqtt-client:1.2.2")
    implementation 'net.sourceforge.streamsupport:android-retrostreams:1.7.4'
    implementation 'net.sourceforge.streamsupport:android-retrofuture:1.7.4'
    implementation 'com.android.support:appcompat-v7:28.0.0'
    implementation 'com.android.support.constraint:constraint-layout:2.0.4'
    testImplementation 'junit:junit:4.+'
    androidTestImplementation 'com.android.support.test:runner:1.0.2'
    androidTestImplementation 'com.android.support.test.espresso:espresso-core:3.0.2'
}

apply plugin: 'com.android.application'
apply plugin: 'com.github.sgtsilvio.gradle.android-retrofix'

Notes

Details

whyameye commented 2 years ago

The code runs correctly on Android 5.0 (API 21)

whyameye commented 2 years ago

The errors were a red herring. The client works and the issue was that our server does not support below TLSv1.2 and Android 4.4 does not support TLSv1.2. When we changed a test server to connect using TLSv1.0 the above code worked. There's already issue #423 regarding TLSv1.2 which is closed so I will close out this issue as well.

It looks like TLSv1.2 can be supported if we create our own socketfactory class. If anybody has tips for us on how we could get this library to use our socketfactory class without recompiling this client, please let us know.

whyameye commented 2 years ago

If anybody else wants to support prehistoric Android API 19 (4.4 Kitkat) with MQTT using TLSv1.2 I did manage to get it working but I used the paho mqtt library client not this library to do it, and I added this class to my code. In my onCreate I have:

  MqttClient client = new MqttClient("ssl://SERVER:PORT", clientId, new MemoryPersistence());
  MqttConnectOptions mqttConnectOptions = new MqttConnectOptions();
  mqttConnectOptions.setSocketFactory(getTruststoreFactory());

and the method getTruststoreFactory relevant code is:

    public SocketFactory getTruststoreFactory() throws Exception {
        KeyStore trustStore = KeyStore.getInstance("BKS");
        AssetManager assetManager = getAssets();
        InputStream in = assetManager.open("MY_CERT.bks");
        trustStore.load(in, "MYPASSWORD".toCharArray());
        TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
        tmf.init(trustStore);
        return new TLSSocketFactory(tmf.getTrustManagers());
    }
SgtSilvio commented 2 years ago

Hi @whyameye can you please try specifying TLSv1.2 explicitly:

MqttClient.builder()
    ...
    .sslConfig()
        .trustManagerFactory(tmf)
        .protocols(Arrays.asList("TLSv1.2"))
        .applySslConfig()
    ...

If I understand the links correctly, this should solve your problem without requiring you to write/copy custom code.

whyameye commented 2 years ago

Thanks for the response, Silvio. When I add that line and try to run on Kitkat (API 19) I get: E/MQTT: q2.b: java.lang.IllegalArgumentException: Protocol TLSv1.2 is not supported. which is being printed to the log by my line Log.e("MQTT", String.valueOf(throwable)); in the original code. The code runs fine with or without your added line in API 21.

I'm with you in that if there is a way to do this without custom code I'm all in.

SgtSilvio commented 2 years ago

@whyameye I did some research and it turns out that javax.net.ssl.SSLEngine on Android does support TLSv1.2 only on API 20+ (https://developer.android.com/reference/javax/net/ssl/SSLEngine.html#default-configuration-for-different-android-versions) This library uses the Netty framework which does not use SocketFactory and SSLSocket but only SSLEngine. This means Android does not support using TLSv1.2 with Netty and this library.

whyameye commented 2 years ago

Yes that all makes sense. I'll have to go with the other library and the custom code or update to API 21. It least I have options. :) Thanks for your help.

SgtSilvio commented 2 years ago

Just additional info for someone that might stumble upon this: Installing updates through Google Play Services should make this work according to this stackoverflow post: https://stackoverflow.com/questions/24357863/making-sslengine-use-tlsv1-2-on-android-4-4-2 ProviderInstaller.installIfNeeded(getApplicationContext());

SgtSilvio commented 2 years ago

Also using conscrypt (https://github.com/google/conscrypt, https://source.android.com/devices/architecture/modular-system/conscrypt) should work: implementation("org.conscrypt:conscrypt-android:2.5.2") Security.insertProviderAt(Conscrypt.newProvider(), 1)