Aws IoT connection on Flutter (for Android and iOS) is having exception

[rankush]

Hi,

First of all thanks for writing much required package for MQTT.

I am facing 2 serious issue when thring connecting to AWS IoT broker from Flutter (Android and iOS) while using your lib.

Details: Mqtt_client : 7.2.1 Flutter : 1.17.4 Dart : 2.8.4

I need your comments, whether its a client issue or server issue , or my understanding for integrating the lib is wrong.

• We are getting the pem certificate from our local server, which I am passing to lib through securityContext.setTrustedCertificatesBytes(). I am saving the certificate received from server directly into string and getting bytes through codeUnits API of dart
• There is no user and password authentication
• and endpoint region tried US and South of AWS

Below is the code:

void _setupMqttClient() {
    client = MqttServerClient.withPort(_iotEndpoint, _clientId, 443, maxConnectionAttempts: 5);
    client.logging(on: true);
    client.keepAlivePeriod = 20;
    client.onDisconnected = _onDisconnected;
    client.onConnected = _onConnected;
    client.onSubscribed = _onSubscribed;
    client.secure = true;

    final securityContext = SecurityContext.defaultContext;
    securityContext.setTrustedCertificatesBytes(_cert.codeUnits);
    securityContext.usePrivateKeyBytes(_privateKey.codeUnits);

    client.securityContext = securityContext;
    client.setProtocolV311();

    final MqttConnectMessage connMess = MqttConnectMessage()
        .withClientIdentifier(_clientId)
        .startClean()
        .keepAliveFor(30);

    client.connectionMessage = connMess;
  }

  Future<void> _connectClient() async {
    try {
      print('MQTTClientWrapper::Mosquitto client connecting....');
      connectionState = MqttCurrentConnectionState.CONNECTING;
      await client.connect();
    } on Exception catch (e) {
      print('MQTTClientWrapper::client exception - $e');
      connectionState = MqttCurrentConnectionState.ERROR_WHEN_CONNECTING;
      client.disconnect();
    }

    if (client.connectionStatus.state == MqttConnectionState.connected) {
      connectionState = MqttCurrentConnectionState.CONNECTED;
      print('MQTTClientWrapper::Mosquitto client connected');
    } else {
      print(
          'MQTTClientWrapper::ERROR Mosquitto client connection failed - disconnecting, status is ${client.connectionStatus}');
      connectionState = MqttCurrentConnectionState.ERROR_WHEN_CONNECTING;
      client.disconnect();
    }
  }

Issues:

1. Not able to connect AWS iot broker, client (iOS application) is exhausting with max trials, and throwing "The broker is not responding to the connection request message (Missing Connection Acknowledgement?"

The broker is working fine as if I try to hook from a native iOS app(using mosquito mqtt lib) or native android app (aws iot lib based on paho mqtt lib), is able to.

Am i doing some thing wrong while building mqttClient object?

Logs:

flutter: 2020-06-24 16:32:19.073334 -- SynchronousMqttServerConnectionHandler::internalConnect - initiating connection try 4
flutter: 2020-06-24 16:32:19.073682 -- SynchronousMqttServerConnectionHandler::internalConnect - secure selected
flutter: 2020-06-24 16:32:19.073949 -- MqttSecureConnection::connect
flutter: 2020-06-24 16:32:19.855360 -- MqttSecureConnection::connect - securing socket
flutter: 2020-06-24 16:32:19.855670 -- MqttSecureConnection::connect - start listening
flutter: 2020-06-24 16:32:19.855893 -- MqttServerConnection::_startListening
flutter: 2020-06-24 16:32:19.856224 -- SynchronousMqttServerConnectionHandler::internalConnect - connection complete
flutter: 2020-06-24 16:32:19.856464 -- SynchronousMqttServerConnectionHandler::internalConnect sending connect message
flutter: 2020-06-24 16:32:19.856736 -- MqttConnectionHandlerBase::sendMessage - MQTTMessage of type MqttMessageType.connect
Header: MessageType = MqttMessageType.connect, Duplicate = false, Retain = false, Qos = MqttQos.atMostOnce, Size = 87
Connect Variable Header: ProtocolName=MQTT, ProtocolVersion=4, ConnectFlags=Connect Flags: Reserved1=false, CleanStart=true, WillFlag=false, WillQos=MqttQos.atMostOnce, WillRetain=false, PasswordFlag=false, UserNameFlag=false, KeepAlive=30
MqttConnectPayload - client identifier is : 5e4bb87f3a914424e0a3fe76_stgspr_mobile_C001489E-BDB0-4DFE-B6F8-58F78A7E1ADE
flutter: 2020-06-24 16:32:19.857457 -- SynchronousMqttServerConnectionHandler::internalConnect - pre sleep, state = Connection status is connecting with return code of noneSpecified and a disconnection origin of none
flutter: 2020-06-24 16:32:24.861702 -- SynchronousMqttServerConnectionHandler::internalConnect - post sleep, state = Connection status is connecting with return code of noneSpecified and a disconnection origin of none
flutter: 2020-06-24 16:32:24.862216 -- SynchronousMqttServerConnectionHandler::internalConnect failed
flutter: MQTTClientWrapper::client exception - mqtt-client::NoConnectionException: The maximum allowed connection attempts ({5}) were exceeded. The broker is not responding to the connection request message (Missing Connection Acknowledgement?
flutter: 2020-06-24 16:32:24.863332 -- SynchronousMqttServerConnectionHandler::disconnect
flutter: MQTTClientWrapper::OnDisconnected client callback - Client disconnection
flutter: MQTTClientWrapper::ERROR Mosquitto client connection failed - disconnecting, status is Connection status is disconnected with return code of noneSpecified and a disconnection origin of solicited
flutter: MQTTClientWrapper::OnDisconnected client callback - Client disconnection

2. And if I just change the connectionmessage 's QOS to atLeastOne, lib is throwing exception : Unhandled Exception: RangeError (index): Invalid value: Not in range 0..6, inclusive: 101.

Code change:

 final MqttConnectMessage connMess = MqttConnectMessage()
        .withClientIdentifier(_clientId)
        .startClean()
        .withWillQos(MqttQos.atLeastOnce).  **_<----- this i have added in second trial_**
        .keepAliveFor(30);

Logs:

flutter: 2020-06-24 15:35:45.628628 -- SynchronousMqttServerConnectionHandler::internalConnect - initiating connection try 3
flutter: 2020-06-24 15:35:45.629057 -- SynchronousMqttServerConnectionHandler::internalConnect - secure selected
flutter: 2020-06-24 15:35:45.629460 -- MqttSecureConnection::connect
flutter: 2020-06-24 15:35:47.886323 -- MqttSecureConnection::connect - securing socket
flutter: 2020-06-24 15:35:47.886632 -- MqttSecureConnection::connect - start listening
flutter: 2020-06-24 15:35:47.886915 -- MqttServerConnection::_startListening
flutter: 2020-06-24 15:35:47.887275 -- SynchronousMqttServerConnectionHandler::internalConnect - connection complete
flutter: 2020-06-24 15:35:47.887500 -- SynchronousMqttServerConnectionHandler::internalConnect sending connect message
flutter: 2020-06-24 15:35:47.887733 -- MqttConnectionHandlerBase::sendMessage - MQTTMessage of type MqttMessageType.connect
Header: MessageType = MqttMessageType.connect, Duplicate = false, Retain = false, Qos = MqttQos.atMostOnce, Size = 87
Connect Variable Header: ProtocolName=MQTT, ProtocolVersion=4, ConnectFlags=Connect Flags: Reserved1=false, CleanStart=true, WillFlag=false, WillQos=MqttQos.atLeastOnce, WillRetain=false, PasswordFlag=false, UserNameFlag=false, KeepAlive=30
MqttConnectPayload - client identifier is : 5e4bb87f3a914424e0a3fe76_stgspr_mobile_C001489E-BDB0-4DFE-B6F8-58F78A7E1ADE
flutter: 2020-06-24 15:35:47.888342 -- SynchronousMqttServerConnectionHandler::internalConnect - pre sleep, state = Connection status is connecting with return code of noneSpecified and a disconnection origin of none
flutter: 2020-06-24 15:35:48.268803 -- MqttConnection::_onData
flutter: 2020-06-24 15:35:48.269255 -- MqttServerConnection::_onData - message received MQTTMessage of type MqttMessageType.publishAck
Header: MessageType = MqttMessageType.publishAck, Duplicate = true, Retain = false, Qos = MqttQos.atMostOnce, Size = 84
PublishAck Variable Header: MessageIdentifier={21584}
flutter: 2020-06-24 15:35:48.269531 -- MqttServerConnection::_onData - message not processed, disconnecting
flutter: 2020-06-24 15:35:48.269821 -- MqttServerConnection::_onData - message received MQTTMessage of type MqttMessageType.publishRelease
Header: MessageType = MqttMessageType.publishRelease, Duplicate = false, Retain = false, Qos = MqttQos.exactlyOnce, Size = 44
PublishRelease Variable Header: MessageIdentifier={8242}
flutter: 2020-06-24 15:35:48.270066 -- MqttServerConnection::_onData - message not processed, disconnecting
[VERBOSE-2:ui_dart_state.cc(157)] Unhandled Exception: RangeError (index): Invalid value: Not in range 0..6, inclusive: 101
#0      List.[] (dart:core-patch/array.dart:169:52)
#1      MqttVariableHeader.readReturnCode (package:mqtt_client/src/messages/mqtt_client_mqtt_variable_header.dart:183:46)
#2      MqttConnectAckVariableHeader.readFrom (package:mqtt_client/src/messages/connectack/mqtt_client_mqtt_connect_ack_variable_header.dart:35:5)
#3      new MqttVariableHeader.fromByteBuffer (package:mqtt_client/src/messages/mqtt_client_mqtt_variable_header.dart:51:5)
#4      new MqttConnectAckVariableHeader.fromByteBuffer (package:mqtt_client/src/messages/connectack/mqtt_client_mqtt_connect_ack_variable_header.dart:17:15)
#5      MqttConnectAckMessage.readFrom (package:mqtt_client/src/messages/connectack/mqtt_client_mqtt_connect_ack_message.dart:36:51)
#6      new MqttConnectAckMessage.fromByteBuffer (package:mqtt_client/src/messages/connectack/mqtt_client_mqtt_connect_ack_message.dart:25:5)
#7      MqttM<…>
flutter: 2020-06-24 15:35:48.272595 -- MqttConnectionBase::_onDone - calling disconnected callback
flutter: MQTTClientWrapper::OnDisconnected client callback - Client disconnection

Kindly help!!!

Thanks Rankush

[shamblett]

From your log above you are connecting, but instead of receiving a connect ack you are receiving a puback followed by a pubrel, this is not what the client is expecting, either the broker is sending these or the client is interpreting the incoming byte stream from the network incorrectly, this unlikely but not unheard of if you haven't completely flushed the network buffer between tests.

[rankush]

Hi,

Thanks for prompt reply.

1) We are following the standard ways for AWS iot broker. None of the feature we are customising, everything is of AWS defaults. We are just added our own policies to authenticate the client. that's it.

2) We are making sure the network buffer is flushed between the test cycle at-least by uninstalling/or closing the process.

3) Just a info : i can see the lib is also sending a message to broker as "MQTTMessageType.connect", is it having the same type which AWS iot is expecting ?

I have tried with pem and pkcs12 ssl certificate but again getting same kind of error, then I have tried to debug the your lib to understand the issue, it seems library is expecting some bit under some range. And I got little bit confused about the port , SocketExceptions port is telling 458, where i was trying to connect over 443.

attached is the screen shot.

Kindly provide your valuable feedback.

[rankush]

And for your reference , if I am using the mobile client lib for iOS and Android, I am able to communicate broker.

https://docs.aws.amazon.com/iot/latest/developerguide/iot-sdks.html

Even in linux, paho mqtt client is able to communicate with SSL.

[shamblett]

The client sends a connect message to the broker as its first message, this is nothing to do with AWS , its the MQTT standard.

Looking here at the AWS IOT documentation, it states that to use port 443 you must implement the Application Layer Protocol Negotiation (ALPN). Without looking I'm not sure if Dart supports this(or the rest of the stuff the page mentions) or not, you need to ask the Dart guys this, its nothing to do with the client.

The non ALP secure socket port is 8883, the MQTT standard, this is the port other AWS users use, try this port, this should verify your setup.

[rankush]

Ok , I will try on PORT 8883, and will let you know the results in 2 days (at max).

[rankush]

Bingo!!

I am able to connect if port is 8883.

What I understood from other documentation of different types of brokers, 443 is basically for web socket connection. Normal port is 8883.

Thanks for your support !!

[sureace]

_cert

Hi @rankush , Could you please share the working code. I am facing an issue when i try to connect with AWS IOT Core. Thanks, Sureace

[SankalpMe]

The client sends a connect message to the broker as its first message, this is nothing to do with AWS , its the MQTT standard.

Looking here at the AWS IOT documentation, it states that to use port 443 you must implement the Application Layer Protocol Negotiation (ALPN). Without looking I'm not sure if Dart supports this(or the rest of the stuff the page mentions) or not, you need to ask the Dart guys this, its nothing to do with the client.

The non ALP secure socket port is 8883, the MQTT standard, this is the port other AWS users use, try this port, this should verify your setup.

Dart does support ALPN, here is a short snippet: Function: context.setAlpnProtocols(List<String> protocols,bool isServer) An example:

final context = SecurityContext.defaultContext;
context.setClientAuthorities('AmazonRootCA1.pem');
context.setAlpnProtocols(["x-amzn-mqtt-ca"], false);
context.useCertificateChain('pem.crt');
context.usePrivateKey('private.pem.key');
client.securityContext = context;

If you want to connect over the 443 port.