Trying to connect to AWS IoT, connection not upgraded to websocket

[felixeriksson]

Hi! First of all, thanks for making this neat library. I started with Dart (through Flutter) just a few months ago, and it makes me happy to see that there's so many useful libraries with a clear API, given that the language is still quite young.

I'm trying to connect to AWS IoT with Cognito credentials. My goal is to get a JSON document representing the so-called "device shadow", a sort of digital twin, of an IoT device. I found the library https://pub.dev/packages/aws_iot_device which wraps this library, but it was outdated as it directly instantiated MqttClient(). I tried to adapt it to use MqttServerClient() instead, but I haven't been completely successful so far. I'm receiving the exception Connection to 'https://<endpoint>/$aws/things/<thing_name>/shadow/get?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIARBRC4UMDLW2PASOE%2F20210207%2Feu-north-1%2Fiotdevicegateway%2Faws4_request&X-Amz-Date=20210207T140702Z&X-Amz-Expire=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=2dabd87fe3bc8f4279cf2d99e67982b46289bdf70bb52dc8853d98874ba117ae&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGYaDGV1LWNlbnRyYWwtMSJGMEQCICZii%2FR47I1mfyHr5MPhAKcZSays8oc9CAYK6uWJM7k8AiAqaZKP16hIojqHJ6924faO5OMWa6wacSsugnhuviyPuSrWBAhfEAAaDDA3MjAxMzg4MjExOCIMd0QVybMaI5I5%2FhTGKrMEZAEkJ%2B5zJ%2BMt9COBU4Sd3ein6W6bCiUZKlbkKDuDHynO%2BnUIN2Fh3f2N7m%2F05qWT7eAAp%2F5DtcVYiGil6gpOdTyZy4uQ6lPYPaH5PCqN8EqGtgeUDJqfxKICtokQTl42QOTMbhuHpcf5JL0Ad5yw76asgLsK412uNjdQ77EUJLg2%2Fpvqou7xl%2FzTA8UlLCLjcKz6Bc9neCHdPQEAw1sPIxVYt2m8HcR2ggBu9kLfWsoDNppTrbuTwAWl5yx2osMPp2Gx3noQkzj0f3LBXboNNM%2BrwV7ofcp%2BHuC3vz%2FOAMAxztqKN3owf6cWQblylNA7qF76R5M5AdhgGPK59Hg6QKc%2Bq8EqPn2nWwpolsjDv0FWWI0bgG4FmfgIwF6CnI0WeK2fxYyXvKmCOqcYmb1IqEGb7UDxH3RBqG3FP6nQwflYhN8WzrRESX64Dy3SZrVth1QwfWKcibjh%2B5E6mMr9IgkoqSH5JAeZ8CMZ0I%2BUMudvYUpwJzRW6F2AhriKWnYpCeNP0Stb90yJiKlRu%2BLNU5ty7gQC0iFe0csYPnBtX2MqzyMnQ7Bfhff0ROrn6iPyGyf3BelUegLWYufgFfyiju1%2FYqSAKL8KyYMoYhMnPCV2FhtmNQU054Tn8sYPl2WZYurPCv1XpPWgZ90PjtX7%2BkwvcWX7hpMx97vp%2FNt%2FhV52lull3wyye%2FqCC2pq7E%2BTHLFmHERTtRUZP6tHD6BmosckClv4KptEYHlq2QLu1PC7bcow%2FOb%2FgAY6hgJQuQ7rqEtg81veHY1PGXkHTA5Qd2wFgCDm4wmnFmVR%2BBWmMzxzJXE4%2FYt%2B6VpjpaZG2U%2FZmjKyeyNwGfWq%2FtZ%2BcRjOCoGoUCHJJJn4KKcxUD4nVuqmwIBmc%2B1%2BhfbTvHqi5f%2F6YJe4mBp5OoWCmL4aUqwv1LTr%2B4IaBrT6aGWviwu2tbxyQMRLJLaqZX5u6QO9YzTfyJNMXEpTiwuvfa7kxGTpSJJyvDEZG%2FEQerBzjzKfil%2Buky65h7fWH4oHcAu%2BOKx7D3Q1Jzi3WEzKOxQql4gZKOzfvkO37K9CXzn7Q1KlsGGEzfKzw8Vqa8YKHbOEV7ZzjKTXbSSbGroCW07N6%2BlKeUk2#' was not upgraded to websocket and I can't figure out what I'm doing wrong.

My code looks like this

import 'dart:async';
import 'package:mqtt_client/mqtt_client.dart';
import 'package:mqtt_client/mqtt_server_client.dart';
import 'package:amazon_cognito_identity_dart_2/sig_v4.dart';
import 'package:tuple/tuple.dart';

class AWSIoTDevice {
  final _SERVICE_NAME = 'iotdevicegateway';
  final _AWS4_REQUEST = 'aws4_request';
  final _AWS4_HMAC_SHA256 = 'AWS4-HMAC-SHA256';
  final _SCHEME = 'wss://';

  String _region;
  String _accessKeyId;
  String _secretAccessKey;
  String _sessionToken;
  String _host;
  bool _logging;

  var _onConnected;
  var _onDisconnected;
  var _onSubscribed;
  var _onSubscribeFail;
  var _onUnsubscribed;

  get onConnected => _onConnected;
  set onConnected(val) => _client?.onConnected = _onConnected = val;
  get onDisconnected => _onDisconnected;
  set onDisconnected(val) => _client?.onDisconnected = _onDisconnected = val;
  get onSubscribed => _onSubscribed;
  set onSubscribed(val) => _client?.onSubscribed = _onSubscribed = val;
  get onSubscribeFail => _onSubscribeFail;
  set onSubscribeFail(val) => _client?.onSubscribeFail = _onSubscribeFail = val;
  get onUnsubscribed => _onUnsubscribed;
  set onUnsubscribed(val) => _client?.onUnsubscribed = _onUnsubscribed = val;
  get connectionStatus => _client?.connectionStatus;

  MqttServerClient _client;

  StreamController<Tuple2<String, String>> _messagesController =
      StreamController<Tuple2<String, String>>();

  Stream<Tuple2<String, String>> get messages => _messagesController.stream;

  AWSIoTDevice(
    this._region,
    this._accessKeyId,
    this._secretAccessKey,
    this._sessionToken,
    String host, {
    bool logging: true,
    var onConnected,
    var onDisconnected,
    var onSubscribed,
    var onSubscribeFail,
    var onUnsubscribed,
  }) {
    _logging = logging;
    _onConnected = onConnected;
    _onDisconnected = onDisconnected;
    _onSubscribed = onSubscribed;
    _onSubscribeFail = onSubscribeFail;
    _onUnsubscribed = onUnsubscribed;

    if (host.contains('amazonaws.com')) {
      _host = host.split('.').first;
    } else {
      _host = host;
    }
  }

  Future<Null> connect(String clientId) async {
    if (_client == null) {
      _prepare(clientId);
    }

    try {
      await _client.connect();
    } on Exception catch (e) {
      _client.disconnect();
      throw e;
    }
    _client.updates.listen((List<MqttReceivedMessage<MqttMessage>> c) {
      for (MqttReceivedMessage<MqttMessage> message in c) {
        final MqttPublishMessage recMess = message.payload;
        final String pt =
            MqttPublishPayload.bytesToStringAsString(recMess.payload.message);
        _messagesController.add(Tuple2<String, String>(message.topic, pt));
      }
    });
  }

  _prepare(String clientId) {
    final url = _prepareWebSocketUrl();
    _client = MqttServerClient(url, clientId);
    _client.logging(on: _logging);
    _client.useWebSocket = true;
    _client.port = 443;
    _client.connectionMessage =
        MqttConnectMessage().withClientIdentifier(clientId).keepAliveFor(300);
    _client.keepAlivePeriod = 300;
  }

  _prepareWebSocketUrl() {
    final now = _generateDatetime();
    final hostname = _buildHostname();

    final List creds = [
      this._accessKeyId,
      _getDate(now),
      this._region,
      this._SERVICE_NAME,
      this._AWS4_REQUEST,
    ];

    const payload = '';

    const path =
        '/\$aws/things/<thing name>/shadow/get';

    final queryParams = Map<String, String>.from({
      'X-Amz-Algorithm': _AWS4_HMAC_SHA256,
      'X-Amz-Credential': creds.join('/'),
      'X-Amz-Date': now,
      'X-Amz-SignedHeaders': 'host',
      'X-Amz-Expire': '86400',
    });

    final canonicalQueryString = SigV4.buildCanonicalQueryString(queryParams);
    final request = SigV4.buildCanonicalRequest(
        'GET',
        path,
        queryParams,
        Map.from({
          'host': hostname,
        }),
        payload);

    final hashedCanonicalRequest = SigV4.hashCanonicalRequest(request);
    final stringToSign = SigV4.buildStringToSign(
        now,
        SigV4.buildCredentialScope(now, _region, _SERVICE_NAME),
        hashedCanonicalRequest);

    final signingKey = SigV4.calculateSigningKey(
        _secretAccessKey, now, _region, _SERVICE_NAME);

    final signature = SigV4.calculateSignature(signingKey, stringToSign);

    final finalParams =
        '$canonicalQueryString&X-Amz-Signature=$signature&X-Amz-Security-Token=${Uri.encodeComponent(_sessionToken)}';

    return '$_SCHEME$hostname$path?$finalParams';
  }

  String _generateDatetime() {
    return new DateTime.now()
        .toUtc()
        .toString()
        .replaceAll(new RegExp(r'\.\d*Z$'), 'Z')
        .replaceAll(new RegExp(r'[:-]|\.\d{3}'), '')
        .split(' ')
        .join('T');
  }

  String _getDate(String dateTime) {
    return dateTime.substring(0, 8);
  }

  String _buildHostname() {
    return '$_host.iot.$_region.amazonaws.com';
  }

  void disconnect() {
    return _client.disconnect();
  }
}

this class is used with the following code:

final credentials = await getAWSCredentials();

const region = 'eu-north-1';
const host = '<hostname>';
const deviceId = 'greencontrol-1';

var device = AWSIoTDevice(region, credentials.awsAccessKey,
  credentials.awsSecretKey, credentials.sessionToken, host);

await device.connect(deviceId);

My logs

I/flutter (13131): 1-2021-02-07 15:33:50.976602 -- MqttConnectionHandlerBase::connect - server wss://<hostname>.iot.eu-north-1.amazonaws.com/$aws/things/<thing name>/shadow/get?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIARBRC4UMDPAKXTOG5%2F20210207%2Feu-north-1%2Fiotdevicegateway%2Faws4_request&X-Amz-Date=20210207T143350Z&X-Amz-Expire=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=07693741ed5f08ad86a5843c09831d42c7fb4ae44d29f5df2ce53f5eaf68530e&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGcaDGV1LWNlbnRyYWwtMSJHMEUCIQCnfdy72gXzr4DjmLH%2Fc57jyFQnEP08%2FH9pkUvKvmrvJQIgPvvTvYSMBohfNEIp1%2BaFvsUFDhr%2FY6Nv0ys9vHqRwngq1gQIYBAAGgwwNzIwMTM4ODIxMTgiDCGPhabFR0nCthmBBCqzBPxziz8we6m9sKb%2BOqd%2B%2Fur9rGAcFimjTRJlzOdY3QU7%2BtSMPY8td9QOUUEa5nvoJNCH9py9tSqoCMVWFvZaUVXl1ps0oWIYjvlfmLp3mvXw4XTKWorm39wQQt%2Bt60OaRl6sZwEf%2F3c3utRv0eZDdIcDBJRMNltPqr97PpiSzI2%2Bfw4vPvc3YjRes2JQ%2FqFQK3KZ36vIq%2F8MNeo8BAXR9bPZ9wV1IkOTPTnQ3RHdEOBEQ4EHNNrRzpSHr4bXY03ftx7URoDhR0Mx0WVvuGZ%2FcuA05ADtn7UdYvhxNpEA5MtVRbS47fr%2BM9KCX7btaGteWhkUAXcKmBwHGAC0X
I/flutter (13131): 1-2021-02-07 15:33:50.979355 -- SynchronousMqttServerConnectionHandler::internalConnect entered
I/flutter (13131): 1-2021-02-07 15:33:50.979530 -- SynchronousMqttServerConnectionHandler::internalConnect - initiating connection try 0, auto reconnect in progress false
I/flutter (13131): 1-2021-02-07 15:33:50.979722 -- SynchronousMqttServerConnectionHandler::internalConnect - websocket selected
I/flutter (13131): 1-2021-02-07 15:33:50.980131 -- SynchronousMqttServerConnectionHandler::internalConnect - calling connect
I/flutter (13131): 1-2021-02-07 15:33:50.980961 -- MqttWsConnection::connect - entered
I/flutter (13131): 1-2021-02-07 15:33:50.984831 -- MqttWsConnection::connect - WS URL is wss://<hostname>.iot.eu-north-1.amazonaws.com:443/$aws/things/<thing name>/shadow/get?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIARBRC4UMDPAKXTOG5%2F20210207%2Feu-north-1%2Fiotdevicegateway%2Faws4_request&X-Amz-Date=20210207T143350Z&X-Amz-Expire=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=07693741ed5f08ad86a5843c09831d42c7fb4ae44d29f5df2ce53f5eaf68530e&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGcaDGV1LWNlbnRyYWwtMSJHMEUCIQCnfdy72gXzr4DjmLH%2Fc57jyFQnEP08%2FH9pkUvKvmrvJQIgPvvTvYSMBohfNEIp1%2BaFvsUFDhr%2FY6Nv0ys9vHqRwngq1gQIYBAAGgwwNzIwMTM4ODIxMTgiDCGPhabFR0nCthmBBCqzBPxziz8we6m9sKb%2BOqd%2B%2Fur9rGAcFimjTRJlzOdY3QU7%2BtSMPY8td9QOUUEa5nvoJNCH9py9tSqoCMVWFvZaUVXl1ps0oWIYjvlfmLp3mvXw4XTKWorm39wQQt%2Bt60OaRl6sZwEf%2F3c3utRv0eZDdIcDBJRMNltPqr97PpiSzI2%2Bfw4vPvc3YjRes2JQ%2FqFQK3KZ36vIq%2F8MNeo8BAXR9bPZ9wV1IkOTPTnQ3RHdEOBEQ4EHNNrRzpSHr4bXY03ftx7URoDhR0Mx0WVvuGZ%2FcuA05ADtn7UdYvhxNpEA5MtVRbS47fr%2BM9KCX7btaGteWhkUAXcKmBwHGAC0X%2
I/flutter (13131): 1-2021-02-07 15:33:51.416187 -- MqttConnectionBase::_onError - calling disconnected callback

If I misspell e.g. things in the URL, and write thinngs instead, the error still stays the same. If I instead try to connect to /mqtt, instead of the device shadow, the error changes:

NoConnectionException (mqtt-client::NoConnectionException: The maximum allowed connection attempts ({3}) were exceeded. The broker is not responding to the connection request message (Missing Connection Acknowledgement?)

and my logs look like:

I/flutter (13131): 8-2021-02-07 16:05:42.305473 -- MqttConnectionHandlerBase::connect - server wss://<hostname>.iot.eu-north-1.amazonaws.com/mqtt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIARBRC4UMDAMPDGZWJ%2F20210207%2Feu-north-1%2Fiotdevicegateway%2Faws4_request&X-Amz-Date=20210207T150542Z&X-Amz-Expire=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=c1beeed92c9aa6d85922ce9c1c2ea4c196b93020593cb2d62cead1cad1d080e6&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGcaDGV1LWNlbnRyYWwtMSJIMEYCIQCL2XjhoH0mtKAyjE0e3CLXuOpcgZ%2BrBpiFpbdjmmOqUQIhALcYMqqecYBB0ejBvlaKljaBlVY5Pyf3hw8nu7Ruah%2FxKtYECGAQABoMMDcyMDEzODgyMTE4Igyzgrz4HpV%2FBcZ9EmoqswQsWza7Rg%2BlKwfRVnnod4KGXJWKbgXxjJ2lzySvzgGbHHDWqhHqD%2BMIyxHg5wVFp559k8JJ%2FHd2YyTNxpBb624BYtbcywGYiSt4BHSO753iXnLFB9Z0wSHDxIdnkvd%2FLkm%2BVRlYqpPPoKXb8oEFpVrNkhFKBAfFfUSKA82URGaR2DwdQJq7UeBwv4gbXk7b4yrKs%2B2ZXxcZWM7wSQAQ%2FS6tgn3YyF5KkIEI8Aq3yV531I1Tw0UvZ%2FsUcpOvWWzQcIInc7BWCtbTu16J0EexyVX2fxpHm7cCpNxh5wDWirNTel7RhaKhRZfoGKwROfORt%2BEvMJcSFGtoL7AKI%2BzWjo4amvoZaDP9e4hUE%2F8TwzGMvY5NxPAHGx
I/flutter (13131): 8-2021-02-07 16:05:42.305650 -- SynchronousMqttServerConnectionHandler::internalConnect entered
I/flutter (13131): 8-2021-02-07 16:05:42.305688 -- SynchronousMqttServerConnectionHandler::internalConnect - initiating connection try 0, auto reconnect in progress false
I/flutter (13131): 8-2021-02-07 16:05:42.305722 -- SynchronousMqttServerConnectionHandler::internalConnect - websocket selected
I/flutter (13131): 8-2021-02-07 16:05:42.305757 -- SynchronousMqttServerConnectionHandler::internalConnect - calling connect
I/flutter (13131): 8-2021-02-07 16:05:42.305790 -- MqttWsConnection::connect - entered
I/flutter (13131): 8-2021-02-07 16:05:42.306016 -- MqttWsConnection::connect - WS URL is wss://<hostname>.iot.eu-north-1.amazonaws.com:443/mqtt?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIARBRC4UMDAMPDGZWJ%2F20210207%2Feu-north-1%2Fiotdevicegateway%2Faws4_request&X-Amz-Date=20210207T150542Z&X-Amz-Expire=86400&X-Amz-SignedHeaders=host&X-Amz-Signature=c1beeed92c9aa6d85922ce9c1c2ea4c196b93020593cb2d62cead1cad1d080e6&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEGcaDGV1LWNlbnRyYWwtMSJIMEYCIQCL2XjhoH0mtKAyjE0e3CLXuOpcgZ%2BrBpiFpbdjmmOqUQIhALcYMqqecYBB0ejBvlaKljaBlVY5Pyf3hw8nu7Ruah%2FxKtYECGAQABoMMDcyMDEzODgyMTE4Igyzgrz4HpV%2FBcZ9EmoqswQsWza7Rg%2BlKwfRVnnod4KGXJWKbgXxjJ2lzySvzgGbHHDWqhHqD%2BMIyxHg5wVFp559k8JJ%2FHd2YyTNxpBb624BYtbcywGYiSt4BHSO753iXnLFB9Z0wSHDxIdnkvd%2FLkm%2BVRlYqpPPoKXb8oEFpVrNkhFKBAfFfUSKA82URGaR2DwdQJq7UeBwv4gbXk7b4yrKs%2B2ZXxcZWM7wSQAQ%2FS6tgn3YyF5KkIEI8Aq3yV531I1Tw0UvZ%2FsUcpOvWWzQcIInc7BWCtbTu16J0EexyVX2fxpHm7cCpNxh5wDWirNTel7RhaKhRZfoGKwROfORt%2BEvMJcSFGtoL7AKI%2BzWjo4amvoZaDP9e4hUE%2F8TwzGMvY5NxPAHGxur
I/flutter (13131): 8-2021-02-07 16:05:42.572953 -- MqttServerConnection::_startListening
I/flutter (13131): 8-2021-02-07 16:05:42.573182 -- SynchronousMqttServerConnectionHandler::internalConnect - connection complete
I/flutter (13131): 8-2021-02-07 16:05:42.573222 -- SynchronousMqttServerConnectionHandler::internalConnect sending connect message
I/flutter (13131): 8-2021-02-07 16:05:42.573291 -- MqttConnectionHandlerBase::sendMessage - MQTTMessage of type MqttMessageType.connect
I/flutter (13131): Header: MessageType = MqttMessageType.connect, Duplicate = false, Retain = false, Qos = MqttQos.atMostOnce, Size = 0
I/flutter (13131): Connect Variable Header: ProtocolName=MQIsdp, ProtocolVersion=3, ConnectFlags=Connect Flags: Reserved1=false, CleanStart=false, WillFlag=false, WillQos=MqttQos.atMostOnce, WillRetain=false, PasswordFlag=false, UserNameFlag=false, KeepAlive=300
I/flutter (13131): MqttConnectPayload - client identifier is : greencontrol-1
I/flutter (13131): 8-2021-02-07 16:05:42.573918 -- SynchronousMqttServerConnectionHandler::internalConnect - pre sleep, state = Connection status is connecting with return code of noneSpecified and a disconnection origin of none
I/flutter (13131): 8-2021-02-07 16:05:42.643286 -- MqttWsConnection::::onDone - calling disconnected callback

I'm not completely sure where the problem lies, and I believe it's more likely that it the issue is me doing something wrong, rather than an issue with mqtt_client. But maybe you could point me in the right direction or help me interpret the logs.

Have a good day!

[kd-s-t]

@felixeriksson hey dude, check this issue, it might be helpful https://github.com/shamblett/mqtt_client/issues/196

[shamblett]

This looks more like a websocket header problem, have a look at the websocket protocol API in the client -

List<String> websocketProtocolString;

  /// User definable websocket protocols. Use this for non default websocket
  /// protocols only if your broker needs this. There are two defaults in
  /// MqttWsConnection class, the multiple protocol is the default. Some brokers
  /// will not accept a list and only expect a single protocol identifier,
  /// in this case use the single protocol default. You can supply your own
  /// list, or to disable this entirely set the protocols to an
  /// empty list , i.e [].
  set websocketProtocols(List<String> protocols) {
    websocketProtocolString = protocols;
    if (connectionHandler != null) {
      connectionHandler.websocketProtocols = protocols;
    }
  }

[felixeriksson]

@ollolollollooloo Thanks, I looked into that. Regarding the port, it is my understanding that I should indeed use 443 for the type of auth I'm using according to the AWS IoT docs (using Cognito credentials to sign the URL with Signature Version 4). I tried 8883 but it gives me Connection terminated during handshake. I also tried changing my MqttConnectMessage to include .startClean() and .withWillQos(MqttQos.atLeastOnce) (they were the differences I found that I figured could be applicable to my case), as in the linked issue, but it didn't help.

@shamblett Thanks for the suggestion! I tried setting

final List<String> protocols = <String>['wss'];
_client.websocketProtocols = protocols;

which didn't change the error produced. I also tried with an empty List. I think I'm not completely clear on how this works - my error message states that the connection to https://<my query> wasn't upgraded to websocket. Is it so that the MQTT client first makes an HTTP request, and then switches protocols to WSS if the host answers happily, and everything works as intended?

[shamblett]

Try it with -

final List<String> protocols = <String>['mqtt'];

The client just uses websockets when instructed to do so, it then expects the url to start with 'ws' or 'wss' what happens next is controlled by the Dart runtime you are using, not the package, the package does not perform the handshake as such. You do however have to set your port correctly.

I can't really help you with AWS other than to say the user who raised #196 did get it working.

[felixeriksson]

I tried it, but still no luck. I noticed that my logs say MqttWsConnection::connect - WS URL is wss://<my endpoint>[...] - shouldn't it be MqttWs2Connection handling a WSS connection or am I misinterpreting it?

The people in #196 are using certificates. Maybe I'll try to get it working that way first for troubleshooting, but I need cognito auth in the end. I also found #134 and #135, they are using Cognito credentials, but neither has reported that they were successful. Also #32 is related but outdated, it instantiates MqttClient().

[shamblett]

MqttWs2Connection is an alternative ws/wss handler that is in fact deprecated now and should be removed, you should just be setting use websocket true which you are doing.

[Obaidiaa]

I faced this problem. The problem was an authentication failure from AWS IoT. You must attach an IoT policy (exactly like the ones that are attached to your devices) to the Cognito identity using the AttachPrincipalPolicy API.

https://stackoverflow.com/questions/40301345/connect-to-aws-iot-using-web-socket-with-cognito-authenticated-users/40305905

[felixeriksson]

I faced this problem. The problem was an authentication failure from AWS IoT. You must attach an IoT policy (exactly like the ones that are attached to your devices) to the Cognito identity using the AttachPrincipalPolicy API.

https://stackoverflow.com/questions/40301345/connect-to-aws-iot-using-web-socket-with-cognito-authenticated-users/40305905

Thanks for your answer! I believe you're right, and I hadn't done this. I tried doing this from the AWS CLI, but got informed that AttachPrincipalPolicy was deprecated, and I should use AttachPolicy instead. So I ran aws iot attach-policy --policy-name myPolicyName --target eu-north-1:xxxxxxxxxxxx (the last part being my Cognito identity ID) from the AWS CLI, and the command seems to have executed successfully - however, I still get the Connection not upgraded to websocket error when I try to run the app.

[Obaidiaa]

This main code:

  _fetchSession() async {
    try {
      CognitoAuthSession res = await Amplify.Auth.fetchAuthSession(
          options: CognitoSessionOptions(getAWSCredentials: true));
      return res.credentials;
    } on AuthException catch (e) {
      print(e.message);
    }
  }

void _mqttConnect() async {
    AWSCredentials awsCredentials = await _fetchSession();
    String accessKeyId = awsCredentials.awsAccessKey;
    String secretKey = awsCredentials.awsSecretKey;
    String sessionToken = awsCredentials.sessionToken;

    const region = 'us-east-2';
   // host found of AWS IoT Home page > Settings > Endpoint
    const host = '############-ats.iot.us-east-2.amazonaws.com';

    //This is the ID of the AWS IoT device
    const deviceId = 'Mobile_00000';

    device = AWSIoTDevice(region, accessKeyId, secretKey, sessionToken, host);

    try {
      await device.connect(deviceId);
    } on Exception catch (_) {
      print('Failed to connect, status is ${device.connectionStatus}');
    }
}

This pubspec.yaml

  mqtt_client: ^8.2.0
  amazon_cognito_identity_dart_2: ^0.1.25+1
  tuple: ^1.0.3

this is the code that I using to connect.

import 'dart:async';
import 'package:mqtt_client/mqtt_client.dart';
import 'package:mqtt_client/mqtt_server_client.dart';
import 'package:amazon_cognito_identity_dart_2/sig_v4.dart';
import 'package:tuple/tuple.dart';

class AWSIoTDevice {
  final _SERVICE_NAME = 'iotdevicegateway';
  final _AWS4_REQUEST = 'aws4_request';
  final _AWS4_HMAC_SHA256 = 'AWS4-HMAC-SHA256';
  final _SCHEME = 'wss://';

  String _region;
  String _accessKeyId;
  String _secretAccessKey;
  String _sessionToken;
  String _host;
  bool _logging;

  var _onConnected;
  var _onDisconnected;
  var _onSubscribed;
  var _onSubscribeFail;
  var _onUnsubscribed;

  get onConnected => _onConnected;
  set onConnected(val) => _client?.onConnected = _onConnected = val;
  get onDisconnected => _onDisconnected;
  set onDisconnected(val) => _client?.onDisconnected = _onDisconnected = val;
  get onSubscribed => _onSubscribed;
  set onSubscribed(val) => _client?.onSubscribed = _onSubscribed = val;
  get onSubscribeFail => _onSubscribeFail;
  set onSubscribeFail(val) => _client?.onSubscribeFail = _onSubscribeFail = val;
  get onUnsubscribed => _onUnsubscribed;
  set onUnsubscribed(val) => _client?.onUnsubscribed = _onUnsubscribed = val;
  get connectionStatus => _client?.connectionStatus;

  MqttServerClient _client;

  StreamController<Tuple2<String, String>> _messagesController =
      StreamController<Tuple2<String, String>>();

  Stream<Tuple2<String, String>> get messages => _messagesController.stream;

  AWSIoTDevice(
    this._region,
    this._accessKeyId,
    this._secretAccessKey,
    this._sessionToken,
    String host, {
    bool logging: false,
    var onConnected,
    var onDisconnected,
    var onSubscribed,
    var onSubscribeFail,
    var onUnsubscribed,
  }) {
    _logging = logging;
    _onConnected = onConnected;
    _onDisconnected = onDisconnected;
    _onSubscribed = onSubscribed;
    _onSubscribeFail = onSubscribeFail;
    _onUnsubscribed = onUnsubscribed;

    if (host.contains('amazonaws.com')) {
      _host = host.split('.').first;
    } else {
      _host = host;
    }
  }

  Future<Null> connect(String clientId) async {
    if (_client == null) {
      _prepare(clientId);
    }

    /// Check we are connected
    if (_client.connectionStatus.state == MqttConnectionState.connected) {
      print('EXAMPLE::Mosquitto client connected');
    } else {
      /// Use status here rather than state if you also want the broker return code.
      print(
          'EXAMPLE::ERROR Mosquitto client connection failed - disconnecting, status is ${_client.connectionStatus}');
      _client.disconnect();
      try {
        await _client.connect();
      } on Exception catch (e) {
        _client.disconnect();
        throw e;
      }
    }

    _client.updates.listen((List<MqttReceivedMessage<MqttMessage>> c) {
      for (MqttReceivedMessage<MqttMessage> message in c) {
        final MqttPublishMessage recMess = message.payload;
        final String pt =
            MqttPublishPayload.bytesToStringAsString(recMess.payload.message);
        _messagesController.add(Tuple2<String, String>(message.topic, pt));
      }
    });
  }

  _prepare(String clientId) {
    final url = _prepareWebSocketUrl();
    _client = MqttServerClient(url, clientId);
    _client.logging(on: _logging);
    _client.useWebSocket = true;
    _client.port = 443;
    _client.connectionMessage =
        MqttConnectMessage().withClientIdentifier(clientId).keepAliveFor(300);
    _client.keepAlivePeriod = 300;
  }

  _prepareWebSocketUrl() {
    final now = _generateDatetime();
    final hostname = _buildHostname();

    final List creds = [
      this._accessKeyId,
      _getDate(now),
      this._region,
      this._SERVICE_NAME,
      this._AWS4_REQUEST,
    ];

    const payload = '';

    const path = '/mqtt';

    final queryParams = Map<String, String>.from({
      'X-Amz-Algorithm': _AWS4_HMAC_SHA256,
      'X-Amz-Credential': creds.join('/'),
      'X-Amz-Date': now,
      'X-Amz-SignedHeaders': 'host',
      'X-Amz-Expire': '86400',
    });

    final canonicalQueryString = SigV4.buildCanonicalQueryString(queryParams);
    final request = SigV4.buildCanonicalRequest(
        'GET',
        path,
        queryParams,
        Map.from({
          'host': hostname,
        }),
        payload);

    final hashedCanonicalRequest = SigV4.hashCanonicalRequest(request);
    final stringToSign = SigV4.buildStringToSign(
        now,
        SigV4.buildCredentialScope(now, _region, _SERVICE_NAME),
        hashedCanonicalRequest);

    final signingKey = SigV4.calculateSigningKey(
        _secretAccessKey, now, _region, _SERVICE_NAME);

    final signature = SigV4.calculateSignature(signingKey, stringToSign);

    final finalParams =
        '$canonicalQueryString&X-Amz-Signature=$signature&X-Amz-Security-Token=${Uri.encodeComponent(_sessionToken)}';

    return '$_SCHEME$hostname$path?$finalParams';
  }

  String _generateDatetime() {
    return new DateTime.now()
        .toUtc()
        .toString()
        .replaceAll(new RegExp(r'\.\d*Z$'), 'Z')
        .replaceAll(new RegExp(r'[:-]|\.\d{3}'), '')
        .split(' ')
        .join('T');
  }

  String _getDate(String dateTime) {
    return dateTime.substring(0, 8);
  }

  String _buildHostname() {
    return '$_host.iot.$_region.amazonaws.com';
  }

  void disconnect() {
    return _client.disconnect();
  }

  Subscription subscribe(String topic,
      [MqttQos qosLevel = MqttQos.atMostOnce]) {
    /// Check we are connected
    if (_client.connectionStatus.state == MqttConnectionState.connected) {
      print('MQTT Connect now subscribe to topic');
      return _client?.subscribe(topic, qosLevel);
    } else {
      /// Use status here rather than state if you also want the broker return code.
      print(
          'EXAMPLE::ERROR Mosquitto client connection failed - disconnecting, status is ${_client.connectionStatus}');
      _client.disconnect();
      return null;
    }
  }

  void publishMessage(String topic, String payload) {
    final MqttClientPayloadBuilder builder = MqttClientPayloadBuilder();
    builder.addString(payload);
    _client.publishMessage(topic, MqttQos.atMostOnce, builder.payload);
  }
}

I hope it helps you.

[felixeriksson]

Thanks a lot Obaidiaa! It did help, now I'm getting messages across between AWS and Flutter. I'm not sure what problem I had when trying last time, as your code is very close to identical to what I used, I didn't see any obvious explanation when comparing the snippets. I haven't investigated it for very long though, as I'm excited to build stuff with this :)

[harishkthedeveloper]

Mosquitto client connection failed - disconnecting, status is Connection status is disconnected with return code of noneSpecified and a disconnection origin of none

[harishkthedeveloper]

Failed to connect, status is Connection status is disconnected with return code of noneSpecified and a disconnection origin of solicited

after following all the things all libray not able to connect to aws iot broker please help , thanks in advacne

[kd-s-t]

hi guys, check this out it might be helpful React Native, Raspberry Pi 4 and AWS IoT https://www.youtube.com/watch?v=QPBwH_MwWWA&list=PL8kT3V3swTBluh_HQ98YHrUacqZWCfmyU&index=4

[dinesh-parmar]

Hello @Obaidiaa By any chance if that code is working? I tried your code and I'm getting this error : "A value of type 'AuthSession*' can't be assigned to a variable of type 'CognitoAuthSession'."

Please Help me.

[kd-s-t]

Hello @Obaidiaa By any chance if that code is working? I tried your code and I'm getting this error : "A value of type 'AuthSession*' can't be assigned to a variable of type 'CognitoAuthSession'."

Please Help me.

@dinesh-parmar try checking this out, might help How to know if you configured correctly the AWS IoT? https://www.youtube.com/watch?v=p8Vb9l2dW6Q&t=0s

you need to configure your iam role

[dinesh-parmar]

Hello @Obaidiaa By any chance if that code is working? I tried your code and I'm getting this error : "A value of type 'AuthSession*' can't be assigned to a variable of type 'CognitoAuthSession'."

Please Help me.

@dinesh-parmar try checking this out, might help How to know if you configured correctly the AWS IoT? https://www.youtube.com/watch?v=p8Vb9l2dW6Q&t=0s

you need to configure your iam role

I watched this video. It's for React Native and I'm using Dart/Flutter. Although I created IAM role on my console. Gave the Administrative access. Got access key and Secret Key. I'm getting issues on "secretToken" and all these are part of Aws credentials and for that we are using Amplify.Auth.fetchAuthsession() but I'm getting the above error. Which is I guess from Dart side.

[kd-s-t]

@dinesh-parmar no, the video is about configuring AWS IoT, AWS IAM, AWS Cognito. It doesn't involved with react native.

Your issue sounds like you need to edit your Unauth JSON in IAM Role.

its at 6:18 mins https://youtu.be/p8Vb9l2dW6Q?t=378

[kd-s-t]

@dinesh-parmar clone this library and put all aws credentials and run it if you can communicate with aws iot testing https://github.com/ollolollollooloo/aws-iot-nodejs-tester

[MohammedNoureldin]

Hello @felixeriksson and @Obaidiaa,

where is this iotdevicegateway coming from? I could not find it anywhere in the docs.

[Obaidiaa]

In this page at AWS IoT Data Plane

https://docs.aws.amazon.com/iot/latest/apireference/Welcome.html

[MohammedNoureldin]

Thanks a lot @Obaidiaa! Actually I found that page after I wrote the comment, but I would say this is really a bad documentation for that. It is mentioned only here in a SINGLE minor place, while you need to make exactly the correct mixture between iot and iotgateway, this is very confusing. Your code helped me to get it working, thanks a lot again!

May I just ask if you implemented all of it by yourself or have you found any resource that shows when you should use iot and when iotgateway? Because though I understand your code, I couldn't figure it out by myself.

[Obaidiaa]

I have modified this file and use it in my app. Iotdevicegateway used to sign the request to aws. https://bitbucket.org/TheBosZ/aws_iot_device/src/master/lib/src/device.dart

[MohammedNoureldin]

@Obaidiaa thank you for posting that, I have also enhanced the implementation a bit, I will post mine later when I am quite happy with it (and after comparing it with you previous comment for any potential additional enhancements).

[amizer12]

Unfortunatelly any of the code pasted above works anymore since update to null safety. Can anyone share working example on connecting to AWS IoT with cognito user identities ? Thanks !!

[kd-s-t]

@amizer12 check this out https://www.youtube.com/watch?v=QPBwH_MwWWA&t=4s

[amizer12]

@ollolollollooloo Thanks for your reply but i dont know how this is related to the original topic - the link shows react native mobile app with certification authentication - where here im asking about a code in Flutter for Cognito based authentication

[kd-s-t]

@amizer12 if you watch it, there might be little things that relate to your problem, just saying, if ur instinct says nothing is there then sorry just trying to help