Type error with mqtt_server_client, wss and Ably: message is String, treated as List<int>

[alexeyinkin]

Run this:

final uri = 'wss://invalid@mqtt.ably.io';
final client = MqttServerClient.withPort(uri, '', 8883);

client.useWebSocket = true;
client.logging(on: true);
client.keepAlivePeriod = 60;

await client.connect();

An exception is thrown:

E/flutter (27963): [ERROR:flutter/lib/ui/ui_dart_state.cc(209)] Unhandled Exception: type 'String' is not a subtype of type 'List<int>'
E/flutter (27963): #0      MqttServerConnection._onData (package:mqtt_client/src/connectionhandling/server/mqtt_client_mqtt_server_connection.dart:54:26)

The data there is something like

{"action":9,"error":{"message":"Invalid key in request: invalid. (See https://help.ably.io/error/40005 for help.)","code":40005,"statusCode":400,"href":"https://help.ably.io/error/40005"},"timestamp":1634628118747}

But the code expects it to be a List<int>.

The same thing happens with a valid key. The message is again a String of this kind:

{"action":4,"connectionId":"cwimUxHpt7","connectionKey":"...","connectionSerial":-1,"connectionDetails":{"clientId":"*","connectionKey":"...","maxMessageSize":16384,"maxInboundRate":15,"maxOutboundRate":15,"maxFrameSize":262144,"serverId":"...","connectionStateTtl":120000,"maxIdleInterval":15000}}

[shamblett]

This -

{"action":4,"connectionId":"cwimUxHpt7","connectionKey":"...","connectionSerial":-1,"connectionDetails":{"clientId":"*","connectionKey":"...","maxMessageSize":16384,"maxInboundRate":15,"maxOutboundRate":15,"maxFrameSize":262144,"serverId":"...","connectionStateTtl":120000,"maxIdleInterval":15000}}

is not valid response as per the MQTT specification the broker should reply with correctly formatted MQTT messages, not json strings. The client won't process this, although whether we should raise an exception here could be questioned.