shamblett / mqtt_client

A server and browser based MQTT client for dart
Other
552 stars 179 forks source link

BrowserClient WSS Issues #487

Closed adojang closed 1 year ago

adojang commented 1 year ago

Hello! First off, thank you for making such an epic library!

I'm having some issues when trying to use the browserclient. The default example works for me. However, my server requires me to use WSS instead of just WS.

I'm able to access my WSS server using another MQTT client, but for some reason it doesn't work when I'm using dart.

Details to test from other MQTT client:

IP: 93.90.207.70 [Anonymous Access is Enabled] [Uses TLS Encryption from Cloudflare]

[x] -Try using that in any MQTT client and it'll work. [x] -It does not work when I run it in dart. [x] - have tried tons of different ports [x] - Double checked my host/provdier's firewall isn't blocking anything [x] - it works for plain WS on my server but not WSS (use port 8083 and change to 'ws://...' instead of 'wss://...',

The error I get is MqttBrowserWsConnection::connect - websocket has erred. Reading a few other posts, I thought it might be my mosquitto config:

listener 8091
protocol websockets
allow_anonymous true
password_file /etc/mosquitto/passwd
capath /etc/ssl/certs
certfile /etc/ssl/cloudflare_certificate.pem
keyfile /etc/ssl/cloudflare_private_key.pem

But I'm not sure if that is the case, because I AM able to access it user other MQTT clients as shown here (Using MQTT Explorer):

image

Any help or suggestions would be highly appreciated!

Here's my adapted file:

`import 'dart:async'; import 'package:mqtt_client/mqtt_client.dart'; import 'package:mqtt_client/mqtt_browser_client.dart';

//THIS WORKS

// final client = MqttBrowserClient('wss://test.mosquitto.org', ''); // final port = 8091; // final String usern = 'rw'; // final String passw = 'readwrite';

//THIS DOES NOT WORK. (BUT WORKS WHEN YOU TRY ON ANOTHER CLIENT) final client = MqttBrowserClient('wss://93.90.207.70', ''); final port = 8091; final String usern = ''; final String passw = '';

Future main() async { /// Set logging on if needed, defaults to off client.logging(on: true);

/// Set the correct MQTT protocol for mosquito // client.setProtocolV311();

/// If you intend to use a keep alive you must set it here otherwise keep alive will be disabled. client.keepAlivePeriod = 20;

/// The connection timeout period can be set if needed, the default is 5 seconds. client.connectTimeoutPeriod = 2000; // milliseconds

/// The ws port for Mosquitto is 8080, for wss it is 8081 client.port = port;

/// Add the unsolicited disconnection callback client.onDisconnected = onDisconnected;

/// Add the successful connection callback client.onConnected = onConnected;

/// Add a subscribed callback, there is also an unsubscribed callback if you need it. /// You can add these before connection or change them dynamically after connection if /// you wish. There is also an onSubscribeFail callback for failed subscriptions, these /// can fail either because you have tried to subscribe to an invalid topic or the broker /// rejects the subscribe request. client.onSubscribed = onSubscribed;

/// Set a ping received callback if needed, called whenever a ping response(pong) is received /// from the broker. client.pongCallback = pong;

/// Set the appropriate websocket headers for your connection/broker. /// Mosquito uses the single default header, other brokers may be fine with the /// default headers. // client.websocketProtocols = MqttClientConstants.protocolsSingleDefault;

/// Create a connection message to use or use the default one. The default one sets the /// client identifier, any supplied username/password and clean session, /// an example of a specific one below. final connMess = MqttConnectMessage().withClientIdentifier('Mqtt_MyClientUniqueId'); // .withWillTopic('willtopic') // If you set this you must set a will message // .withWillMessage('My Will message') // .startClean() // Non persistent session for testing // .withWillQos(MqttQos.atLeastOnce); print('EXAMPLE::Mosquitto client connecting....'); client.connectionMessage = connMess;

/// Connect the client, any errors here are communicated by raising of the appropriate exception. Note /// in some circumstances the broker will just disconnect us, see the spec about this, we however will /// never send malformed messages. try { await client.connect(usern, passw); } on Exception catch (e) { print('EXAMPLE::client exception - $e'); client.disconnect(); return -1; }

/// Check we are connected if (client.connectionStatus!.state == MqttConnectionState.connected) { print('EXAMPLE::Mosquitto client connected'); } else { /// Use status here rather than state if you also want the broker return code. print( 'EXAMPLE::ERROR Mosquitto client connection failed - disconnecting, status is ${client.connectionStatus}'); client.disconnect(); return -1; }

/// Ok, lets try a subscription print('EXAMPLE::Subscribing to the test/lol topic'); const topic = 'pelicans/lol'; // Not a wildcard topic client.subscribe(topic, MqttQos.atMostOnce);

/// The client has a change notifier object(see the Observable class) which we then listen to to get /// notifications of published updates to each subscribed topic. client.updates!.listen((List<MqttReceivedMessage<MqttMessage?>>? c) { final recMess = c![0].payload as MqttPublishMessage; final pt = MqttPublishPayload.bytesToStringAsString(recMess.payload.message);

/// The above may seem a little convoluted for users only interested in the
/// payload, some users however may be interested in the received publish message,
/// lets not constrain ourselves yet until the package has been in the wild
/// for a while.
/// The payload is a byte buffer, this will be specific to the topic
print(
    'EXAMPLE::Change notification:: topic is <${c[0].topic}>, payload is <-- $pt -->');
print('');

});

/// If needed you can listen for published messages that have completed the publishing /// handshake which is Qos dependant. Any message received on this stream has completed its /// publishing handshake with the broker. client.published!.listen((MqttPublishMessage message) { print( 'EXAMPLE::Published notification:: topic is ${message.variableHeader!.topicName}, with Qos ${message.header!.qos}'); });

/// Lets publish to our topic /// Use the payload builder rather than a raw buffer /// Our known topic to publish to const pubTopic = 'Dart/Mqtt_client/testtopic'; final builder = MqttClientPayloadBuilder(); builder.addString('Hello from mqtt_client');

/// Subscribe to it print('EXAMPLE::Subscribing to the Dart/Mqtt_client/testtopic topic'); client.subscribe(pubTopic, MqttQos.exactlyOnce);

/// Publish it print('EXAMPLE::Publishing our topic'); client.publishMessage(pubTopic, MqttQos.exactlyOnce, builder.payload!);

/// Ok, we will now sleep a while, in this gap you will see ping request/response /// messages being exchanged by the keep alive mechanism. print('EXAMPLE::Sleeping....'); await MqttUtilities.asyncSleep(60);

/// Finally, unsubscribe and exit gracefully print('EXAMPLE::Unsubscribing'); client.unsubscribe(topic);

/// Wait for the unsubscribe message from the broker if you wish. await MqttUtilities.asyncSleep(2); print('EXAMPLE::Disconnecting'); client.disconnect(); return 0; }

/// The subscribed callback void onSubscribed(String topic) { print('EXAMPLE::Subscription confirmed for topic $topic'); }

/// The unsolicited disconnect callback void onDisconnected() { print('EXAMPLE::OnDisconnected client callback - Client disconnection'); if (client.connectionStatus!.disconnectionOrigin == MqttDisconnectionOrigin.solicited) { print('EXAMPLE::OnDisconnected callback is solicited, this is correct'); } }

/// The successful connect callback void onConnected() { print( 'EXAMPLE::OnConnected client callback - Client connection was sucessful'); }

/// Pong callback void pong() { print('EXAMPLE::Ping response client callback invoked'); } `

shamblett commented 1 year ago

In the code that doesn't work you seem to be not using username and password i.e. setting them to '' and using them in the connect call -

await client.connect(usern, passw);

In the code that does work you are setting these to real values. if you pass a username and password to the connect method that are not null it will set them as authentication parameters in the connect message so the broker may be failing the connection on bad authentication. Either set these to the correct values or if you are not using them then just call connect -

await client.connect();

Its odd to see ws working and wss not working in either case the client just checks the connection URL and passes them straight to the Dart runtime. If the suggestion above doesn't work you will have to look at your broker logs to see why it is refusing the connection.

adojang commented 1 year ago

Ah thanks for catching that! I've tried with both the appropriate username/password I've set a testing credentials: username: octopus password: octopus

Here is my log file when using that username/password for WSS. These credentials are verified to work with other MQTT clients.

`Restarted application in 95ms. EXAMPLE::Mosquitto client connecting.... 1-2023-10-11 12:25:13.386 -- Authenticating with username '{octopus}' and password '{octopus}' 1-2023-10-11 12:25:13.387 -- MqttClient::connect - Connection timeout period is 2000 milliseconds 1-2023-10-11 12:25:13.387 -- MqttClient::connect - keep alive is enabled with a value of 20 seconds 1-2023-10-11 12:25:13.388 -- MqttConnectionKeepAlive:: Initialised with a keep alive value of 20 seconds 1-2023-10-11 12:25:13.388 -- MqttConnectionKeepAlive:: Disconnect on no ping response is disabled 1-2023-10-11 12:25:13.388 -- MqttConnectionHandlerBase::connect - server wss://93.90.207.70, port 8091 1-2023-10-11 12:25:13.388 -- SynchronousMqttBrowserConnectionHandler::internalConnect entered 1-2023-10-11 12:25:13.388 -- SynchronousMqttBrowserConnectionHandler::internalConnect - initiating connection try 0, auto reconnect in progress false 1-2023-10-11 12:25:13.388 -- SynchronousMqttBrowserConnectionHandler::internalConnect - calling connect 1-2023-10-11 12:25:13.388 -- MqttBrowserWsConnection::connect - entered 1-2023-10-11 12:25:13.388 -- MqttBrowserWsConnection::connect - WS URL is wss://93.90.207.70:8091 1-2023-10-11 12:25:13.389 -- MqttBrowserWsConnection::connect - connection is waiting 1-2023-10-11 12:25:14.022 -- MqttBrowserWsConnection::connect - websocket has erred 1-2023-10-11 12:25:14.023 -- SynchronousMqttBrowserConnectionHandler::internalConnect - connection complete 1-2023-10-11 12:25:14.023 -- SynchronousMqttBrowserConnectionHandler::internalConnect sending connect message 1-2023-10-11 12:25:14.023 -- MqttConnectionHandlerBase::sendMessage - MQTTMessage of type MqttMessageType.connect Header: MessageType = MqttMessageType.connect, Duplicate = false, Retain = false, Qos = MqttQos.atMostOnce, Size = 0 Connect Variable Header: ProtocolName=MQIsdp, ProtocolVersion=3, ConnectFlags=Connect Flags: Reserved1=false, CleanStart=false, WillFlag=false, WillQos=MqttQos.atMostOnce, WillRetain=false, PasswordFlag=true, UserNameFlag=true, KeepAlive=20 MqttConnectPayload - client identifier is : someID

1-2023-10-11 12:25:14.023 -- SynchronousMqttBrowserConnectionHandler::internalConnect - pre sleep, state = Connection status is connecting with return code of noneSpecified and a disconnection origin of none 1-2023-10-11 12:25:16.037 -- SynchronousMqttBrowserConnectionHandler::internalConnect - post sleep, state = Connection status is connecting with return code of noneSpecified and a disconnection origin of none 1-2023-10-11 12:25:16.037 -- SynchronousMqttBrowserConnectionHandler::internalConnect - initiating connection try 1, auto reconnect in progress false 1-2023-10-11 12:25:16.038 -- SynchronousMqttBrowserConnectionHandler::internalConnect - calling connect 1-2023-10-11 12:25:16.038 -- MqttBrowserWsConnection::connect - entered 1-2023-10-11 12:25:16.038 -- MqttBrowserWsConnection::connect - WS URL is wss://93.90.207.70:8091 1-2023-10-11 12:25:16.038 -- MqttBrowserWsConnection::connect - connection is waiting 1-2023-10-11 12:25:16.514 -- MqttBrowserWsConnection::connect - websocket has erred 1-2023-10-11 12:25:16.514 -- SynchronousMqttBrowserConnectionHandler::internalConnect - connection complete 1-2023-10-11 12:25:16.514 -- SynchronousMqttBrowserConnectionHandler::internalConnect sending connect message 1-2023-10-11 12:25:16.514 -- MqttConnectionHandlerBase::sendMessage - MQTTMessage of type MqttMessageType.connect Header: MessageType = MqttMessageType.connect, Duplicate = false, Retain = false, Qos = MqttQos.atMostOnce, Size = 38 Connect Variable Header: ProtocolName=MQIsdp, ProtocolVersion=3, ConnectFlags=Connect Flags: Reserved1=false, CleanStart=false, WillFlag=false, WillQos=MqttQos.atMostOnce, WillRetain=false, PasswordFlag=true, UserNameFlag=true, KeepAlive=20 MqttConnectPayload - client identifier is : someID

1-2023-10-11 12:25:16.514 -- SynchronousMqttBrowserConnectionHandler::internalConnect - pre sleep, state = Connection status is connecting with return code of noneSpecified and a disconnection origin of none 1-2023-10-11 12:25:18.519 -- SynchronousMqttBrowserConnectionHandler::internalConnect - post sleep, state = Connection status is connecting with return code of noneSpecified and a disconnection origin of none 1-2023-10-11 12:25:18.519 -- SynchronousMqttBrowserConnectionHandler::internalConnect - initiating connection try 2, auto reconnect in progress false 1-2023-10-11 12:25:18.519 -- SynchronousMqttBrowserConnectionHandler::internalConnect - calling connect 1-2023-10-11 12:25:18.520 -- MqttBrowserWsConnection::connect - entered 1-2023-10-11 12:25:18.520 -- MqttBrowserWsConnection::connect - WS URL is wss://93.90.207.70:8091 1-2023-10-11 12:25:18.520 -- MqttBrowserWsConnection::connect - connection is waiting 1-2023-10-11 12:25:19.195 -- MqttBrowserWsConnection::connect - websocket has erred 1-2023-10-11 12:25:19.195 -- SynchronousMqttBrowserConnectionHandler::internalConnect - connection complete 1-2023-10-11 12:25:19.195 -- SynchronousMqttBrowserConnectionHandler::internalConnect sending connect message 1-2023-10-11 12:25:19.195 -- MqttConnectionHandlerBase::sendMessage - MQTTMessage of type MqttMessageType.connect Header: MessageType = MqttMessageType.connect, Duplicate = false, Retain = false, Qos = MqttQos.atMostOnce, Size = 38 Connect Variable Header: ProtocolName=MQIsdp, ProtocolVersion=3, ConnectFlags=Connect Flags: Reserved1=false, CleanStart=false, WillFlag=false, WillQos=MqttQos.atMostOnce, WillRetain=false, PasswordFlag=true, UserNameFlag=true, KeepAlive=20 MqttConnectPayload - client identifier is : someID

1-2023-10-11 12:25:19.195 -- SynchronousMqttBrowserConnectionHandler::internalConnect - pre sleep, state = Connection status is connecting with return code of noneSpecified and a disconnection origin of none `

shamblett commented 1 year ago

You have your setProtocol line commented out by the looks of it, try uncommenting this i.e. set the protocol to v311.

adojang commented 1 year ago

Tried that but still no luck.

After some debugging with GPT, I've got the feeling it might have something to do with the SSL/TLS authentication. I don't have any local certificate on my client, but some reading suggests I might need one?

Would appreciate any further insight!

adojang commented 1 year ago

Digging around reveals that this is executed - where that 'websocket has erred' comes from:

  errorEvents = client.onError.listen((e) {
    MqttLogger.log(
        'MqttBrowserWsConnection::connect - websocket has erred');
    openEvents?.cancel();
    closeEvents?.cancel();
    errorEvents?.cancel();
    return completer.complete(MqttClientConnectionStatus());
  });

  Inside the mqtt_client_mqtt_browser_ws_connection.dart file.

  How can I make more sense of this?
adojang commented 1 year ago

I have completely delete my previous mosquitto MQTT broker and now I'm using EMQX instead. The issue persists. I'm unable to connect using WSS.

I've tried every combination of enabling/disabling client.setProtocol31() and 311() I've tried variations on

client.websocketProtocols = MqttClientConstants.protocolsSingleDefault; client.websocketProtocols = ['mqtt'];

Alas, nothing seems to be working.

When using the same adderss/port on MQTTX it works 100%. So weird.

shamblett commented 1 year ago

Ok, the only recourse you have is to look at your broker logs and find out why the broker is failing the connection everything else is guess work.

AFAIK you do not need a cert for wss working only for secure sockets on the server side.

The code is the same for ws or wss see the Dart docs for websockets, only the URL is different and as you say ws works.

If it is TLS error of some kind then its in the Dart runtime not the client. Other clients dont usevthe Dart runtime so its immaterial what they do.

There are many users using wss in the browser with no problems please look at your broker logs.

adojang commented 1 year ago

Wow, what a JOURNEY.

After switching from Mosquitto to EMQX, several snapshot and reinstalling my entire server I've got it working.

Solution for anyone else with this issue:

Use Certbot Easy way to install: sudo apt install certbot python3-certbot-nginx

Next, follow along this tutorial to generate the certificates for yourself.

https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-20-04

If you get stuck or something breaks, its probably nginx or something else using the ports. Turn that off first before trying to generate a certificate using sudo systemctl stop nginx or whatever you're using.

Finally, install mosquitto and secure it using these steps: https://www.digitalocean.com/community/tutorials/how-to-use-certbot-standalone-mode-to-retrieve-let-s-encrypt-ssl-certificates-on-ubuntu-1804

Important Sidenote: I'm using cloudflare for security and proteciton.

In order for everything to work properly, I had to make a subdomain that is used exclusively for mqtt. Cloudflare should not be used to proxy this. Test that your domain mqtt.mywebsite.com resolves to the IP address of your server cluster.

Thanks for the great library again @shamblett .

shamblett commented 1 year ago

Great, thanks for posting the solution, I'll bear this in mind if any other users have wss issues.

ferrantejake commented 1 year ago

Though this is closed, I'd like to add to why the Let's Encrypt certificates work for future folks (and why people might be having trouble connecting via WSS).

I was unable to get secure websockets working against my mosquitto instance that was secured with self-signed certificates. Well, after some digging I found out this is because the self-signed certificates are not trusted by browsers (unless you manually add your cert, but that's tough in prod) and this appears to be why I'm seeing the MqttBrowserWsConnection::connect - websocket has erred issue. Switching to a Let's Encrypt cert solved this issue.

As of writing this comment there currently is no way to trust self-signed certificates or bypass the verification step of the certificate in flutter web. Trusting a self-signed certificate (or bypassing the verification step) is only something that can be done using dart:io which is not available to flutter web.

So, what makes the Let's Encrypt certs work? The chain of trust. Let's Encrypt is trusted by major browsers. When your browser tries to connect to your broker, it sees the Let's Encrypt cert and allows the connection.

Hope that helps! Get a cert signed by a trusted CA!