implement WSS

[mehotkhan]

it's possible to implement WSS IOTSA framework? with https cert/key?

• I need data to communicate with ESP and browser, on mDns domain
• COAP is a good choice but it seems not possible to use UDP in the browser ( there are no browser js clients )

[jackjansen]

Let me ask you for help:-)

I have (for a long time) been thinking about adding streaming interfaces (wss/websockets is an option, but also http push events, or raw TCP sockets) to iotsa, but I can't think of a way to design it in an elegant way.

The problem is that the whole design of iotsa is state-based, with the client getting or setting variables, so it fits the REST (or COAP, or BLE GATT) design very nicely. Callbacks (from iotsa to something else on the net) are somewhat handled by the IotsaRequest module, and I know how I want to do this better (when I have the time:-)

But streaming data from client to iotsa or from iotsa to another agent on the net is something that I don't know how to fit in the architecture, in a decent way.

How would you want to use WSS? With iotsa in control (for example transmitting new sensor readings whenever available) or with the other side in control? Or both? How would you envision integrating it in the existing architecture?

[mehotkhan]

my User App write in js and load with a browser (PWA app + Https + WASM) for now , I use WS to send sensor data -> ESP01 -> WIFI -> Browser -> PWA app

for security reason, WS blocked on https domain, my Dev code uploaded here :

ESP01-Signaling-Server JSON data read from serial, send to WS client

• my main purpose: data must stream from sensor to js app, as fast as possible
• data must be encrypted in some way (based on the protocol )
• js app Calculate data from the incoming signal , don't trigger any event, just listen and calculate and draw on Canvas
• I tested MQTT before on cloud service, MQTT need a broker, I find this, not tested : uMQTTBroker
• COAP, no UDP on the browser, find this code https://github.com/jdeblese/jscoap/wiki/API,7 years old and don't work I'm working on this js module, maybe in some way, I make it work
• on TCP API mode, I have no idea to create an observer object to get data whenever they come (like WS)
• your Framework has a lot potential for ESP01 and modularity philosophy, I think on COAP TCP mode ( your COAP API in TCP mode ? ), maybe I achieve that
• my final idea, for now, is in some way create WSS with your Framework or find a machine to machine protocol that works on Arduino + JS Browser
• my main language is not English, Grammarly help me, sorry for confusing text :)

[jackjansen]

Ok, that sounds like you want your "WebSocket channel" to be sort-of independent of the iotsa framework. So use iotsa for control, for example, and use a streaming connection for the actual data transfer between the iotsa device and the JS app.

And you said that you've looked at MQTT and that isn't a viable option, because MQTT needs a broker.

I think that that also rules out using a WebSocket client on the iotsa device (because that would also require a WSS server somewhere on the net, that would forward the data from your iotsa device to your JavaScript app (which would then also be a WSS client). Correct?

Problem is that I'm not aware of any Websocket server implementation for the esp8266 (I know there are some WebSocket client libraries, but I haven't investigated them).

I do know a WebSocket server for the esp32, however: https://github.com/fhessel/esp32_https_server. And: that implementation is already used by iotsa for the https implementation on the esp32, so it may be fairly easy to integrate. And you'll get things like https encryption automatically. But: this library doesn't work on esp8266, and fhessel said that it is pretty impossible. So this would only be a solution if you can make the switch to esp32.

[jackjansen]

Completely different idea just struck me: what is the amount of data you need to stream? I.e. how many bytes/kilobytes per second?

Because a completely different solution could be (if this is practical for the amount of data you want to transfer) that the sensor reader code on your iotsa puts the data into a buffer (which would grow over time) and then your JS app uses a iotsa REST GET request to get the content of the buffer. The buffer would then be cleared after the GET, and slowly start filling again until the next GET.

[mehotkhan]

I don't like to change the problem parameter, so my main Tiny server is ESP01 :))

• WS on Arduino is easy, https://github.com/Links2004/arduinoWebSockets
• about your second comment, the data is not too large, Some FFT bin, but the rate in time, I think is 10/15 massage/second ( for dev version ) but your buffer idea is good, the main problem is, we neet and observer function to Get-listen, like a loop or some package signal or something other let's try some code, thanks for your idea

[jackjansen]

I had a look at https://github.com/Links2004/arduinoWebSockets and it looks very promising. I wasn't aware of it, thanks!

I think you can drop this right in: just create a module that doesn't do much in the way of REST API or HTML api, and handle the websocket in the setup, serverSetup and loop methods of that module. You can look at iotsaOta.cpp for an example of such a module. The only thing is that you cannot use port 80 (or 443 for https) for your websocket server because that port is already taken by the iotsa webserver. If you do want to use 80/443 then you would need to add some glue code to iotsaWebServer.cpp to send to WSS-related connections to arduinoWebSockets based on the URL.

[mehotkhan]

thanks for your tips , WSS not so complicated and i think it can implement with iotsa , i work on connect iotsa with arduinoWebSockets

[mehotkhan]

Look this codes : https://github.com/me-no-dev/AsyncTCP https://github.com/me-no-dev/ESPAsyncWebServer

[jackjansen]

I have stayed away from async services for iotsa, because I want to keep the Arduino paradigm of setup() and loop() as the basic structure. And if you start using async services you have to either be very careful about what you do in the async routines in what order, or use locking on the boundaries between the async code and the loop-based code. And locking is potentially dangerous if you are not 100% sure about how the underlying async library is structured (because if it is interrupt-driven, and you're holding a lock in your loop() routine, and then the interrupt comes in and calls your async routine, which then also tries to lock: you're deadlocked).

But, all that said, I am using esp32_https_server which is asynchronous, and I'm reasonably sure I got it right without locking. I hope:-)

[jackjansen]

By the way: if you implement the WSS in a way that may be reusable, and if you're willing to share it (iotsa is MIT license) I'd love to add it to iotsa.

[jackjansen]

@mehotkhan did you make any progress on WSS? As said: I'd love to incorporate the functionality in iotsa, and I'm working on a new release at the moment, so if you have anything you'd be willing to share: right now I could use sample code (even if incomplete) if you're willing to share...

[mehotkhan]

hi, jack, I change the main plan to : create Secure custom Event source, Every path in un event source, so the webserver can send data on-alive read here :async-event-source-plugin

the main concept for using this method is: update server code something like

<?php
header('Content-Type: text/event-stream');
header('Cache-Control: no-cache');

$time = date('r');
echo "data: The server time is: {$time}\n\n";
flush();
?>

I read and about some code but I can't find a way to do this on iotsa and I lost my free time :))