WAN messaging with intermittent connections

[rob42]

The signalk protocol works well in a local LAN, and using TCP is robust enough for practical purposes. This also applies to internet connections, assuming both sender and receiver have a reliable internet connection. Even small(1-5 secs) outages are not critical as the clients and servers self-heal via TCP and retries etc.

But when the client comes and goes with long gaps (minutes or hours) the TCP connection will time-out, and the client must either retry forever (inefficient), or eventually give up. So we have a use case where one end sends, and reception may happen at some much delayed time at the receiver. In fact both sender and receiver may never be online at the same time.

Lets refer to this as an async connection. Its a well worn path for the internet, email works like this, and many social apps. The signalk protocol works nicely across many suitable transport systems, but we cant support all of them, and selecting a single protocol would simplify things at this stage.

XMPP seems popular, other ideas welcome.

[tkurki]

Another way to look at the data produced onboard is as time series data. You want to replicate the complete time series (which may or may not have gaps in it) over an intermittently available communication channel.

So far my solution has been logging everything locally in raw format (NMEA0183, canboat ascii output, SK delta for custom providers) and rsyncing those to S3. Raw format so that in case SK changes I can reconvert everything.

[rob42]

I log timestamp, key,value for all changes so I can replay based on last timestamp, but it doesnt work for passive subs unless they send acks and I track the timestamp. What I want is to fire + forget the delta, and have a local or remote message layer, eg Xmpp, handle delivery. Then it will be delivered eventually, in spite of me being offline again.

[rob42]

I am thinking specifically about remote connections not the localboat network

[tkurki]

The optimal policy in the face of intermittent connectivity is really dependent on what you are trying to achieve / the needs of the remote service:

• "I only care about the most recent values" => keep the latest values in the buffer and throw away everything else. Once connectivity is reestablished send the latest values, even if they are a bit old
• "I only care about data not more than x seconds old"=> don't send old values once reconnected
• "I want complete time series, but only in 1 minute resolution" => summarise the data in outgoing growing buffer, with min/max/avg/median, and once reconnected start sending oldest first
• "I want complete time series, all data, but I value newer values over old ones" => start sending newest update when reconnected, using leftover bandwidth for catching up

At this stage I would keep this issue open and wait for practical needs to drive the solution instead of overengineering. Once we have a few real world applications working we should see what has come up and see what should be included.

Don't want to silence a discussion, I just feel that there are other more fruitful avenues to pursue.

[tkurki]

I have lately experimented with a local InfluxDB/Grafana instance on the onboard Pi. Serves me real well, for example electrical power monitoring is a lot easier when you see the charge/discharge graphs instead of periodically eyeballing the gauges.

However I consider the onboard data disposable (or eventually it will get destroyed, one way or another). To have all the historical data available I've been planning a processing chain where

• data is written to hourly files
• files are synced to S3 (automation TBD, oneliner with amazon cli)
• file upload triggers an import job to InfluxDB in the cloud (TBD)

I think this demonstrates one WAN messaging scenario leveraging off the shelf infrastructure. As InfluxDB is accessed via http this approach can be used with other, for example SaaS data logging services.

[tkurki]

What would be the simplest store-and-forward-when-connectivity exists solution from a local server to a cloud server, forgetting the policies I mentioned above?

[rob42]

Its not really about storing my data as a backup/cloud copy.

My use case is sharing signalk data with a group of others, and sharing data with external services like weather, trip reports etc. Kind of like a chat app.

In my home ground I have intermittent connection to the internet as I pass in and out of range of localised WIFI and cellphone coverage.When I have coverage I want to collect all the deltas Ive missed from them, and push the ones I have buffered. I dont expect the service to store them long term, just until they have been delivered once to every-one (or expire). Then when I leave the marina I know where my mates are, the current weather nowcast beacons update, etc

Chat protocols like XMPP do this nicely and have libs for most environments. Its relatively easy to create a chat group, add users, and share text data (deltas). They also have user discovery mechanisms, and do distributed server hopping.

[rob42]

In fact we could do this over a slack channel, except that the volume of messages would cause our 10K message limit to eat all our real history :-(

[rob42]

I setup an XMPP server yesterday (Prosody), very easy. Then I was able to create and join a chat room, and send my signalk stream to it from my java server. I also joined with a normal client (pidgin) and could watch the messages stream past! Today I will try setting up multiple servers sharing signalk this way. In fact setting up Prosody was so easy Im tempted to have one on my RPi and let it deal with server2server xmpp. Just dont mispell your domain name and waste 3 hours debugging your code to try and fix it :-(

[rob42]

Ive completed the setup, and it works nicely :-) Its seems able to deal with connect/disconnect/reconnect ok, and offers a lot of additional functionality like presence and persistant history etc although Ive not tried much of that. Going to expose my xmpp server outside my firewall, and try that too.