add support for UDP, fix a few things in README

[astuder]

This pull request extends sentences_from_source() to receive AIS data over UDP. Input from UDP is activated when source matches the pattern ip-address:port.

I also fixed the example in the README, which wasn't working. And also updated the link to the AIVDM documentation as it moved a few years ago.

[wpietri]

Thanks! This is interesting. Could you say more about the use case? Normally when I set up network things with this I'll use a separate recorder that listens and then logs the received sentences. Then I'll use the AIS parsing tools on the log. But it sounds like you're up to something different here.

Also, what's the use case for listening to a specific IP? It seems like the most common case would be to listen on all addresses. If so, we should probably make that easier. And as an aside, an IP address ending in .255 is unusual, but still legitimate.

And is it worth supporting TCP as long as we're at it?

Or depending on use case, another option here is suggesting people use something like nc -l -u 12345 | aist, which might be more Unixy.

[astuder]

I'm a Windows user, so "unixy" things don't tend to work very well, or at all :)

My main use case is writing simple Python scripts that do something with live data. A bit like the simpleais command line tools, but more specialized one-offs. For example forwarding AIS data in a bespoke format, or logging it into a database.

UDP, including UDP broadcast is the way most WiFi enabled AIS receivers transmit their data. It's also the preferred protocol for AIS aggregators. Therefore UDP enables easy integration with commonly used tools like AIS dispatcher, AIS-catcher, Signal K, OpenCPN, etc. which all support UDP output. For example connect the AIS receiver over serial to one of these apps for reporting to aggregators, or local navigation/chart display. Then configure an output to relay AIS data via UDP to other (local) AIS consumers.

The IP address was for the case that AIS traffic should only be sent to a specific address instead of broadcasted on the network. But TIL, that the port alone seems sufficient no matter what target IP the UDP packet was sent to. If that's always the case, then I agree that IP address doesn't make sense for the UDP source.

TCP could be added for completeness. With a TCP client, one could tap into AIS firehouses like for example Norway's. I think TCP server is lower priority, though it may be helpful when UDP is not possible/desirable for reasons.

[astuder]

Added three more commits:

• changed pattern for UDP to look for :port, for example aist :8000
• added TCP client which will be invoked for ip:port , for example 127.0.0.1:8000
• change pattern for TCP client to also work for name:port, for example myais.local:8000

Unfortunately, the Norway's AIS stream doesn't work with simpleais due to the message format looking like this:

\s:2573145,c:1711995681*03\!BSVDM,1,1,,A,13m6Jo00010LaPTQHI8=mo100<04,0*05
\s:2573305,c:1711995681*05\!BSVDM,1,1,,A,33mVsf50000HkspSUhSbl86R0De:,0*40
\s:2573349,c:1711995681*0D\!BSVDM,1,1,,B,13mGaw?P00PvB9hUq;=v4?v`20Ru,0*34

But it works with another live stream I've tested against. I can share the IP address of that one privately.

One open question is, whether AIS TCP servers always send one line per packet, and if not, how simpleais handles multiple lines and/or partial lines.

[astuder]

Yep, multiple/partial messages per TCP packet was the issue for Norway.

I updated the implementation to yield only complete lines. But probably something that should be centralized for other sources that don't have a native readline function.

[astuder]

Refactored TCP after testing with the Norwegian data stream as the first approach didn't work reliably, and applied the same fixes to UDP. If think I'm done with this PR :)

[wpietri]

Thanks! I like where you're going with this, and will figure out how to add some testing.

[wpietri]

I think you have a couple of good use cases here (inbound UDP and outbound TCP), but I'm still not sure this functionality belongs in this tool.

If we do put it here, I'd at least want to have a story about how to handle inbound TCP. And for the inbound cases we'd need to figure out whether people need to bind to specific addresses and if so how to handle that at the CLI (we can't just use a .255 address; those are perfectly valid sometimes). But that pushes toward having specific command-line options, which means adding them to the tools.

With all that, I start to think we just need a separate program to manage the network interactions and produce an output stream, which is what many of these tools do already. Or maybe the network stuff should go just in aiscat, whose only job is to take in stuff and output a steady stream of valid AIS fragments. People could then pipe the results to one of the other more specific tools.

That's a plausible direction, but tools for turning network connections into local streams already exist. So have you tried just using ncat for the network part of your needs? It has a Windows version, and I'm pretty sure Windows CLIs have enough pipe support that you should be able to pipe ncat output right into, say, aist.

[astuder]

I guess it depends on your goal for simpleais. I initially thought the command line tools were just reference implementations to demo the library. But if they are the main purpose of this project, then I can see your point of view.

Unfortunately, on Windows piping data from CLI command to the next doesn't work well for streamed data sources. The output is only passed on after a command completes, i.e. only after the file ends, or a serial or network connection is closed. But you could also make the ncat argument about serial support as cat /dev/ttyxyz | aist works on Unix.

I'm fine to not add UDP/TCP capabilities directly to the library if you think that's feature creep beyond the scope of this project. It isn't that hard to implement networking outside the library and just pass in the NMEA data line by line.

As for how to pass protocol, direction, ip address etc. from the CLI: We could either add command line flags, or invent prefixes in addition to http/https (tcpclient, tcpserver, udp, serial) to remove the guesswork.