OceanDataTools / openrvdas

An open source data acquisition system designed for use on research vessels and other scientific installations
http://openrvdas.org
Other
39 stars 20 forks source link

Restore eol option on UDPWriter #387

Closed webbpinner closed 3 months ago

webbpinner commented 4 months ago

WIth the network re-factor the eol kwarg was removed. This was a useful argument for detecting the end of a data record and removing it before passing the record down the pike. Without it several ships are reporting duplicate carriage returns in the data files.

Would like to see the eol kwarg added back with the default set to '\n'

veggiemike commented 3 months ago

Sorry, I'm late to the party. ;-)

The eol flag was removed because that's for stream parsing, and that's just not how UDP works. You send and receive entire datagrams at a time, unlike TCP streams. These setups that are getting duplicate '\n' are being sent strings ending in '\n', and then they're adding a 2nd '\n'. In other words, it's a bug somewhere else. We're just hiding it now by using eol to strip the '\n' off. I'd suggest using a transform to strip it off instead.

I haven't run into anything that sends multiple messages in a single UDP packet, but I suppose it is possible. These cases would benefit from eol I guess, although it still feels to me like something that should be handled downstream of UDPReader.

Setting eol='\n' certainly shouldn't be the default, as that would silently split any incoming UDP record with a '\n' in it. Just my $.02.