search

paulmandal / atak-forwarder

Forwards packets to/from ATAK over an off-grid communication layer
MIT License
404 stars 42 forks source link

protobuf RFE: floating point considered harmful :) #98

Closed a-f-G-U-C closed 2 years ago

a-f-G-U-C commented 3 years ago

Floating point fields (and especially double-precision floats) are large, difficult to compress and overhead-y in both layout and operation.

In bandwidth-constrained contexts, like the atak-forwarder protobuf messages where every byte counts, a significant size reduction can be achieved by using exclusively integer fields.

Example

A latitude or longitude value (currently represented as a 64-bit double) can be instead represented using a 32-bit sfixed32 integer field with a 7-decimal precision (sufficiently accurate to "point to Waldo on a page" *) - and achieve an 8-byte saving per lat/lon point.

Proposal

Banish all floating-point fields from the protobuf definition, now and forever, and replace them with conservatively-sized integer fields. Where possible, the integer values should be used verbatim. However, if the range of the chosen format does not fit the range of the variable comfortably, scaling and linear range mapping rules can be used to force a better range coverage.

*source: https://xkcd.com/2170/

a-f-G-U-C commented 3 years ago

Example of range mapping

Elevation is a particularly tough value to represent efficiently, because it can reach large positive numbers, but also small negative numbers. By range-mapping (0 .. 2^16-1) to (-900 .. 19000)meters, any real-world elevation can be represented in only 16 bits unsigned with a resolution of ~0.3 meters, sufficiently accurate for most applications considering that typical GPS vertical accuracy is ~3 meters.

source: MISB-ST-0601

paulmandal commented 2 years ago

https://github.com/paulmandal/libcotshrink/pull/14 both addressed here, thanks for the suggestions!