GPS accuracy and float precision

[ctorney]

Hi I've recently started using this library and I've noticed there appears to be an issue with GPS accuracy loss due to parsing as a single precision float.

This was raised here #10 and a PR #12 was started but it looks like both were closed without fixing the issue.

It seems that the reason for closing the issue was because the truncated value can be printed to 6 decimal places (this is also written in the docs). But this isn't really a fix.

A float can be printed to more digits than its precision because of the error created when converting between binary and decimal.

Here's a quick illustration

>>> x = float('9.87654321')
>>> x
9.87654
>>> print("Original: 9.87654321, Truncated: {0:.8f} ".format(x))
Original: 9.87654321, Truncated: 9.87654114 

So while the float is printed to 8 decimal places the last 3 digits aren't related to the original value. It seems the same thing is happening with the latitude and longitude values - accuracy is lost from the location depending on the number of digits in the degrees.

Thanks

[lesamouraipourpre]

Ref: https://learn.adafruit.com/circuitpython-essentials/circuitpython-expectations

CircuitPython's floats have 8 bits of exponent and 22 bits of mantissa (not 24 like regular single precision
floating point), which is about five or six decimal digits of precision.

Assuming the information at the link is still correct, I don't think the float in CiruitPython can cope with the precision provided by GPS unfortunately.

An option may be to parse the string to an integer instead, so 23.123456 would be returned as 2312345600 (8-digits of float precision). But this would be a breaking change for any existing users, so would need some discussion.

[ctorney]

Hi, Yes I guess there isn't a straightforward fix. Other non-breaking options would be to have a separate set of coordinates (latitude_full_precision,longitude_full_precision) defined as int, string or named tuple, or have a full_precision flag set on inittialization that switches to the full precision values.

The documentation should also be changed though, e.g. here:

This isn't ideal for GPS data as this lowers the accuracy from 0.1m to 11m.

This can be fixed by using string formatting when the GPS data is output.

This should really be saying accuracy is ~11m if your location has two digits in the degrees.

[laldew]

What if the parsing function just split the string by the decimal, floated and dealt with each side individually, and then pasted them back together? There will be enough digits to float each part that way:

import re

nmea_data = '18040.1234'
data_spl = nmea_data.split('.')
ddd = float(data_spl[0]) // 100
mm = ( float(data_spl[0]) % 100 + float(".".join(['0',data_spl[1]])) ) / 60

fin = '{}.{}'.format(int(ddd), str(mm).split(".")[1])
fin

This results in the correct precision, all the way to the 6th decimal place and doesn't have to be a full integer, disrupting previous. You still can't do math on the lat/long, but I assume most units just need to report or store the data somewhere.