Floating point precision

[mikee47]

So we currently have number to support floating point values. These are implemented as float which is a 32-bit floating-point implementation. The issue we have is of rounding, and of how to represent these values by default.

For example:

pival  3.1415926535897932384626433
float  3.14159274101257324219
double 3.14159265358979311600

So float can get about 7 digits of precision, double about 16.

Question is how to format these numbers when serialising / de-serialising.

[mikee47]

ArduinoJson6 has a more sophisticated method for serialisation, including exponentiation.

[mikee47]

ConfigDB support for double-precision floats could be used if required. Depend on application needs. Could always store values as strings...

[pljakobs]

Do we currently have any architectures that support hardware floating point? I think the new RP2350 does? My initial thought would be to use double if the hardware provides it (we can always cast down) and to default to float on hardware that has no fp unit.

[mikee47]

I think whether there is hardware support or not is only relevant for speed of calculations (and library size). The main concern is accuracy, so that whatever values are provided in the config. don't get changed into something different.

I've been digging further into this and it all boils down to how many bits the floating-point allocates for the mantissa. For 32-bit float it's 23 bits, so a range of +/- 16777216 (0x100'0000), and for 64-bit double it's 52 bits (53 effective) with a range of +/- 9007199254740992 (0x0020'0000'0000'0000).

Values outside these ranges can be stored but with truncated precision, as highlighted above.

I can think of a few options to handle this:

1. Store numbers as strings, which get converted internally into float/double as needed
2. Don't store floating-point values at all, but use rationals (e.g. 1/2)
3. Use float by default but allow option to 'upgrade' to double (e.g. set "ctype": "double")
4. Store everything as double

There are 128-bit formats, fixed-point formats, etc. but this is overcomplicating things. I'm leaning towards (3) as it's simplest to deal with.

This does still leave the issue of converting values to strings. Clearly 1e15 is a lot more concise than 1000000000000000.0!

[mikee47]

I'm now leaning towards option (5) - none of the above :-)

The problem is fundamentally that float and double both use base 2 (optimised for computation) but we require base 10. So I think a special Number class is required here. Numbers can be stored efficiently using 32 bits and without any rounding issues. The class then provides float and double operators which applications can use as required.

(NB. It's the reason that python has a decimal library.)

[pljakobs]

are we going to use BCD now? :o I'm not sure if coding up an extra number format is worth the effort, given that this is supposed to be a store for configuration values. My expectation would be that those are of limited precision, in my case, for example, I have percent values that are float but nobody in their right mind would store more than one digit, maybe two (realistically, one is plenty since the PWM is 10 Bit so .1% describes pretty much exactly the needed precision). Now, clearly my use case is not the only one, but: if we were to draw the frequency of use of n precise digits, would we not get to very low use numbers once we get to more than 4? okay, we could theoretically have many digits in front of the decimal dot, but what is a sensible median of digits we might see? six maybe? Storing a six-digit number as a string would take eight bytes, right? With shorter numbers taking less and longer numbers taking more but I would, with having those numbers picked out of thin air, assume that eight bytes would be a sensible median. That's exactly the space that a double takes for every value, no matter of it's size. Personally, I would think String is it and have getter overloads for float and double, let the compiler deal with the rounding.