Rounding to a fixed number of decimal places leads to over rounding small numbers and under-rounding big numbers. I first encountered this problem when dealing parameter values in the range of 10^-6 that kept getting way rounded off. That's not a common order of magnitude for a parameter in ArduPilot, but we shouldn't rule it out (in my case, they were part of a lua script; I don't think any official params are in this range).
Some samples from the "Immediate Window" in Visual Studio
Now, another thing to consider, do we actually need to round at all, or should we just be casting to double and be done with it? The only application I could think of where this rounding was important was string conversion, but we already correctly handle that in the ToString method.
Rounding to a fixed number of decimal places leads to over rounding small numbers and under-rounding big numbers. I first encountered this problem when dealing parameter values in the range of 10^-6 that kept getting way rounded off. That's not a common order of magnitude for a parameter in ArduPilot, but we shouldn't rule it out (in my case, they were part of a lua script; I don't think any official params are in this range).
Some samples from the "Immediate Window" in Visual Studio
Now, another thing to consider, do we actually need to round at all, or should we just be casting to double and be done with it? The only application I could think of where this rounding was important was string conversion, but we already correctly handle that in the
ToString
method.