ferram4 / Ferram-Aerospace-Research

Aerodynamics model for Kerbal Space Program
Other
238 stars 131 forks source link

Incorrect rounding of numbers in Flight Data display in flight #183

Closed Virindi-AC closed 7 years ago

Virindi-AC commented 7 years ago

In the "Flight Data" display, sometimes the angle of attack number reads as ".10", when it is not actually .1. That is, the number will go from eg. 7.7, 7.8, 7.9, 7.10, 8.0, 8.1. In the tests it does smoothly run through those values so it is obvious that it is not actually "7.1" in that example.

1.2.2 Win64, FAR f38f1f36784a43a471ac583ab12ad15112de4eb7

Screenshot: http://www.virindi.net/junk/far_decimal.png

Virindi-AC commented 7 years ago

Hold on, my checked out code doesn't match what is showing on github. Jeez, this might be my bad.

Virindi-AC commented 7 years ago

Nope, this report is correct. Seems to be a problem with the stringbuilder extensions.

Ran a simple test, the following code: StringBuilder sb = new StringBuilder(); sb.Concat(7.99f,1); Console.WriteLine(sb.ToString());

produces the result "7.10"

Virindi-AC commented 7 years ago

The logic of this library code to cat a number to a stringbuilder is completely broken; it fails in many cases.

Some examples of incorrect output of Concat( this StringBuilder string_builder, float float_val, uint decimal_places, uint pad_amount, char pad_char ):

as well as numerous possible ways that rounding can be wrong due to floating point inaccuracy.

ferram4 commented 7 years ago

Merged the PR, so this can close.