search

nasa / MCT-Plugins

35 stars 29 forks source link

[Scenario] Can't get rid of decimal places, and they overlap #109

Open VWoeltjen opened 10 years ago

VWoeltjen commented 10 years ago

Reported 12/3/2013 during followup on timeline-building.

Extraneous decimal places sometimes appear in cost graph and cannot be suppressed. Additionally, these overlap and hinder readability.

VWoeltjen commented 10 years ago

Observed 12/9/2013 in telecon

May be related to error accumulation in double-precision floats (during cost summation) or inability of floating point to express certain precise values

VWoeltjen commented 10 years ago

Verified that this is error accumulation (for same reason System.out.println(0.4+0.2) gives 0.6000000000000001)

Switching to DecimalFormat instead of Double.toString fixes the issue from a user interface perspective. The default DecimalFormat seems to have a maximum of 3 digits to right of decimal place, but I can't find this documented anywhere. This is probably enough precision for GraphView (some redesign would be required to accommodate more to avoid overlap.)

The root cause could resurface as an issue if there are any situations where that level of precision is really important. The solution there (from a correctness perspective) would be to use BigDecimal instead of double for cost computations (BigDecimal supports arbitrary precision) but would need to assess both the performance impact of that change, as well as the user interface impact of when and how to display that extra precision.

VWoeltjen commented 10 years ago

Easiest way to reproduce issue is to have an Activity with two Activity Types, one of which has a cost of 0.4 for power (or comms) and the other with a cost of 0.2 for the same. Expect 0.6 cost for the Activity in question but get something like 6.00000000001 instead (actual number of zeroes may differ)

VWoeltjen commented 10 years ago

Self-verified on Windows 7, hacktahon 1/2/2014, Scenario build 177

VWoeltjen commented 10 years ago

This has been observed as recently as 2/11/2014. Have not been able to reproduce locally.