search

axheron / ffxiv-dps-calc

A Final Fantasy XIV DPS simulator for comparing gear sets
https://dpscalc.xivresources.com
GNU Affero General Public License v3.0
2 stars 4 forks source link

changes tests to be accurate to some relevant decimal place #89

Closed joooooooooe-star closed 3 years ago

joooooooooe-star commented 3 years ago

fixes that float point test issue

joooooooooe-star commented 3 years ago

tests now fail so we have to increase the places or keep the old values, please lmk what you decide on

ackao commented 3 years ago

the test failures are... ugh...

AssertionError: 20381.368 != 20381.368540000003 within 3 places (0.0005400000045483466 difference)

from the documentation:

Fail if the two objects are unequal as determined by their difference rounded to the given number of decimal places (default 7) and comparing to zero, or by comparing that the between the two objects is more than the given delta.

Note that decimal places (from zero) are usually not the same as significant digits (measured from the most signficant digit).

this is happening because of rounding nonsense. the tests that fail fail because 0.0005 gets rounded to 0.001. sigh.

i think we fix this issue by having the value for comparison have 4 digits after the decimal point but we keep places=3

joooooooooe-star commented 3 years ago

is ok now