codeforboston / cliff-effects

Cliff effects guidance prototype (archived)
https://codeforboston.github.io/cliff-effects/#/
MIT License
30 stars 64 forks source link

How consistent does our math need to be? #66

Closed knod closed 6 years ago

knod commented 6 years ago

[Edit] Rather, how does the government treat numbers? What we really want is to be consistent with governmental calculations that will actually determine the client's benefits. Are there standards or guidelines? Some possible questions: Do number stay as raw as possible, only being rounded when displayed? Are numbers rounded on some occasions and floored on others? Is the emphasis on precision or on consistency? Another situation is described in the second comment on this thread. How is a situation like that handled?

If the government doesn't have such a thing, how consistent does this app want to be? And how precise does it want to be?

[Original below]

See the second comment in this thread for an explanation of the actual, and even more interesting, question.

It's impossible to have accurate math (see below), but we can either be more or less accurate. How accurate do we need to be? It's better if we decide sooner rather than later. So two questions come up in my mind: 1. How big of an error is acceptable? 2. What is order of magnitude of the problem if we use our current, plain javascript, math?

Does anyone have any idea of what order of magnitude these inaccuracies will build up to? Is it going to be a few cents, tens of dollars, or thousands of dollars? Is there a way to know?

More detailed description: When programming, it's impossible to have completely accurate math. First of all, our computers use binary math (base 2) and have to convert to base 10 whenever we want numbers. Secondly, there are always going to be times when a program will have to round an number to some decimal place. Each language decides how many decimal places to allow before rounding. So we get stuff like 0.1 + 0.2 come out as 0.30000000000000004.

(Brought up in the math test PR, #63)

yhou8 commented 6 years ago

I was more worried about whether rounding behavior would be intuitive than internal accuracy. A value written as x.x05 might be stored as x.x049... internally, so after rounding it looks like x.x0 instead of x.x1. The native number in Javascript has about 15-17 digits of precision and most arithmetic operations except subtraction of very close numbers only introduce some error in the last digit.

The part I'm not sure of is whether the programs require calculations to follow some financial standard. If we want to be extra safe, maybe consider adding decimal.js or bignum.js libraries.

scapp281 commented 6 years ago

https://github.com/MikeMcl/big.js/wiki : This page gives a brief comparison between big.js, bignumber.js and decimal.js Let's pick one.

knod commented 6 years ago

I'm leaning towards big.js since it's the smallest one. We don't need all the trig and such. As long as it's just as easy/hard to use.

jyw2116 commented 6 years ago

(Ref #63: Created a test suite for the roundMoney function in math.js)

Hi, I had a chance to look at the tests yesterday for a bit- For the discussion of roundMoney() failing 2 tests, it passes if you use .ceil() and .floor() instead of .round(), so depending on how critical it is to be able to round up and down, we might have to separate the roundMoney() method into two. Depending on where things are with testing, I'd be happy to look at separating some of the math functions this coming Tuesday with @yhou8 or anyone else who's interested.

yhou8 commented 6 years ago

I took a look at both the online and Excel versions of the SNAP calculator from Mass Legal Services. Both versions use native math (binary64 for online, decimal for Excel) instead of trying to make sure their calculations match. The actual calculations made might be different from either calculators, maybe by rounding to the nearest dollar or cent at the end of each section. So both calculators may be an approximation, although I expect whatever tools they use to be more similar to Excel then javascript.

In terms of testing, using big.js allows testing smaller math functions for exact matches, although for longer calculations testing for approximate matches should be sufficient. Since the input is expected to be accurate to two decimal places, we can test whether the output matches two or a few more decimal places. Rounding operations are tricky to test since they throw out all uncertainty about less significant decimal places. These concerns could be left for after the MVP.

I won't be at the meeting this Tuesday; have an appointment Wednesday morning.

lianilychee commented 6 years ago

Because we're pushing towards MVP, I think whatever exists currently is good enough.

However, we should definitely have a "please keep in mind" section when we hand off recommendations.

knod commented 6 years ago

Maybe more important to know: What does the government do with those values and this kind of math?

Examples of concerns: Do number stay as raw as possible, only being rounded when displayed? Are numbers rounded on some occasions and floored on others? Is the emphasis on precision or on consistency?

ManickYoj commented 6 years ago

If we're looking for the simplest solution, it's fairly common (eg. Stripe) to handle monetary amounts as integer cents (or whatever the smallest currency unit is for each currency). Makes math really easy, at the expense of needing to have the view layer parse it to the more common dollars.