Numerical stability

[kouyk]

It seems like currencies are handled as javascript floats. This is probably going to cause quite a lot of numerical instability issues. For example, I was trying to add a transaction that is $11.10 in total, then by using the exact split option, entering $5.20 for the first person results in $5.90 remaining, the calculated value is inexact $5.8999... This made it difficult to pass the verification. While there are workarounds such as splitting by shares, it doesn't solve the fundamental issues with using floats with currencies. Larger values will cause even more issues. Maybe this aspect could use some tests for such edge cases and ensure that the sums are correct.

[KMKoushik]

Yes, rounding off to 2 digits will actually fix this issue easily, As I am storing everything as whole number in DB. Good catch will work on it. Thanks

[kouyk]

As I used the application more frequently, I'm starting to find edge cases where storing as whole number in DB does not seem to be able to solve the issue since the split is determined in the frontend.

The set up consists of two users, Alice (current user) and Bob in a new group.

Steps:

1. Splitting $32.61 equally, paid by Alice.
2. Observe in the split dialogue that both parties will be liable for $16.30 which clearly does not sum up to the actual total
3. After the transaction has been added, it can be seen that Alice lent $16.30, while for the transaction just added, it shows $16.31 is being lent.
4. Tapping into the transaction will reveal that Alice and Bob each owe $16.30.
5. If a similar transaction is added, this time paid by Bob, Alice will consistently own $16.30 from the point of view of the current user.

Splitwise is observed to randomly assign the final cent in this case, which is updated each time the transaction is updated as well.

[alexanderwassbjer]

Not sure but maybe this fixes it: https://github.com/oss-apps/split-pro/pull/135

[kouyk]

Not sure but maybe this fixes it: #135

I've checked and this does not fix the issue, this issue is currently a design flaw in how currency is being handled in this application, which manifests itself in several way, some of which might be worked around through the mentioned "fix" that simply relies on creating a threshold. However, the application should not be relied upon since the missing cent problem will gradually rear its ugly head.

I'm not a javascript developer, but I do know that this is a result of the language handling all numbers as floats. Avoiding writing the backend in JS is of course the most optimal way, but probably not going to happen since that would be a rewrite. The proper fix is likely going to involve multiplying values by a hundred (at least for dollar like currencies that has up to cents) and then use tricks to ensure that the calculations work exactly like integers in other languages. Nevertheless, the issue with floats will still come up when the number get big which is sooner than we think in countries that went through absurd inflation levels.

Till this is fixed, I suppose this application should only be used casually for its purpose.

[KMKoushik]

i don't think this is language issue, but the problem float in computers! actually this is real life problem as well. how do you split 5.01 USD in to 2 exact half? you can't!

the solution as you said is to just assign the extra cent randomly or round robin.

also @kouyk why won't it work for bigger numbers? the error margin is always going to be 0.01 right?

[KMKoushik]

but yh this PR won't fix it!

[kouyk]

i don't think this is language issue, but the problem float in computers!

It is a language issue in that you cannot have actual integers in javascript, I think after all these years the general consensus is that floats are to be avoided when handling currency. I don't think anyone of us will be able to sleep at night if our banking system handles money using floats and arbitrarily rounds number.

actually this is real life problem as well. how do you split 5.01 USD in to 2 exact half? you can't!

the solution as you said is to just assign the extra cent randomly or round robin.

The ideal solution would be to round robin, or at least randomly assign, but that isn't being done so if it is always the same party entering problematic splits, the outcome will always be the same and someone is going to be constantly on the losing end. As I've mentioned previously, splitwise uses random splits to tackle this issue.

I'm not an accountant, but it is not hard to see how this requires a proper algorithm (not complicated) to handle cases such as 3 cents split among 5 people. While random split is an option, another school of thought splits it according to whose exact split is closer to the next cent. Some may prefer one over another, but it would be good if the algorithm is made known to users for an application where splitting is the core feature. Better yet, let the user choose the default algorithm should more than one be implemented.

also @kouyk why won't it work for bigger numbers? the error margin is always going to be 0.01 right?

Apologies for not being specific enough. Bigger numbers have their own problem that's has got to do with numerical stability and not just this rounding issue. AFAIK we can handle up to 2^53-1 in javascript without losing precision for integer values, as transactions accumulate it is possible that this threshold will be broken silently. Granted that this is not probable unless you're in countries like Zimbabwe with hyperinflation or handling business scale transactions, of which the latter probably should be using enterprise grade software, it is still very unsettling that this is sort of a ticking time bomb. Not exactly a functional requirement but just one to be mindful of.

I'm most definitely being pedantic on this issue overall, but correctness can never be overrated right?

[KMKoushik]

It is a language issue in that you cannot have actual integers in javascript, I think after all these years the general consensus is that floats are to be avoided when handling currency. I don't think anyone of us will be able to sleep at night if our banking system handles money using floats and arbitrarily rounds number.

yes and that's why i chose not use float for DB, it's a concious choice, when comes to frontend it's easy to swap the algo anytime!

I'm not an accountant, but it is not hard to see how this requires a proper algorithm (not complicated) to handle cases such as 3 cents split among 5 people. While random split is an option, another school of thought splits it according to whose exact split is closer to the next cent. Some may prefer one over another, but it would be good if the algorithm is made known to users for an application where splitting is the core feature. Better yet, let the user choose the default algorithm should more than one be implemented.

agreed and this will get better at 2.0, i don't want to hurry this and have a proper thought before doing this. i think it's fine for now cuz we always have exact number split, and if you have problem with also, you can choose exact split atleast.

AFAIK we can handle up to 2^53-1 in javascript without losing precision for integer values, as transactions accumulate it is possible that this threshold will be broken silently. Granted that this is not probable unless you're in countries like Zimbabwe with hyperinflation or handling business scale transactions, of which the latter probably should be using enterprise grade software, it is still very unsettling that this is sort of a ticking time bomb. Not exactly a functional requirement but just one to be mindful of.

i'll think about this, may be we can use big int!

[alexanderwassbjer]

@KMKoushik I just thinking of a wild idea. What if we handled the splitting and the splittype with only percentage? Not the solution to the problem that are discussed here but I am just thinking for a future refactor of it and the pros cons of just using percentage of the total expense amount in all splitype scenarios.