HairyFotr / linter

Static Analysis Compiler Plugin for Scala
Apache License 2.0
268 stars 34 forks source link

Loss of Precision error not correct #4

Closed non closed 11 years ago

non commented 11 years ago

It's not enough to use a string constant with BigDecimal and a large decimal expansion, you have to actually provide your own MathContext:

scala> BigDecimal("0.12958129581958395829583295823958329852395832958295382")
res0: scala.math.BigDecimal = 0.1295812958195839582958329582395833

scala> BigDecimal("0.12958129581958395829583295823958329852395832958295382", new java.math.MathContext(200))
res1: scala.math.BigDecimal = 0.12958129581958395829583295823958329852395832958295382

I'm not sure the best way to word the warning, but I think the current warning is misleading.

HairyFotr commented 11 years ago

Interesting, I'll change the warning, and also try to detect the loss of precision for String. Thanks!

non commented 11 years ago

That's a good idea. For numbers that don't use scientific notation you can just remove the dot, and count the digits.

scala> BigDecimal("1.234567", new java.math.MathContext(7))
res7: scala.math.BigDecimal = 1.234567

scala> BigDecimal("1.234567", new java.math.MathContext(6))
res8: scala.math.BigDecimal = 1.23457

The default MathContext used by BigDecimal has 34 digits of precision. I don't know that the language guarantees this, but I also don't imagine it changing soon, so it's probably safe to hardcode that for now.

non commented 11 years ago

(That said, if you're using a compiler plugin or macro, you can actually create your own BigDecimal and see how much precision its MathContext has, which should be relatively future-proof.)

HairyFotr commented 11 years ago

I did a very hacky checker for strings, but I think it works for most cases... Will revisit in a few days to fix a few issues and to clean it up, and then I'm closing.