calrissian / mango

Common utilities for rapid application development
Apache License 2.0
17 stars 7 forks source link

BigDecimalLexiEncoder has a bug with lexi encoding 0 #203

Closed eawagner closed 8 years ago

eawagner commented 8 years ago

The encoding for 0 is incorrect. The problem is that is is setting the exponent to 0 then using the integer encoding like with all other numbers. The problem is that when it is encoded with sign bit the following is the start of the encoding 1800000000. A 0.1 is correctly encoded with the value 17fffffff1.

The problem is that the encoded 0 will lexicographically appear after the 0.1 because we aren't treating 0 as special.

The most correct solution is to encode 0 as 1000000000, where the exponent is simply zeroed out. The only problem with this is that the decode method will need special logic to treat the exponent differently in the condition that all the chars are '0'