Behavior difference in rails to_json of decimal numbers with Oj.optimize_rails()

[kwerle]

Briefly, Oj rounds one shorter than rails does by default:

{foo: 0.1234567890123456789}.to_json # "{\"foo\":0.12345678901234568}"
Oj.optimize_rails()
{foo: 0.1234567890123456789}.to_json # "{\"foo\":0.1234567890123457}"

Oh - this is Rails 6.1. Have not tried with 7.x

[ohler55]

Will probably be the same in 7.x. You can configure the way Oj handles rounding using the default options.

[kwerle]

=> "{\"foo\":0.12345678901234568}"
irb(main):002:0> Oj.optimize_rails()
=> nil
irb(main):003:0> {foo: 0.1234567890123456789}.to_json 
=> "{\"foo\":0.1234567890123457}"
irb(main):004:0> Rails.version
=> "7.0.4.3"

Will probably be the same in 7.x. You can configure the way Oj handles rounding using the default options.

If you're saying I can configure it to behave like rails does, I would sure appreciate a pointer at how to do that!

Thanks.

[ohler55]

Looking at the code and realized that while you can configure Oj to use different float encodings the rails encoder is locked at 16 decimal places. So my mistake on saying the encoding for rails mode could be configured. The only fix for that is to sacrifice performance and use the Ruby to_s() call. I can think about alternate ways to detect the need for the 17th decimal and drop back to the to_s() call.

[kwerle]

I actually looked at this a little before our company punted and truncated to make it simple. Turns out we absolutely are not accurate down to 17 decimals anyway.

As I recall, I had some success checking to see if the log base 10 was equal to some number (-15?). If so, multiplying by 10**15 and seeing if rounding went even or odd. Something like that.

If you want me to look into this and maybe put together a PR, lemme know.

[ohler55]

Probably not worth the effort. I reality if someone wants more than 16 significant digits they should not being using a 64 bit float but a BigDecimal. Trying to squeeze out the 17th digit on a standard that is accurate to between 16 and 17 digits is not realistic. I suspect in most cases people don't actually need the 17th digit.