com-lihaoyi / upickle

uPickle: a simple, fast, dependency-free JSON & Binary (MessagePack) serialization library for Scala
https://com-lihaoyi.github.io/upickle
MIT License
706 stars 158 forks source link

The 18 digit numbers generated by the snowflake algorithm will lose accuracy after reading #522

Closed mokexinxin closed 9 months ago

mokexinxin commented 9 months ago

Hello: I want to use JSON to read the 18 digit numbers generated by the snowflake algorithm, but I encountered an accuracy loss issue after reading, as follows {'a':738529527931269425} But after I ran it, I showed the following results: {"a":738529527931269376}

The running result shows incorrect numbers, missing is accuracy. If I want to fully obtain this number without converting it to a string, is there a solution?

Note: This issue has been encountered while running sjsonnet. Preliminary troubleshooting may be in this project, and we hope the author can help solve it. Thank you very much!

lihaoyi commented 9 months ago

duplicate of https://github.com/com-lihaoyi/upickle/issues/520#issue-1895759120

lihaoyi commented 9 months ago

Sjsonnet works with a Javascript data model. That means double precision floating point numbers. There really isn't anything we can do here: Even if uPickle is able to read the raw JSON number as a full un-truncated string, Sjsonnet does not, and

  1. It's unlikely we can use a different number representation for Sjsonnet, due to performance considerations: BigDecimals are orders of magnitude slower and more memory-intensive than Doubles, and Sjsonnet is somewhat performance sensitive (perf is it's raison d'etre for existing)

  2. databricks/sjsonnet is meant to be equivalent to google/jsonnet, and google/jsonnet also truncates long numbers. It truncates your example differently - which is its own issue I've opened as https://github.com/databricks/sjsonnet/issues/186 - but it still truncates it, so your use case is not supported on any implementation of the Jsonnet language

You'll have to wrap your long numbers it in a string, or use a different configuration language that uses arbitrary precision numbers