pkulchenko / ZeroBranePackage

Packages for ZeroBrane Studio (https://studio.zerobrane.com)
MIT License
245 stars 149 forks source link

Redis: Float precision is lost in Stack and Remote Console #66

Closed tw-bert closed 6 years ago

tw-bert commented 6 years ago

Redis 4.0.8, redis.lua 0.36

High precision floats/doubles (numbers) have precision loss, while viewing for debugging only. The actual value is fine, but can't be shown in ZeroBrane. I have no idea if this is a Redis limitation or a ZeroBrane limitation.

See the following code:

    request_curr.now = request_argv.now 
    request_curr.now_debug_str = string.format("%015.6f",request_argv.now) -- Just for debugging, to show no precision has been lost. This is near impossible to see from ZeroBrane and/or Redis stack directly.

And the Stack: image (obviously, 'now' has lost precision) Remote Console gives similar issues.

I tried (user.lua):

debugger.numformat = "%015.15f"

But that gives: image

pkulchenko commented 6 years ago

I have no idea if this is a Redis limitation or a ZeroBrane limitation.

As far as I remember, it was Redis limitation; you may be able to confirm if you turn verbose option on, as it will show details on what was sent and received from Redis: redis = {verbose = true}.

tw-bert commented 6 years ago

Checked:

<redis rcvd> +<value> request_curr = {[\"now\"]=2.01802e+07; [\"now_debug_str\"]=\"20180212.104500\";}

(@pkulchenko:)

As far as I remember, it was Redis limitation

Yes, that's my conclusion as well: Redis limitation / issue. @itamarhaber : Is there a setting or parameter to set the redis-side serialization format for floats?

pkulchenko commented 6 years ago

ok; thank you for the update. closing.