search

tableau / tableau-log-viewer

Tableau Log Viewer is a cross-platform tool for quickly glancing over Tableau log files
MIT License
169 stars 50 forks source link

Large integers do not show correctly in TLV (double precision loss) #104

Open mringwald opened 5 years ago

mringwald commented 5 years ago

When the log contains large 64bit integers, they do not get correctly displayed. The last digits differ from the actual value.

Given the example log line: {"ts":"2018-11-28T17:33:01.230","pid":43075,"tid":"70000dbe3000","sev":"info","req":"-","sess":"10483","k":"large-integer","v":{"some-id":9223372036855065056}} TLV will show "some-id: 9223372036855064576"

This is probably due to parsing the number as a double first and loosing precision on the way. I am not sure if the underlying JSON parser supports the whole range of int64 and this is a TLV issue or if this is a limitation of the JSON parser itself.

luiseeo commented 5 years ago

Hi, I think this is a limitation of the JSON parser and how Javascript does numbers (if I remember correctly, every number is a double precision float). There might be ways to improve this in TLV; right now we are getting the default behavior of the Qt JSON parser.

Do you know if Tableau or Hyper is already logging 64-bit integers in real world scenarios?

mringwald commented 5 years ago

From a quick search, it looks like the JSON standard leaves it up to the implementation how to handle int64s.

Yes, Hyper does log some 64-bit integers. There are some cases where information is encoded in the most significant bit, resulting in very large integers when logged.