HazyResearch / deepdive

DeepDive
deepdive.stanford.edu
1.95k stars 539 forks source link

Fix jq load_probabilities precision error using python #621

Closed zifeishan closed 7 years ago

zifeishan commented 7 years ago

jq has a precision bug while loading probabilities with big vids, due to a known issue that it's using double floating numbers to store integers. There seems no good way in jq to support better precision: https://github.com/stedolan/jq/issues/369

We've tried using awk with --bignum but versioning seems a problem.

Here we just try to use python3 to do the job.