josephg / node-foundationdb

Modern Node.js FoundationDB bindings
Other
115 stars 17 forks source link

Support BigInt in the tuple layer #38

Closed josephg closed 4 years ago

josephg commented 4 years ago

The tuple encoding semi-officially supports large integers (up to 255 bytes in length). Arbitrary precision integers are implemented in the python and java bindings - although apparently the encodings are subtly different :/

Anyway, we should map these across to bigints. Its a bit awkward because javascript considers the numbers 10 and 10n to be different, but the tuple encoding seems to consider them as just part of a sliding continuum.

There's an implementation decision here. We could:

  1. Quietly use bigints for any integer we read in the tuple decoder outside the safe integer range, which would have the downside that round-tripping some integers through the tuple encoder could convert numbers to bigints or vice versa. Or
  2. Only use the arbitrary precision integer tag for bigints (of any size), and only use the other integer encoding tags for smaller integers. But this would violate the tuple ordering guarantees - since lexographically this would make tuple(1n) > tuple(10) and tuple(-1n) < tuple(-10). We would also need another encoding for zero (0n) - like maybe a zero-length arbitrary precision positive integer? Anyway, this feels gross and hacky.

I think the most sensible answer is (1) - though I might put the behaviour behind a flag, and if that flag isn't set continue to disallow any unsafe integer from being encoded.

josephg commented 4 years ago

For posterity, I've implemented (1). I think its the only real choice: