Spec compliant number representation & precision

comunica / sparqlee

⚙️ SPARQL expression evaluator library - Moved to @comunica/expression-evaluator

https://www.npmjs.com/package/sparqlee

14 stars 7 forks source link

Spec compliant number representation & precision #45

Closed wschella closed 1 year ago

wschella commented 5 years ago

The [SPARQL-spec] uses multiple numeric datatypes from the xsd-spec. These types have a fixed representation, minimal value, maximum value, precision, ...

Currently none of this is checked by us, and everything is the JS number datatype, which is a "a double-precision 64-bit binary format IEEE 754 value", i.e. a double in C, f64 in Rust.

[ ] Parse (and mostly validate) incoming values correctly
[ ] Store internally at the correct precision through libraries such as decimal.js
[ ] Handle correctly during operations such as addition (i.e. should we overflow, widen the type?)
[ ] Handle correctly during Type Promotion (#12 )
[ ] Export and print in the correct lexical form (although I think this is already the case)

Ideally the strictness is all quite configurable.

jitsedesmet commented 3 years ago

Handle correctly during operations such as addition (i.e. should we overflow, widen the type?)

We already fixed and documented this I think? In code: https://github.com/comunica/sparqlee/blob/master/lib/functions/Helpers.ts#L214-L224

And more precisely in the url talked about there: https://www.w3.org/TR/xpath20/#mapping We see a table that talks about the types.

wschella commented 3 years ago

I'm aware we do it correctly at the type level. I added it here among the checklist to remind us that we should also keep it in mind when doing it with at the value/representation level.

jitsedesmet commented 1 year ago

I think all of these things are already handled?

1: Should be okey? (after this many years) 2: https://github.com/comunica/sparqlee/pull/44 3: I think the OverloadTree handles these things 4: https://github.com/comunica/sparqlee/issues/12 ? 5: Just like you, I also think this is done

wschella commented 1 year ago

I'm not sure about anything, it's been a while and there have been many changes.

I do think however that 3 might not be implemented correctly. If you add two limited width integers (e.g. shorts), and the result of their addition or multiplication exceeds the bounds of the datatype, what are we supposed to do? JS won't complain, and we will serialize a number too big, but still call it xsd:short. What does the spec mention about overflow? Either we have to widen the datatype, do modulo arithmetics?

Related, I don't think we validate that when we receive an xsd:short, whether it iactually fits into an xsd:short. Not saying we should error, but a warning would be good, or at least an optino to trigger a warning.

I would make two separate new issues for this in the communica repo.

jitsedesmet commented 1 year ago

Oh yeah! I misunderstood. Thank you!

jitsedesmet commented 1 year ago

I will close this issue in favour of three issues created on the Comunica repo: