search

miloyip / nativejson-benchmark

C/C++ JSON parser/generator benchmark
MIT License
1.97k stars 262 forks source link

Universal conformance testing #51

Open maciejhirsz opened 8 years ago

maciejhirsz commented 8 years ago

It would be nice for the conformance tests to allow all valid implementations of JSON. For example, the roundtrip tests include negative zero with a fraction [-0.0] and will fail for any implementation that doesn't return [-0.0] after parsing and re-serializing, which is not necessary for a valid JSON implementation.

The spec doesn't differentiate between floating points and integers, it only describes a vague number which is neither (as it doesn't include NAN or infinities, but does include decimals), therefore 0.0 and 0 are different representations of the same number and can be printed either way. There is also no mention of negative zero and whether or not it should be preserved.

In fact, the native JSON implementation in all major web browsers would fail this test as they all print simply [0], event though the base number type in JavaScript is IEEE 754 double float, and negative zero is available in the language.

maciejhirsz commented 8 years ago

To add to that: many tests included test for high number precision, however RFC7159 states:

This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754-2008 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available.

Note that when such software is used, numbers that are integers and are in the range [-(253)+1, (253)-1] are interoperable in the sense that implementations will agree exactly on their numeric values.

A lot of the included conformance tests, test for values outside of that scope, such as max values for (un)signed 64 integers. It's good to have higher precision when available, but it is not required by the specification.