bytecodealliance / javy

JS to WebAssembly toolchain
Apache License 2.0
2.29k stars 110 forks source link

javy-fuzz: Compare values instead of strings. #706

Closed saulecabrera closed 4 months ago

saulecabrera commented 4 months ago

This commit introduces value comparison for the json-differential target.

The motivation behind this change is to overcome the complexity of float value formatting.

JavaScript has very complex rules for Number formatting, for example, numbers that have magnitude of 10^21, independent of the sign, will be formatted via scientific notation. I was able to achieve certain format parity by writing a custom serde serializer formatter and relying on crates like ryu and lexical. Even though this worked to an extent, there are some unwritten rules to my knowledge that make it harder to achieve full formatting parity. For example, in some cases the scientific notation would include the exponent with a sign e.g., e+ or e- depending on how big the number is. In the case of lexical or ryu even though the numerical value is the same, no signs are used, which makes it harder to perform differential testing.

Having to post-process the resulting string, will most likely incur in a performance penalty that I'm not sure is worth, especially given that no issues have been reported regarding the numerical value during my testing.

Due to all these reasons, this commit switches toward value comparison rahter than format/string comparison by parsing the stringified JSON into serde_json::Value and asserting equality on top of it.

Checklist