Closed winksaville closed 7 years ago
Better using 5. Because machine epsilon for float is 1.19e-07, but last 2 digits contain rounded error
I'll change it but I'm curious, Is 1.19e-07 for float32 I'd have to believe float64 is larger. Do you have a link for the wasm machine spec?
So on wikipedia I see float32 has 7.22 decimal digits of and float64 has 15.95 should we assume float32 or taylor the test depending on float32 or float64?
In current test we just validate import / export routines, so even 2 digits enough. For simplicity and perfection we can use 5 digits. This will cover as float and double. Do not complicate such a simple test.
If two is enough, then no need to merge, simple is better.
closing, we can add tight precision check when testing stdlib implementation.
According to this the default precision is 2 that seems to few, so using 10.
Also, added the
precision
test to validate the test.