Graphics fuzz test

[aduros]

We should have a cart that draws random garbage to the screen using the different drawing functions (rect, line, oval, blit, etc) and compares each pass using a known precomputed hash of the framebuffer. If the hash differs it can trace something like FAILED on test #5393.

The main value of this, besides catching regressions, would be to ensure renderer parity across the web and C runtimes, as well as assist in developing 3rd party runtimes.

[JerwuQu]

I built a proof-of-concept thing in Zig a few months back that would embed the native runtime renderer into the WASM and compare it against whatever it's being ran on.

Basically there'd be functions that draw to the screen. It would run it both on the native runtime implementation and through the WASM-4 API, then compare the two framebuffers to see if they match. No hashing needed! It would have the advantage of being able to run thousands of tests through simple loops, basically fuzzing the graphics API.

Not perfect, but might spark some ideas for a more proper solution: https://github.com/JerwuQu/wasm4-gfx-test-cart/blob/master/src/main.zig https://github.com/JerwuQu/wasm4-gfx-test-cart/blob/master/src/tests.zig

Late edit:

I made something that's starting to be pretty complete: https://github.com/JerwuQu/wasm4-test-cart/

[aduros]

Oh nice, yeah, that sounds like an even better idea!