jonathanhogg / flitter

A functional programming language and declarative system for describing 2D and 3D visuals
https://flitter.readthedocs.io
BSD 2-Clause "Simplified" License
34 stars 1 forks source link

Add some basic functional testing #35

Closed jonathanhogg closed 4 months ago

jonathanhogg commented 5 months ago

Assuming that I can get #34 done, then it'd be good to at least add functional tests based on the programs in /examples. What I'd need to do is rig them up to run for a set amount of time offscreen and then dump the framebuffer to an image file. If I add sample images for each of the examples into the repo (which would be good for documentation sake anyway) then I could do some image comparison to check that the examples are rendering correctly.

Possible useful link for image comparison:

https://sewar.readthedocs.io/en/latest/#

jonathanhogg commented 5 months ago

Good news. I've merged support for offscreen rendering and added a few simple tests that create an engine, load a script that creates an output image, execute that script and then check that the output image is correct. See #37. This is now part of the GitHub workflow – so these basic engine functional tests will be executed on commit.

At the moment I'm only generating solid green outputs and confirming that the resulting bytes are correct. Next up would be adding support for comparing interesting output images to see how close the rendering is to the reference image.

jonathanhogg commented 4 months ago

As I've been writing documentation, I've been doing some diagrams in Flitter to explain certain functions' output (see builtins.md). I've now added tests to recreate these diagrams and compare them against the reference ones included in the documentation. Next step will be to add some images into the tutorial and do the same there.

Sadly all of this uses only 2D rendering, so I'll need to do some 3D examples. Probably the smart thing to do is to slightly rework all of the examples to be able to dump their output to an image file and then include reference images for each of the examples.

jonathanhogg commented 4 months ago

Tutorial images added and successfully tested.

jonathanhogg commented 4 months ago

I've got this working for the examples as well. So I'm calling this done.