Devsh-Graphics-Programming / Nabla

Vulkan, OptiX and CUDA Interoperation Modular Rendering Library and Framework for PC/Linux/Android
http://devsh.eu
Apache License 2.0
443 stars 48 forks source link

Automated testing of UI-less examples #686

Open devshgraphicsprogramming opened 2 months ago

devshgraphicsprogramming commented 2 months ago

Description

Examples 01-03,05-07,20-24 have no Input handling and could have their testing automated.

Description of the related problem

We do lots of manual testing right now.

Solution proposal

Different levels of testing are possible:

  1. Simple stdout checks (what code the process exited with)
  2. Checkpoint and filtered log checks (no errors, expected errors)
  3. Checking of artefacts (e.g. output files, and images)
  4. checking of gfxrecon/renderdoc captures, replays and extracted data

To begin with we could do up to stage (3)

Additional context

We've planned something like this before with Arek, taking into account that some tests might need to be ran on multiple machines.