Recently the lack of comprehensive unit tests has bit us painfully. Updating some things lead to breakage of other things. We might need to elevate the discussion about increasing unit test coverage.
Not a lot of things are unit testable in isolation. Currently only alolcator and urmom are covered by miniature ad-hoc unit tests. There are not much other things that can be as easily tested in isolation (i.e. without the rest of the engine, vulkan drivers/devices, and the rest of the entire circus).
Thing we might want to cover with integration-like tests:
Certain map locations rendering being the same as a reference screenshot.
Internal structures having expected value. E.g. that we have loaded all surfaces, all lights, applied all patches, have all PBR materials. And that there are no extra unexpected things.
How to engage tests:
From a savefile.
From a command/script (e.g. load map, then teleport).
How to specify verification:
Have a screenshot.
Have a file describing expected renderer state.
Issues:
Requires a GPU with RT cores. Cannot run on a regular GH Actions CI.
Likely needs headless operation, which requires engine changes.
Recently the lack of comprehensive unit tests has bit us painfully. Updating some things lead to breakage of other things. We might need to elevate the discussion about increasing unit test coverage.
Not a lot of things are unit testable in isolation. Currently only alolcator and urmom are covered by miniature ad-hoc unit tests. There are not much other things that can be as easily tested in isolation (i.e. without the rest of the engine, vulkan drivers/devices, and the rest of the entire circus).
Thing we might want to cover with integration-like tests:
How to engage tests:
How to specify verification:
Issues: