We have bugs and regressions with various sites as our code changes, which it does frequently! Some of these are soft-crashes in JS (throws), some is hardcrashes in the native bindings (SEGV, hangs), and some is incorrect rendering.
Solution
We could automate checking a list of sites against a build of Exokit in our CI environments on Appveyor and Travis CI.
In particular it seems it would be straightforward to check that some sites load and run to completion -- the harder part of this is differential rendering verification, which we might want to punt on till later.
One wrinkle is that we might want to make this work for all three OSes we support (Windows, OSX, Linux), so we might need three harnesses. Also, since Exokit is a graphical app that requires either OpenGL or an emulation of it (e.g. mesa on Linux), we'll need to make sure the CI environment runs such support.
This could be broken up into separate issues if this proves to be too much to chew at once 🍖.
Getting started
The three.js examples are a goldmine of simple but thorough test suites.
The ratio is high but not all of them examples load and render. However, those that do should continue to do so and can be whitelisted for verification.
avaer
commented
6 years ago
Background
Exokit essentially reimplements all of WebGL.
Problem
We have bugs and regressions with various sites as our code changes, which it does frequently! Some of these are soft-crashes in JS (throws), some is hardcrashes in the native bindings (SEGV, hangs), and some is incorrect rendering.
Solution
We could automate checking a list of sites against a build of Exokit in our CI environments on Appveyor and Travis CI.
In particular it seems it would be straightforward to check that some sites load and run to completion -- the harder part of this is differential rendering verification, which we might want to punt on till later.
One wrinkle is that we might want to make this work for all three OSes we support (Windows, OSX, Linux), so we might need three harnesses. Also, since Exokit is a graphical app that requires either OpenGL or an emulation of it (e.g.
mesaon Linux), we'll need to make sure the CI environment runs such support.This could be broken up into separate issues if this proves to be too much to chew at once 🍖.
Getting started
The three.js examples are a goldmine of simple but thorough test suites.
The ratio is high but not all of them examples load and render. However, those that do should continue to do so and can be whitelisted for verification.