We currently run unit tests as well as a couple of the example Dockerfiles in CI, but this existing coverage does not seem to cover all Polytracker commands. Some features such as grammar generation and now info (see #6530, https://github.com/trailofbits/polytracker/tree/fix_polytracker_info) have bitrotted over time.
While I'm not sure if functional testing comes from "agile" full stack development, that's where I'm familiar with it from!
If the existing test suite and example Dockerfiles running in CI are not sufficient to ensure all the Polytracker features continue to work (preventing regressions), some ideas to broaden test coverage:
use a "Does It All" Dockerfile which runs all the commands, proving each works on each CI run, or maybe:
add functional-esque tests to the Polytracker test suite (we currently primarily have unit tests, which to me are more about proving a single unit or function works correctly than proving WYSIWYG from the customer POV / feature level) to prove each command works on each CI run
We currently run unit tests as well as a couple of the example Dockerfiles in CI, but this existing coverage does not seem to cover all Polytracker commands. Some features such as grammar generation and now
info(see #6530, https://github.com/trailofbits/polytracker/tree/fix_polytracker_info) have bitrotted over time.While I'm not sure if functional testing comes from "agile" full stack development, that's where I'm familiar with it from!
If the existing test suite and example Dockerfiles running in CI are not sufficient to ensure all the Polytracker features continue to work (preventing regressions), some ideas to broaden test coverage: