[ ] Audit tests in each repo and ensure that fuzz tests are able to be isolated. The pattern we use in the rust repo is fuzz_* if the function is fuzzing and test_* if it is not. However I've noticed & fixed inconsistencies with this, so whoever does this task should check them all & update as needed.
[ ] Write a fuzzing script for each repo that isolates the fuzz tests and runs them
when a test fails, the error should be reported to rollbar with a crash report
[ ] Setup a job on the aws fuzz box that executes the rs and sol fuzz scripts repeatedly
[ ] Sync with frontend team to check if they also have fuzz tests to run.
[ ] Consider reducing fuzz amounts in the repos themselves to speed up CI on PRs, at least while this is running.
[ ] Ensure that both repo fuzz systems use new random seeds each time, and store that seed for the crash report. Some permutation of the system time should do.
This might not be necessary if we think we can reproduce the issue completely from the crash report. If that's the case, let's skip the extra work. We should still verify that repeated fuzz runs use different random seeds, so it's not testing the same states each time we run the tests. I'm pretty sure it is new each time; the default behavior should be to use the system rng without a seed, which uses moon reflections off of a soul crystal or something. But best to double check.
fuzz_*
if the function is fuzzing andtest_*
if it is not. However I've noticed & fixed inconsistencies with this, so whoever does this task should check them all & update as needed.