en-milie
commented
7 months ago Hi @qarmin. I'm actually working on a continuous fuzzing sub-command as we speak. It's almost ready. The current fuzzing is quite deterministic i.e. if you run CATS twice with the same configuration it will return the same results. This is because the fuzzers are aware of structural constraints and data types and therefore they will always create the fuzz values in the same way. Maybe I'm missing something on your setup, but running the fuzzers for 30 minutes will always return same result.
The functionality I'm working on now is a bit different. It will work exactly as you mentioned: stop after X errors or after X time, but the values for fuzzing will be random without considering the context. It's a new fuzzer that will get invoked using cats random [arguments] which has a set of Mutators that will do random fuzzing. I'll paste a link here with the documentation once updated it.
Is your feature request related to a problem? Please describe. I wanted to run app for several minutes with only 3 fuzzers, but I couldn't find any option to set number of repeats or command timeout.
Currently this 3 fuzzers completes in 2/3 seconds and I need to check for errors in previous run and run app again.
I would like to set number of repeats to ~30 or timeout to 30 minutes, to test at once as much as possible - currently I need to manually run app, wait for results, run it again...