Open cloudhead opened 4 years ago
@pop if anything above looks interesting, let me know and I can provide more detail. Would be great to have some help in making rx more robust.
If I had to pick one, fuzz-testing the commands that take user input seems interesting. i.e.,
session.rs: all commands that take user input should be tested with unexpected inputs to ensure there is no division by zero for example
That said I have not dug too far into any of the topics so if there is something else you'd like to pass off to a contributor I'm open to one of the other topics.
I'm not entirely sure how to do headless testing but I'm sure once I get started writing the headless fuzz-tests would be pretty straight forward.
Great, I also think the fuzz testing will be the biggest ROI at this point, but it's also possibly more work than the others.
A good place to start looking is the execution
module and the current tests in tests/
, as there's already the ability to run rx in headless mode and just feed it inputs from a script. There's two levels at which the fuzz testing could be useful:
Command
objects with random parameters and feed them into the Session
and try to make it crash.Both might be useful, but 2. is less likely to trigger complex behavior, since most inputs won't be valid commands.
Another thing that might be helpful alongside this is to add another input which is the current session mode
, so run the fuzz tests in Visual
, Normal
and Command
mode; I think this will increase coverage quite a bit.
Finally there's the question of whether to turn the renderer and platform layer off or not. I think to start out it would make sense, ie. to use a dummy renderer (trait Renderer
) and context manager (platform
). This way the fuzz tests can run 100x faster and test many more inputs.
These are my initial thoughts.
parser.rs
: the parser should be property tested with something likequickcheck
cmd.rs
:CommandLine
should be tested with ascii and utf8resources.rs
: Gif export should be testedsession.rs
: inputs (mouse, keyboard) should be fuzz-tested to ensure nothing can crash the program.session.rs
:selection/*
commands should be tested in headless modesession.rs
: all commands that take user input should be tested with unexpected inputs to ensure there is no division by zero for examplelib.rs
: rx's entry point should be tested with different inputs[modified]
flag on it