search

trailofbits / deepstate

A unit test-like interface for fuzzing and symbolic execution
Apache License 2.0
814 stars 99 forks source link

Define a testing workflow similar to orthrus #294

Open neuromancer opened 4 years ago

neuromancer commented 4 years ago

https://github.com/test-pipeline/orthrus

ex0dus-0x commented 4 years ago

Cool recommendation!

One thing that kind of works towards improving the workflow aspect of the framework is with how we manage parsing configurations by including a reserved manifest section, which can define steps that can be used to provision an environment (ie installing particular dependencies, and performing necessary build-related commands):

[manifest]
name = MyTest
executor = afl

provision_steps = [
    "mkdir build/ && cd build/",
    "cmake ..",
    "make"]

[compile]
compile_test = Test.cpp
compiler_args = -fsanitize=address -I../cpp -I../include -L../build
out_test_name = test

[test]
input_seeds = input
output_test_dir = out

timeout = 3600
num_workers = 2
mem_limit = 100

At the current moment, DeepState does not really care about manifest, but does recognize it part of the configuration parser routine. It is more for an auxiliary orchestrator to use when building tests for cloud and/or container fuzzing. In the future, it would definitely be nice to have something similar to the orthrus pipeline where we can run different "execution components" of the test (ie run provisioning and compilation, but not actually fuzz/symbolically execute), and with a more robust and modern configuration to be consumed (ie YAML).