Front-ends to new fuzzers

[agroce]

We currently have a dedicated front-end to Eclipser (https://github.com/trailofbits/deepstate/blob/master/bin/deepstate/eclipser.py), which makes it easy to use Eclipser with DeepState. However, even using AFL (for example) requires a bit more typing and user suffering. In the case of AFL, it's not urgent, but there are likely other fuzzers (esp. file-based ones) where figuring out the right command line, or putting the tests generated in a useful location, is more difficult.

[ex0dus-0x]

It seems like a lot of fuzzers work similarly, and that a lot of the front-ends would end up looking almost the same. Would it be a good idea to implement a base DeepStateFrontend object in bin/deepstate/common.py that does the work of consuming CLI args, and creating and calling the fuzzer command passed to subprocess.call?

Not entirely sure how the whole front-end base API would work, but here would be a rough example for writing a front-end for Angora:

class Angora(DeepStateFrontend):

    def __init__(self, fuzzer_envvar):
        """
    the base object would determine path to
    fuzzer binary based on envvar, and perform
    other checks
    """
    super().__init__(fuzzer_envvar)

    def make_command(self, args_dict):
        """
    maps dict() of deepstate-angora cli args
    to cli command for fuzzer. Implementer can override
    if arg flags differ from default
    """
    super(Angora, self).make_command(args_dict)

    def execute(self):
        """
    takes fuzzer and command from make_command() and
    executes with subprocess call
    """
    super(Angora, self).execute()

One feature that I think this could also introduce would be the ability to interface compile-time instrumentation through the front-end (for the fuzzers that support it).

[agroce]

I'll have to think, but this sounds reasonable. Can you elaborate on what we might do with compile-time instrumentation here?

[ex0dus-0x]

For example, maybe something like this:

$ deepstate-angora --compile harness.c

The front-end finds the path to angora-clang, (or angora-clang++ for .cpp) and compiles accordingly. A corresponding method in the base DeepStateFrontend object could look like this:

class Angora(DeepStateFrontend):

    ...
    def compile_and_instrument(self, test_file):
        self.compiler_path = self.fuzzer_path + "/bin/angora_clang"
        # ... subprocess.call to perform compilation and instrumentation
        # with the appropriate flags

This goes back to the idea of using DeepState to interface a lot of the steps needed to perform fuzzing, so a user would just need to use a deepstate-{fuzzer_name} front-end to do both compilation and the actual fuzz testing.

[agroce]

Getting it to work with more complex build stuff will be tricky, but for single files etc. yeah makes a lot of sense

[ggrieco-tob]

Can we also include relevant functions to address #195? (for instance a function to save and load inputs for every fuzzer/symbolic execution engine)

[agroce]

Seems reasonable. a "grab current inputs" and "reseed" API

[agroce]

Then implement ensemble at a higher level of abstraction

[ggrieco-tob]

Keep in mind that we need to do some additional research for "concolic execution", since I don't think Manticore has an API for that (but maybe angr has?)

[agroce]

Sure, though we can make a fuzzer ensemble without using any symex (other than what Eclipser and angora do)