trailofbits / differ

Detecting Inconsistencies in Feature or Function Evaluations of Requirements
GNU Affero General Public License v3.0
67 stars 4 forks source link

Generic Trace Setup/Teardown #5

Closed ameily closed 1 year ago

ameily commented 1 year ago

Each trace hook subclass (Variable and Comparator) has setup and teardown hooks that are executed before a trace is run and after it has completed, respectively. We also need a similar definition in the trace template config to run commands in a configurable way, outside of a variable or comparator. For example, when testing a TCP client, we'll need to start the TCP server before running the client and then stop the server after the client has exited. This could be done with a couple new configuration options in the trace template:

templates:
  - setup: |
       echo starting server
       ./server --port 8080 &
       echo $! > ./server.pid

    teardown: |
      echo stopping server
      kill -9 $(cat server.pid)

Both setup and teardown commands must be run through jinja2 and support variables. Once generated, the commands must be saved to a bash script in the trace directory and run as part of the trace hook process.