Concurrent hook

[ameily]

Adds a new concurrent hook that runs a bash script while the trace is running. The use case is initially for testing network server binaries where we'll need to run a client that issues some commands to the server. For example, a concurrent script that uses wget to download a file from the server binary and then compares the concurrent script output (stdout/stderr and exit code) and the downloaded file content:

# The concurrent commands to execute while the trace is running. This will be our call to
# wget to download the file
concurrent:
  delay: 2.0  # allow the HTTP server 2 seconds to start up prior to running wget
  run: |
    wget http://localhost:8080/test.html --output-document downloaded.html
    rc=$?

    # We downloaded the file, we can safely terminate the server
    kill $(DIFFER_BINARY_PID)

    # exit with the exit code of wget to trigger an error if wget failed
    exit $rc

# Finally, the list of comparators we run
comparators:
  # Verify the exit code of the server matches
  - exit_code
  # Verify the stdout and stderr content of the server matches
  - stdout
  - stderr
  # Verify that the stdout/stderr content and the exit code of the concurrent bash script
  # (wget) matches
  - concurrent_script
  # Verify the contents of the downloaded file
  - id: file
    filename: downloaded.html

Fixes #13

[ameily]

I'm testing using netcat:

name: netcat
original: /usr/bin/nc
debloaters:
  chisel: /usr/bin/nc

templates:
  # Server
  - arguments: -l 8080
    concurrent: |
      echo 'hello world!' | nc -N 127.0.0.1 8080
      exit $?

    comparators:
      - stdout
      - stderr
      - exit_code
      - concurrent_script

  # Client
  - arguments: -N 127.0.0.1 8080
    setup: |
      echo 'starting server'
      nc -l 8080 > server-out.bin 2>&1 &

    stdin:
      value: 'hello world!'

    comparators:
      - stdout
      - stderr
      - exit_code
      - setup_script
      - id: file
        filename: server-out.bin

I am getting some inconsistent results when testing the client, I'm guessing because the server is launched in a background process and potentially terminated before it flushes stdout to the server-out.bin file. Adding a teardown: sleep 1 appears to fix the issue by allowing the server time to flush and exit cleanly.

There is another issue where I think the client is timing out if it is execute too soon (e.g.- the concurrent script delay time is not working properly.) I'll look into this next.

[ameily]

Adding more sleeps resolves the issues Im seeing.

• Testing a server application:
  • Sleep prior to launching the client (concurrent.delay configuration option)
• Testing a client application:
  • Sleep after starting the server within the setup script (manual call to sleep X)
  • Launch the server as a background process and redirect server output to a file so that it is properly captured
  • Sleep during the teardown script so that the server has time to flush output and cleanly exit (manual call to sleep X)

There may be an opportunity to add more configuration options within a project so automate some of these delays where possible so that there aren't sleep copy/pasted all over a project configuration.

[ameily]

I've got this working reliably, with necessary sleeps in place, and I've added a recipe for testing a network server and client binary, both targeting netcat so that it is easily reproducible (copy/paste).