search

ChrisTimperley / cb-multios

DARPA Challenges Sets for Linux, Windows, and macOS
https://blog.trailofbits.com/2016/08/01/your-tool-works-better-than-mine-prove-it/
MIT License
0 stars 0 forks source link

Failed tests do not produce a non-zero exit code #4

Closed ChrisTimperley closed 5 years ago

ChrisTimperley commented 5 years ago 
Running test: n1...                                           [UNEXPECTED: PASS]

    /challenge/tools /challenge
    # /challenge/build/challenges/AIS-Lite/pov_1.pov
    # using seed: 11df20aa7c7af21662f6a0d7d50ea39ac24581058c04ea1196a45347dd088430bec9c56f630583856ea7554c9743946c
    # negotiation type: 1
    # type 1 masks: 00000000 00000000
    # type 1 pov: 00000000 00000000 0
    # [DEBUG] pid: 61, sig: 11
    # Process generated signal (pid: 61, signal: 11)
    # register states - eip:8049934 fs:0 gs:63 esp:ff998164 ss:2b edi:0 eax:30303030 ebp:ff998168 es:2b cs:23 eflags:10206 edx:30303030 ebx:f7f29000 ds:2b esi:7 ecx:f7f3304c
    # END REPLAY
    FAIL: 1
ChrisTimperley commented 5 years ago

This test is roughly translated into the following shell command:

$ python cb-replay-pov.py "${test_file}" \
  --cbs "${bin_challenge}" \
  --negotiate \
  --pov_seed 0 \
  --timeout ${timeout}

where ${test_file} resolves to /challenge/build/challenges/AIS-Lite/pov_1.pov, ${bin_challenge} resolves to /challenge/build/challenges/AIS-Lite/AIS-Lite, and ${timeout} resolves to 20.

$ python cb-replay-pov.py       /challenge/build/challenges/AIS-Lite/pov_1.pov       --cbs /challenge/build/challenges/AIS-Lite/AIS-Lite       --negotiate       --pov_seed 0       --timeout 15