jwilk / python-afl

American Fuzzy Lop fork server and instrumentation for pure-Python code
https://jwilk.net/software/python-afl
MIT License
350 stars 33 forks source link

Test case crashed but didn't crash when I run the program manually. #20

Closed Vagente closed 3 years ago

Vagente commented 3 years ago

Screenshot from 2021-07-14 10-07-18 The program simply takes an input, saves it to a variable, and prints out that variable. The test case is only one file with the number 43. I just downloaded the newest version of python-afl. Screenshot from 2021-07-14 10-15-03 Screenshot from 2021-07-14 10-14-55

Vagente commented 3 years ago

I just manually exported AFL_SKIP_BIN_CHECK=1 and AFL_DUMB_FORKSRV=1, then I run "afl-fuzz -i aflinputs -o afloutputs -- python3 wrapper.py @@". It successfully fuzzes the program. So I'm not sure why py-afl-fuzz failed.

Vagente commented 3 years ago

It seems "export PYTHON_AFL_SIGNAL=${PYTHON_AFL_SIGNAL:-SIGUSR1}" made the test case crash error happen because when I manually exported other variables and run afl-fuzz python3, it works fine, but the test case crashed when I export the above-mentioned variable( with and without the "@@" at the end of the command). I rewrote the code in c++ and use afl-fuzz to fuzz it, it works fine if I remove the "@@" at the end of the command. But I don't know what does PYTHON_AFL_SIGNAL do.

jwilk commented 3 years ago

You used @@, which means afl-fuzz provides path to testcase on the command line and redirects stdin to /dev/null. So your program dies with EOFError: EOF when reading a line or something.

PYTHON_AFL_SIGNAL makes the afl module install an exception hook that turns unhandled exceptions (such as EOFError mentioned above) into crashes, so that they're noticed by afl-fuzz. (This is documented in README.)

Vagente commented 3 years ago

You used @@, which means afl-fuzz provides path to testcase on the command line and redirects stdin to /dev/null. So your program dies with EOFError: EOF when reading a line or something.

PYTHON_AFL_SIGNAL makes the afl module install an exception hook that turns unhandled exceptions (such as EOFError mentioned above) into crashes, so that they're noticed by afl-fuzz. (This is documented in README.)

I mentioned that it says test case crashed with and without the "@@" at the end of the command Screenshot from 2021-07-14 14-08-16

Vagente commented 3 years ago

You used @@, which means afl-fuzz provides path to testcase on the command line and redirects stdin to /dev/null. So your program dies with EOFError: EOF when reading a line or something.

PYTHON_AFL_SIGNAL makes the afl module install an exception hook that turns unhandled exceptions (such as EOFError mentioned above) into crashes, so that they're noticed by afl-fuzz. (This is documented in README.)

It can correctly fuzz if I manually export AFL_SKIP_BIN_CHECK=1, AFL_DUMB_FORKSRV=1, and PYTHON_AFL_PERSISTENT=1, then run afl-fuzz. But it would fail and shows test case crash if I export PYTHON_AFL_SIGNAL=${PYTHON_AFL_SIGNAL:-SIGUSR1}

jwilk commented 3 years ago

I can't reproduce it here:

$ printf 'def test():\n    a = input()\n    print(a)\n\n' > main.py
$ printf 'import main\nimport afl\n\nafl.init()\n\nmain.test()\n' > wrapper.py
$ mkdir aflinputs
$ echo 43 > aflinputs/input1
$ py-afl-fuzz -i aflinputs -o afloutputs -- python3 wrapper.py 
afl-fuzz 2.57b by <lcamtuf@google.com>
[+] You have 1 CPU core and 1 runnable tasks (utilization: 100%).
[*] Checking core_pattern...
[*] Setting up output directories...
[*] Scanning 'aflinputs'...
[+] No auto-generated dictionary tokens to reuse.
[*] Creating hard links for all input files...
[*] Validating target binary...
[*] Attempting dry run with 'id:000000,orig:input1'...
[*] Spinning up the fork server...
[+] All right - fork server is up.
    len = 3, map size = 13, exec speed = 4236 us
[+] All test cases processed.

[+] Here are some useful stats:

    Test case count : 1 favored, 0 variable, 1 total
       Bitmap range : 13 to 13 bits (average: 13.00 bits)
        Exec timing : 4236 to 4236 us (average: 4236 us)

[*] No -t option specified, so I'll use exec timeout of 40 ms.
[+] All set and ready to roll!

(Retyping your code from screenshots was NOT fun. Please copy&paste text next time.)

Vagente commented 3 years ago

I restarted the computer and everything seems fine now, don't know what caused the issue earlier though. Issue closed.