Unexpected messages when running testing scripts in docker image

[Irene-ML]

Hi,

I downloaded selectFuzz docker image and tried to run provided bash scripts under scripts/fuzz by following the doc. The doc mentioned, "We have installed all required dependencies in Docker. You thus do NOT need to install it yourself.", so I got on (docker exec) the container, and just run those scripts like ./11728.sh under select fuzz/scripts/fuzz. However, I encountered something unexpected to me:

1. Some scripts (e.g. 20427.sh, 11728.sh, ./libming-CVE-2017-7578.sh) generated the following error, and fuzzing could not be triggered:

   
   aflgo (yeah!) 2.52b
   [+] Running with EXP schedule and time-to-exploitation set to 45 minutes
   [+] You have 5 CPU cores and 1 runnable tasks (utilization: 20%).
   [+] Try parallel jobs - see docs/parallel_fuzzing.txt.
   [*] Checking CPU core loadout...
   [+] Found a free CPU core, binding to #0.
   [*] Checking core_pattern...
   [*] Setting up output directories...
   [*] Scanning 'in'...
   [+] No auto-generated dictionary tokens to reuse.
   [*] Creating hard links for all input files...
   [*] Validating target binary...

[-] PROGRAM ABORT : Program './util/swftophp' not found or not executable Location : check_binary(), afl-fuzz.c:6964

and running ./libming-CVE-2016-9827.sh script with a similar error:

[-] PROGRAM ABORT : Program './util/listswf' not found or not executable Location : check_binary(), afl-fuzz.c:6964

so I guess I still need to install them all, such as`swftophp` and `listswf`?

2. Many "pinging" or handshake messages were shown when running the scripts, and they gave up on their attempts of making connections after 20 tries. It seems that they didn't affect fuzzing since some scripts (e.g. 2018-20427.sh) can still generate the fuzzing result report, so I'm confused about their purpose. Could you please kindly explain?

--2023-11-02 09:06:58-- (try:19) http://condor.depaul.edu/sjost/hci430/flash-examples/swf/bumble-bee1.swf Connecting to proxy.cse.cuhk.edu.hk (proxy.cse.cuhk.edu.hk)|137.189.90.217|:8000... connected. Proxy request sent, awaiting response... No data received. Retrying.

--2023-11-02 09:07:13-- (try:20) http://condor.depaul.edu/sjost/hci430/flash-examples/swf/bumble-bee1.swf Connecting to proxy.cse.cuhk.edu.hk (proxy.cse.cuhk.edu.hk)|137.189.90.217|:8000... connected. Proxy request sent, awaiting response... No data received. Giving up.

3. The following errors were repetitively shown when running script 4491.sh, and they never stopped until I killed it. Could you please kindly explain what was going on here?

476, 503@ 477, 503@ 478, 503@ 479, 503@ 480, 503@ 481, 503@ 482, 503@ 483, 503@ 484, 503@ 485, 503@ 486, 503@ 487, 503@ 488, 503@ 489, 503@ 490, 503@ 491, 503@ 492, 503@ 493, 503@ 494, 503@ 495, 503@ 496, 503@ 497, 503@ 498, 503@ 499, 503@ 500, 503@ 501, 503@ 502, 503@ 503, 503done first passdone dumpdone resolve data flowdone resolve distance(1) Constructing CG for /selectfuzz/scripts/fuzz/lrzip-CVE-2017-8846/obj-aflgo/a.out.. [!] Could not generate call graph. Repeating.. [!] Could not generate call graph. Repeating.. [!] Could not generate call graph. Repeating.. [!] Could not generate call graph. Repeating..

4. One more question from the [selectFuzz paper](https://seclab.cse.cuhk.edu.hk/papers/sp23_selectfuzz.pdf): Did selectFuzz also use source code as input along with binary code, or binary only? Source code was indicated in Fig2 in the paper, but I didn't find out how the source code was used in your repo. Maybe I mis-understood it. Could you please kindly explain and point me to scripts that can trigger testing examples of fuzzing with source code? 

Apologies for the long description and many question, and I'll really appreciate your answers 😃 

[chluo1997]

Hi,

For Q1 and Q2, I believe the reason is that you used our lib proxy (which should not be accessible to you). As a result, you cannot download the software repos and the seed inputs. Please follow the readme to unset proxy when using the docker.

For Q3, I am not sure of the reason cause I did not encounter it. Maybe you can try unsetting proxy and see if this works.

For Q4, SelectFuzz used source code as the input and compiled the source code with its modified compiler. If you check the scripts in selectfuzz/scripts/fuzz/*.sh, their logics are similar: 1: download the software repo; 2: compile the software and do static analysis to compute distance and identify relevant code; 3: compile the software again and instrument distance and coverage information to the relevant code.

[Irene-ML]

Thank you for all the answers! I'll unset proxy and see how it will go.

For Q4, I did notice downloading of the source code, so just to confirm, the fuzzer only used binaries (after compilation) for later analysis instead of combining information of source code and binaries, is that correct?

---

To follow up with unsetting the proxy: Q2 is resolved as packages (e.g. bumble-bee1) have been downloaded, but errors in Q1 still exist:

[-] PROGRAM ABORT : Program './util/swftophp' not found or not executable
         Location : check_binary(), afl-fuzz.c:6964

so I guess I have to download swftophp manually?

[chluo1997]

For Q1, this means you failed to compile the source code into binary. You can runmake clean all under selectfuzz folder and selectfuzz/llvm-mode folder. You might also run other similar scripts like libming-CVE-2018-8807.sh and see if they work.

For Q4, the fuzzer performs static analysis on the LLVM-IR level. We do not analyze binaries.

[Irene-ML]

Thank you again for your replies! After following your instructions for Q1, selectfuzz folder can be built successfully but selectfuzz/llvm-mode failed to build after I did make clean all under it:

root@2e3e4cf59060:/selectfuzz/llvm_mode# make clean all
rm -f *.o *.so *~ a.out core core.[1-9][0-9]* test-instr .test-instr0 .test-instr1 
rm -f ../afl-clang-fast ../afl-llvm-pass.so ../afl-llvm-rt.o ../afl-llvm-rt-32.o ../afl-llvm-rt-64.o ../afl-clang-fast++
[*] Checking for working 'llvm-config'...
[*] Checking for working 'clang'...
[*] Checking for '../afl-showmap'...
[+] All set and ready to build.
clang -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DBIN_PATH=\"/usr/local/bin\" -DVERSION=\"2.52b\"  afl-clang-fast.c -o ../afl-clang-fast 
ln -sf afl-clang-fast ../afl-clang-fast++
clang++ `llvm-config --cxxflags` -Wl,-znodelete -fno-rtti -fpic -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DVERSION=\"2.52b\" -Wno-variadic-macros -shared afl-llvm-pass.so.cc -o ../afl-llvm-pass.so `llvm-config --ldflags` 
warning: unknown warning option '-Wno-maybe-uninitialized'; did you mean '-Wno-uninitialized'? [-Wunknown-warning-option]
afl-llvm-pass.so.cc:597:24: warning: unused variable 'Zero' [-Wunused-variable]
          ConstantInt *Zero =
                       ^
afl-llvm-pass.so.cc:194:9: warning: unused variable 'min_distance' [-Wunused-variable]
  float min_distance = 99999;
        ^
afl-llvm-pass.so.cc:83:20: warning: unused function 'LLVMInstructionAsString' [-Wunused-function]
static std::string LLVMInstructionAsString(Instruction * I) {
                   ^
4 warnings generated.
clang -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DBIN_PATH=\"/usr/local/bin\" -DVERSION=\"2.52b\"  -fPIC -c afl-llvm-rt.o.c -o ../afl-llvm-rt.o
[*] Building 32-bit variant of the runtime (-m32)... success!
[*] Building 64-bit variant of the runtime (-m64)... success!
[*] Testing the CC wrapper and instrumentation output...
unset AFL_USE_ASAN AFL_USE_MSAN AFL_INST_RATIO; AFL_QUIET=1 AFL_PATH=. AFL_CC=clang ../afl-clang-fast -O3 -funroll-loops -Wall -D_FORTIFY_SOURCE=2 -g -Wno-pointer-sign -DAFL_PATH=\"/usr/local/lib/afl\" -DBIN_PATH=\"/usr/local/bin\" -DVERSION=\"2.52b\"  ../test-instr.c -o test-instr 
../afl-showmap -m none -q -o .test-instr0 ./test-instr < /dev/null
echo 1 | ../afl-showmap -m none -q -o .test-instr1 ./test-instr

Oops, the instrumentation does not seem to be behaving correctly!

Please ping <lcamtuf@google.com> to troubleshoot the issue.

Makefile:98: recipe for target 'test_build' failed
make: *** [test_build] Error 1

Please kindly instruct :)

For Q4, I found check_binary in afl-fuzz.c, so I guess the fuzzer was still fuzzing through binaries but your static analysis were done before code compiles. Is my understanding right?

[chluo1997]

1. The readme has mentioned that error message "recipe for target 'test_build' failed" can be ignored.
2. Yes.

[Irene-ML]

I tried to run libming-CVE-2018-8807.sh but I still got the same error for swftophp, and fuzzing did not happen. The error indicates ./util/swftophp is not found or not executable, but I did find swftophp in the util folder (./libming-CVE-2018-8807/obj-aflgo/util/swftophp).

[chluo1997]

That is weird. Some other people can run these cases directly. Did you check if the compiler works, and in which folder did you run libming-CVE-2018-8807.sh? I suggest that you can check libming-CVE-2018-8807.sh to find which step goes wrong. You can also check the distance files to see if there are any contents.

[Irene-ML]

I run libming-CVE-2018-8807.sh under selectfuzz/scripts/fuzz, and llvm-config --version is 4.0.0.

The error came up while running this line: /selectfuzz/afl-fuzz -m none -z exp -c 45m -i in -o out -d ./util/swftophp @@, and there is no distance.cfg.txt under libming-CVE-2018-8807 folder

[chluo1997]

I run libming-CVE-2018-8807.sh under selectfuzz/scripts/fuzz, and llvm-config --version is 4.0.0.

The error came up while running this line: /selectfuzz/afl-fuzz -m none -z exp -c 45m -i in -o out -d ./util/swftophp @@, and there is no distance.cfg.txt under libming-CVE-2018-8807 folder

If 'No distance.cfg.txt' is displayed, it signifies errors in the preceding step. Could you please provide additional information about the process of running this script?

[Irene-ML]

I checked scripts and found my mistake -- I modified libming-CVE-2018-8807.sh, which made the program not compile. After fixing the compilation, now libming-CVE-2018-8807.sh works!

Thank you for your patience and guidance!

[Irene-ML]

To follow up, each testing script has a git clone command with the following proxy.

user.email=you@example.com
https.proxy=http://proxy.cse.cuhk.edu.hk:8000
http.proxy=http://proxy.cse.cuhk.edu.hk:8000

To use the docker image directly, users may also unset the proxy in GitHub config:

git config --global --unset http.proxy
git config --global --unset https.proxy

(This is not mentioned in README, and it is the cause of my issue) Otherwise, git clone will fail with the following error:

fatal: unable to access 'https://github.com/libming/libming.git/': Proxy CONNECT aborted

Another way to fix this is to comment out all rm -rf and git clone command lines at the beginning of these scripts since the docker image has already provided those cloned folders.