hunter-ht-2018 / ptfuzzer

Improving AFL by using Intel PT to collect branch information
289 stars 54 forks source link

The compatible problem for the platform of ubuntu 18.04lts kernel 4.15.0-23-generic and xeon E5-1650 v4 #4

Closed nemo5566 closed 5 years ago

nemo5566 commented 6 years ago

When running the test case of readelf on the platform of ubuntu 18.04lts kernel 4.15.0-23-generic and xeon E5-1650 v4, I got the waring "set filter for fd 9 failed, hardware ip filter may not supported." The ptfuzz stopped at /home/ly/ptfuzzer/pt/pt.h:338 void pt_packet_decoder::tip_pge_handler(uint8_t, uint8_t): Assertion `this->last_tip == 0' failed. Aborted

zhang-li commented 6 years ago

I tried on Ubuntu 16.04 and Intel core i7-5600U CPU, and had the same problem.

I doubt it is the compatibility problem. May I know how can it be resolved? Thanks.

binary type is executable Program base by cle: 0x0 Program entry by cle: 0x402420 sudo ./bin/afl-ptfuzz -r .readelf.text -l 4200800 -h 4503506 -e 4203552 -i ptest/in -o ptest/out ./ptest/readelf -a @@ afl-fuzz 2.52b by lcamtuf@google.com raw_bin: .readelf.text min_addr: 4200800 max_addr: 4503506 entry_point: 4203552 init pt fuzzer. Run ptfuzzer with TNT_MODE [+] You have 4 CPU cores and 1 runnable tasks (utilization: 25%). [+] Try parallel jobs - see docs/parallel_fuzzing.txt. [] Checking CPU core loadout... [+] Found a free CPU core, binding to #0. [] Checking core_pattern... [] Checking CPU scaling governor... [] Setting up output directories... [+] Output directory exists but deemed OK to reuse. [] Deleting old session data... [+] Output dir cleanup successful. [] Scanning 'ptest/in'... [+] No auto-generated dictionary tokens to reuse. [] Creating hard links for all input files... [] Validating target binary... [*] Attempting dry run with 'id:000000,orig:small_exec.elf'... Warning: set filter for fd 9 failed, hardware ip filter may not supported. We stop trying to set ip filter again. afl-ptfuzz: /home/li/ptfuzzer-master/pt/pt.h:338: void pt_packet_decoder::tip_pge_handler(uint8_t, uint8_t): Assertion `this->last_tip == 0' failed. Aborted

zhanggenex commented 6 years ago

@nemo5566 @zhang-li about the ip filter, please refer to Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3 (3A, 3B, 3C & 3D): System Programming Guide, chapter 35.2.4.3 And we use msr tools to set the certain registers (https://01.org/zh/msr-tools?langredirect=1) in PTfuzzer This tool may not run on some configurations.

zhanggenex commented 6 years ago

@nemo5566 The cause may be the Xeon CPU. Because we only tested on Core CUPs, and we know that Intel Core CPU i5/6/7-x000, x >= 5 can work.

zhanggenex commented 6 years ago

@zhang-li The cause may be the version of Linux kernel. It should >= 4.7.0.

zhang-li commented 6 years ago

@zhanggenex Thanks for your reply. The linux kernel of my Ubuntu is 4.13.0, hence I think it should satisfy the requirement.

I found in the config-run.sh file that I should actually run "sudo modprobe msr". However, when I run it, I got a new error (see below). Will you please share with me what is the possible reason?

binary type is executable Program base by cle: 0x0 Program entry by cle: 0x402420 sudo ./bin/afl-ptfuzz -r .readelf.text -l 4200800 -h 4503506 -e 4203552 -i ptest/in -o ptest/out ./ptest/readelf -a @@ afl-fuzz 2.52b by lcamtuf@google.com raw_bin: .readelf.text min_addr: 4200800 max_addr: 4503506 entry_point: 4203552 init pt fuzzer. wrmsr: CPU 0 cannot set MSR 0x00000580 to 0x0000000000401960

zhanggenex commented 6 years ago

@zhang-li Yes you should run config-run.sh before you start fuzzing. The possible reason for this is that currently PTfuzzer can only support one fuzzing task. If you already have a fuzzing task, and you start another job without kill the previous task, this error may come out. I have met this error before.

zhang-li commented 6 years ago

@zhanggenex Thanks. The reason for my error seems to be that the Intel core i7-5600U CPU of my laptop does not have a register with address 0x580 .. I tried with the command "sudo rdmsr 0x580" and it returns "CPU 0 cannot read MSR 0x00005800". I changed to use another PC (with Intel core i7-6700 CPU), and finally I can run the program.

Think that this issue can be closed. Thanks again. And by the way, may I ask how you measure the number of branches covered? Is it simply the reported "total paths" ?

zhanggenex commented 6 years ago

@zhang-li Yes, "total path". Although it is not exactly number of branches, we can use it as an indication of code coverage.

BinLeeBit commented 6 years ago

@zhanggenex My CPU is Intel® Core™ i5-6500. I met the same error. _pt.h:334: void pt_packet_decoder::tnt8_handler(uint8_t**): Assertion `this->pgeenabled' failed. Aborted I opened two fuzzer(a master and a slave). According to AFL, we can open many slave while PC have multicore. thx.

zhanggenex commented 6 years ago

@zhanggenex My CPU is Intel® Core™ i5-6500. I met the same error. _pt.h:334: void pt_packet_decoder::tnt8_handler(uint8_t**): Assertion `this->pgeenabled' failed. Aborted I opened two fuzzer(a master and a slave). According to AFL, we can open many slave while PC have multicore. thx.

Due to the filter mechanism in PTFuzzer, it can only fuzz one target at the same time, which is different to AFL.

BinLeeBit commented 6 years ago

I mean running a master-fuzzer and a slave-fuzzer at the same time for one target. I fuzzed who(complied by gcc), and afl-slave can import the inputs of afl-master. But two fuzzer stopped when they run 45 min. Besides, I only run one fuzzer for one targret, I get the same error after 8 hours.

_pt.h:274: void pt_packet_decoder::tip_pge_handler(uint8_t, uint8_t): Assertion `this->lasttip == 0' failed. Aborted