search

junxzm1990 / x86-sok

124 stars 20 forks source link

Having some trouble with the compareCFG.py script #15

Open AnjaliPare opened 2 years ago

AnjaliPare commented 2 years ago

Hi,

I am having some trouble running the compareCFG script. The cfg edges in ground truth vs compared are significantly different and lead to a very low precision and recall. I am not sure what is wrong but just to double check I ran the compare script with the gtBlock_test_switch.pb both as the ground truth (-g file) and the compared (-c file). The cfg edges found in ground truth and compared do not match even if it is the same .pb file. I have noticed a similar behavior with other binaries in the test suite and other disassembler tools such as angr and ghidra. Below you will find the commands that I executed and the corresponding result. Any help would be appreciated. Thank you!

python3 compareCFG.py -b ../gt/test/test_switch -g /tmp/gtBlock_test_switch.pb -c /tmp/angr_blocks.pb2

INFO:root:[CFG Result]: All cfg edges in ground truth is 0 INFO:root:[CFG Result]: All cfg edges in compared is 21 INFO:root:[CFG Result]: False positive number is 21 INFO:root:[CFG Result]: False negative number is 0 INFO:root:[CFG Result]: Precision 0.000000 Traceback (most recent call last): File "compareCFG_orig.py", line 906, in compareCFG(edges_truth, edges_comp) File "compareCFG_orig.py", line 620, in compareCFG logging.info("[CFG Result]: Recall %f" % (true_pos / len(cfg_ground))) ZeroDivisionError: division by zero

bin2415 commented 2 years ago

Hello, thanks for the reporting! I updated the script of comparing CFG. Note that our CFG do not split basic block at call site. That means we do not consider call site as the edge of CFG.