search

eth-sri / ilf

AI based fuzzer based on imitation learning
Apache License 2.0
149 stars 32 forks source link

understanding logged data #1

Closed sunbeomso closed 4 years ago

sunbeomso commented 4 years ago

Hi, I have three questions on interpreting logs generated by ILF.

I have tried the following command in Usage of README.md:

$ python3 -m ilf --proj ./example/crowdsale/ --contract Crowdsale --fuzzer imitation --model ./model/ --limit 2000

and then I obtained the following logs:

[2019-12-30 15:07:15,768][1577718435.768150][ilf.fuzzers.environment][INFO] {"tx_count": 0, "num_contracts": 1, "insn_coverage": 0.0, "block_coverage": 0.0, "Crowdsale": {"tx_count": 0, "insn_coverage": 0.0, "covered_insns": 0, "all_insns": 525, "block_coverage": 0.0, "covered_blocks": 0, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,768][1577718435.768257][ilf.fuzzers.environment][INFO] initial calls start
[2019-12-30 15:07:15,777][1577718435.777859][ilf.fuzzers.environment][INFO] {"tx_count": 1, "num_contracts": 1, "insn_coverage": 0.022857142857142857, "block_coverage": 0.034482758620689655, "Crowdsale": {"tx_count": 1, "insn_coverage": 0.022857142857142857, "covered_insns": 12, "all_insns": 525, "block_coverage": 0.034482758620689655, "covered_blocks": 2, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,778][1577718435.778866][ilf.fuzzers.environment][INFO] {"tx_count": 2, "num_contracts": 1, "insn_coverage": 0.10095238095238095, "block_coverage": 0.15517241379310345, "Crowdsale": {"tx_count": 2, "insn_coverage": 0.10095238095238095, "covered_insns": 53, "all_insns": 525, "block_coverage": 0.15517241379310345, "covered_blocks": 9, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,779][1577718435.779850][ilf.fuzzers.environment][INFO] {"tx_count": 3, "num_contracts": 1, "insn_coverage": 0.1180952380952381, "block_coverage": 0.1896551724137931, "Crowdsale": {"tx_count": 3, "insn_coverage": 0.1180952380952381, "covered_insns": 62, "all_insns": 525, "block_coverage": 0.1896551724137931, "covered_blocks": 11, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,780][1577718435.780659][ilf.fuzzers.environment][INFO] {"tx_count": 4, "num_contracts": 1, "insn_coverage": 0.13523809523809524, "block_coverage": 0.22413793103448276, "Crowdsale": {"tx_count": 4, "insn_coverage": 0.13523809523809524, "covered_insns": 71, "all_insns": 525, "block_coverage": 0.22413793103448276, "covered_blocks": 13, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,781][1577718435.781489][ilf.fuzzers.environment][INFO] {"tx_count": 5, "num_contracts": 1, "insn_coverage": 0.1523809523809524, "block_coverage": 0.25862068965517243, "Crowdsale": {"tx_count": 5, "insn_coverage": 0.1523809523809524, "covered_insns": 80, "all_insns": 525, "block_coverage": 0.25862068965517243, "covered_blocks": 15, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,782][1577718435.782350][ilf.fuzzers.environment][INFO] {"tx_count": 6, "num_contracts": 1, "insn_coverage": 0.16952380952380952, "block_coverage": 0.29310344827586204, "Crowdsale": {"tx_count": 6, "insn_coverage": 0.16952380952380952, "covered_insns": 89, "all_insns": 525, "block_coverage": 0.29310344827586204, "covered_blocks": 17, "all_blocks": 58, "bugs": {}}}
[2019-12-30 15:07:15,782][1577718435.782432][ilf.fuzzers.environment][INFO] initial calls end
...
[2019-12-30 15:07:21,973][1577718441.973119][ilf.fuzzers.environment][INFO] {"tx_count": 74, "num_contracts": 1, "insn_coverage": 0.8114285714285714, "block_coverage": 0.896551724137931, "Crowdsale": {"tx_count": 74, "insn_coverage": 0.8114285714285714, "covered_insns": 426, "all_insns": 525, "block_coverage": 0.896551724137931, "covered_blocks": 52, "all_blocks": 58, "bugs": {"Leaking": ["withdraw"]}}}
[2019-12-30 15:07:22,076][1577718442.076097][ilf.fuzzers.environment][INFO] {"tx_count": 75, "num_contracts": 1, "insn_coverage": 0.8114285714285714, "block_coverage": 0.896551724137931, "Crowdsale": {"tx_count": 75, "insn_coverage": 0.8114285714285714, "covered_insns": 426, "all_insns": 525, "block_coverage": 0.896551724137931, "covered_blocks": 52, "all_blocks": 58, "bugs": {"Leaking": ["withdraw"]}}}
[2019-12-30 15:07:22,197][1577718442.197205][ilf.fuzzers.environment][INFO] {"tx_count": 76, "num_contracts": 1, "insn_coverage": 0.8114285714285714, "block_coverage": 0.896551724137931, "Crowdsale": {"tx_count": 76, "insn_coverage": 0.8114285714285714, "covered_insns": 426, "all_insns": 525, "block_coverage": 0.896551724137931, "covered_blocks": 52, "all_blocks": 58, "bugs": {"Leaking": ["withdraw"]}}}
...

[Question 1] For each transaction log, how can I know which function has been invoked with what argument values?

Does ILF have options for obtaining such information? The section 2.1 in the CCS '19 paper seems to suggest that ILF has this feature.

[Question 2] Are bugs in the logs accumulated ones or not?

Specifically, for example, is the bug in tx_count 75 the one that was already found in tx_count 74 or the one that is newly found in tx_count 75?

[Question 3] There is a log initial calls end after tx_count 6. What does initial calls end mean?

Environment

I used docker image in this git repository.

LostBenjamin commented 4 years ago

Hi,

Thanks for your interest in ILF. Below are answers for your three questions:

  1. One can obtain such information, .e.g, by looking at the fields of the variable tx at this file.
  2. Yes, they are accumulated.
  3. For each non-payable function, the compiler generates a branch for checking whether the amount is zero. The true branch is the function body and the false branch simply reverts. Our initial calls just call all non-payable functions with positive amount to cover those false branches.

Best, Jingxuan