Unable to instrument binary

[appetrosyan]

Description

When running cargo test-fuzz, the instrumentation seems to be absent.

MWE

Steps to reproduce

1. cargo install cargo-test-fuzz afl
2. in the cargo-test-fuzz examples directory, run cargo test
3. after successful generation run cargo test-fuzz "try_from".

Expected

AFL TUI to show the fuzzing progress.

Actual

[2021-12-16T10:59:42Z DEBUG cargo_test_fuzz] Exec { cargo afl test --frozen --offline --no-run --target-dir /mnt/Archive/Git/test-fuzz/target/afl '--message-format=json' }
Finished test [unoptimized + debuginfo] target(s) in 0.08s
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/auto_generate-25fe5e015ec2bc50 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/hello_world-501063fb5245c849 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/arc-9cb6a89229042858 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/test_fuzz_impl-63162cae74bfdbd6 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/assert-18f4da356afbee5b --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/associated_type-61dacc3c9dc0f92e --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/serde-e3a6d096678c3fa8 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/from-a434498923de99b1 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/conversion-9654d39df6edc403 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/auto_concretize_0-a5e1432595113a7d --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/default-d19364996163e2dc --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/alloc-e2ad00527be7d720 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/lifetime-0f33746c18aa3b02 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/rename-664eb63d7a65faa7 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/generic-080e6563331d2ab5 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/debug-557da8c7bf5eeb34 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/parse_duration-44d06c29a46b55f6 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/qwerty-43423f0c88b7d27c --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/return_type-cdd07c8838778238 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] Exec { AFL_QUIET=1 /mnt/Archive/Git/test-fuzz/target/afl/debug/deps/unserde-c79973c39769ed84 --list }
[2021-12-16T10:59:43Z DEBUG cargo_test_fuzz] "cargo" "afl" "fuzz" "-i" "/mnt/Archive/Git/test-fuzz/target/corpus/from::try_from" "-o" "/mnt/Archive/Git/test-fuzz/target/afl/output/from::try_from" "-D" "-M" "default" "--" "/mnt/Archive/Git/test-fuzz/target/afl/debug/deps/from-a434498923de99b1" "--exact" "try_from_fuzz::entry"
afl-fuzz++3.14c based on afl by Michal Zalewski and a large online community
[+] afl++ is maintained by Marc "van Hauser" Heuse, Heiko "hexcoder" Eißfeldt, Andrea Fioraldi and Dominik Maier
[+] afl++ is open source, get it at https://github.com/AFLplusplus/AFLplusplus
[+] NOTE: This is v3.x which changes defaults and behaviours - see README.md
[*] Getting to work...
[+] Using exponential power schedule (FAST)
[+] Enabled testcache with 50 MB
[*] Checking core_pattern...
[*] Checking CPU scaling governor...
[+] You have 12 CPU cores and 2 runnable tasks (utilization: 17%).
[+] Try parallel jobs - see docs/parallel_fuzzing.md.
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Checking CPU core loadout...
[+] Found a free CPU core, try binding to #0.
[*] Scanning '/mnt/Archive/Git/test-fuzz/target/corpus/from::try_from'...
[+] Loaded a total of 1 seeds.
[*] Creating hard links for all input files...
[*] Validating target binary...

[-] Looks like the target binary is not instrumented! The fuzzer depends on
compile-time instrumentation to isolate interesting test cases while
mutating the input data. For more information, and for tips on how to
instrument binaries, please see docs/README.md.

When source code is not available, you may be able to leverage QEMU
mode support. Consult the README.md for tips on how to enable this.

If your target is an instrumented binary (e.g. with zafl, retrowrite,
etc.) then set 'AFL_SKIP_BIN_CHECK=1'

(It is also possible to use afl-fuzz as a traditional, non-instrumented
fuzzer. For that use the -n option - but expect much worse results.)

[-] PROGRAM ABORT : No instrumentation detected
Location : check_binary(), src/afl-fuzz-init.c:2749

Error: Command failed: "cargo" "afl" "fuzz" "-i" "/mnt/Archive/Git/test-fuzz/target/corpus/from::try_from" "-o" "/mnt/Archive/Git/test-fuzz/target/afl/output/from::try_from" "-D" "-M" "default" "--" "/mnt/Archive/Git/test-fuzz/target/afl/debug/deps/from-a434498923de99b1" "--exact" "try_from_fuzz::entry"

Workarounds

running

AFL_SKIP_BIN_CHECK=1 /usr/bin/cargo test-fuzz "try_from"

doesn't help. The binary really isn't instrumented.

[smoelius]

Thank you for the report. I am able to reproduce this. I look into it as soon as I can.

[smoelius]

This was due to a problem in afl.rs that should now be fixed: https://github.com/rust-fuzz/afl.rs/pull/197

Perhaps you would like to try again?

[appetrosyan]

Very much so!

[appetrosyan]

Ok. So that particular error message is gone, but not it times out on the dry run.

afl-fuzz++3.14c based on afl by Michal Zalewski and a large online community
[+] afl++ is maintained by Marc "van Hauser" Heuse, Heiko "hexcoder" Eißfeldt, Andrea Fioraldi and Dominik Maier
[+] afl++ is open source, get it at https://github.com/AFLplusplus/AFLplusplus
[+] NOTE: This is v3.x which changes defaults and behaviours - see README.md
[*] Getting to work...
[+] Using exponential power schedule (FAST)
[+] Enabled testcache with 50 MB
[!] WARNING: LD_PRELOAD is set, are you sure that is what to you want to do instead of using AFL_PRELOAD?
[*] Checking core_pattern...
[*] Checking CPU scaling governor...
[+] You have 16 CPU cores and 1 runnable tasks (utilization: 6%).
[+] Try parallel jobs - see docs/parallel_fuzzing.md.
[*] Setting up output directories...
[+] Output directory exists but deemed OK to reuse.
[*] Deleting old session data...
[+] Output dir cleanup successful.
[*] Checking CPU core loadout...
[+] Found a free CPU core, try binding to #0.
[*] Scanning '/mnt/Archive/Git/iroha/target/corpus/integration::integration_tests::add_account::register_account'...
[+] Loaded a total of 1 seeds.
[*] Creating hard links for all input files...
[*] Validating target binary...
[*] Spinning up the fork server...
[+] All right - fork server is up.
[*] Target map size: 627184
[*] No auto-generated dictionary tokens to reuse.
[*] Attempting dry run with 'id:000000,time:0,orig:1489f923c4dca729178b3e3233458550d8dddf29'...
[!] WARNING: Test case results in a timeout (skipping)

[-] PROGRAM ABORT : All test cases time out or crash, giving up!
Location : perform_dry_run(), src/afl-fuzz-init.c:1128

Error: Command failed: "cargo" "afl" "fuzz" "-i" "/mnt/Archive/Git/iroha/target/corpus/integration::integration_tests::add_account::register_account" "-o" "/mnt/Archive/Git/iroha/target/afl/output/integration::integration_tests::add_account::register_account" "-D" "-M" "default" "-t" "10000" "--" "/mnt/Archive/Git/iroha/target/afl/debug/deps/integration-da4d11613a91490a" "--exact" "integration_tests::add_account::register_account_fuzz::entry"

[appetrosyan]

Ok. This is not a cargo-test fuzz issue. I just tested on the examples, and they work.

[smoelius]

Thanks for checking, @appetrosyan.

[appetrosyan]

I'm afraid it's recurring. The upgrade to 1.58, seems to be the culprit.

[smoelius]

Thanks. I'll look into this.

[smoelius]

Sorry, but are you sure this is the case? I can't seem to reproduce this.

Also, this problem was previously caused by rust 1.58.0 not using the new LLVM pass manager, as afl.rs expected it would. But AFAICT, rust 1.58.1 also does not use the new LLVM pass manager.

[appetrosyan]

After upgrading to 1.58.1 (or downgrading to 1.57) the issue is indeed resolved.