Open abhinav opened 2 years ago
Could you try to compile the test binary using 'bazel coverage'?
If it does not work, we might want to create a special go_binary option to build fuzz targets 🤔
Could you try to compile the test binary using 'bazel coverage'?
If it does not work, we might want to create a special go_binary option to build fuzz targets thinking
I can confirm that the warning message about coverage still appears when running with bazel coverage.
bazel coverage //path/to:go_default_test --test_filter=Fuzz --test_arg=-test.fuzz=Fuzz --test_arg=-test.fuzzcachedir=/tmp/fuzz --test_output=streamed
warning: the test binary was not built with coverage instrumentation, so fuzzing will run without coverage guidance and may be inefficient
Edit:
rules_go @ v0.31.0 gazelle @ v0.25.0
I don't see that warning here, and fuzzing works, with the command line from @prestonvanloon 's last comment. I've also got --@io_bazel_rules_go//go/config:debug
in my .bazelrc.user
for build
, but that shouldn't make a difference.
rules_go @ v0.31.0 Go toolchain @ v1.18.1 Platform is arm64 macos
EDIT: I just re-checked the output of bazel coverage
and the warning is indeed there.
Just piling on here, I'm running into the same problem and haven't been able to find a workaround
Adding gc_goopts = ["-d=libfuzzer"]
to your go_test
rule ensures (edge) coverage is generated during compilation and subsequently used during fuzzing [1],
bazel run pkg/util:util_test -- -test.run notests -test.fuzz FuzzHex -test.fuzzcachedir /tmp/foobar/ -test.v
Executing tests from //pkg/util:util_test
-----------------------------------------------------------------------------
=== FUZZ FuzzHex
fuzz: elapsed: 0s, gathering baseline coverage: 0/12 completed
fuzz: elapsed: 0s, gathering baseline coverage: 12/12 completed, now fuzzing with 24 workers
fuzz: elapsed: 3s, execs: 2525728 (841880/sec), new interesting: 0 (total: 12)
fuzz: elapsed: 6s, execs: 5150923 (874883/sec), new interesting: 0 (total: 12)
fuzz: elapsed: 9s, execs: 7817339 (888760/sec), new interesting: 0 (total: 12)
...
[1] https://github.com/golang/go/blob/master/src/internal/fuzz/counters_supported.go#L15
gc_goopts = ["-d=libfuzzer"]
option seems no effects for other rules.
Set the option to the all go_binary and go_test is possible solution but it's better to specify string_list_flag like below.
bazel run --@io_bazel_rules_go//go/config:gc_goopts="-d=libfuzzer" pkg/util:util_test -- -test.run notests -test.fuzz FuzzHex -test.fuzzcachedir /tmp/foobar/ -test.v
What version of rules_go are you using?
v0.30.0
What version of gazelle are you using?
6bbfc47f1b0a27ee1efeddcc6671f3e4e03235dc
What version of Bazel are you using?
Does this issue reproduce with the latest releases of all the above?
Yes.
What operating system and processor architecture are you using?
and
What did you do?
Wrote a test with a simple fuzz-test.
I ran it with
bazel run :go_default_test
. This failed because of golang/go#51623. After patching the fix for that in,So I added the fuzzcachedir argument:
But wait, it's running without coverage guidance so I added
--collect_code_coverage
to instrument the binary but it still had the same issue:What did you expect to see?
Coverage-guided fuzzing.
What did you see instead?
Less efficient fuzzing.