bazelbuild / rules_fuzzing

Bazel Starlark extensions for defining fuzz tests in Bazel projects
Apache License 2.0
82 stars 19 forks source link

README is misleading: v.0.1.3 does not support rules_fuzzing_dependencies(jazzer=True) #164

Closed eustas closed 2 years ago

eustas commented 3 years ago

Expected Behavior

rules_fuzzing_dependencies(jazzer=True) should enable jazzer support

Actual Behavior

Error is reported:


ERROR: Traceback (most recent call last):
    File "/Users/eustas/clients/nopb-worker/fuzz/WORKSPACE", line 17, column 27, in <toplevel>
        rules_fuzzing_dependencies(jazzer=True)
    File "/private/var/tmp/_bazel_eustas/a561f35c1ba825cd6dc752e3267f99ab/external/rules_fuzzing/fuzzing/repositories.bzl", line 21, column 5, in rules_fuzzing_dependencies
        def rules_fuzzing_dependencies(oss_fuzz = True, honggfuzz = True):
Error: rules_fuzzing_dependencies() got unexpected keyword argument: jazzer
``1

## Steps to Reproduce the Problem

1. Follow README.md instructions, activate jazzer on most recent release (0.1.3)
stefanbucur commented 3 years ago

Thank you for flagging this. The Jazzer work was actually just recently wrapped up by @fmeum and I'm planning to cut a new release very soon!

stefanbucur commented 3 years ago

In the mean time, if you'd like to depend on this functionality, it should be okay to depend on the mainline tree at HEAD, and later update to a stable release number.

eustas commented 3 years ago

Switched to head. Still no luck: https://github.com/eustas/nopb-worker/runs/3032022445?check_suite_focus=true#step:5:13

ERROR: Skipping '//fuzzing:java_engine': no such package 'fuzzing': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
 - /home/runner/work/nopb-worker/nopb-worker/fuzz/fuzzing
WARNING: Target pattern parsing failed.
INFO: Found applicable config definition build:jazzer in file /home/runner/work/nopb-worker/nopb-worker/fuzz/.bazelrc: --//fuzzing:java_engine=//fuzzing/engines:jazzer --@rules_fuzzing//fuzzing:cc_engine_instrumentation=jazzer --@rules_fuzzing//fuzzing:cc_engine_sanitizer=none --//fuzzing:cc_engine_sanitizer=none
ERROR: //fuzzing:java_engine :: Error loading option //fuzzing:java_engine: no such package 'fuzzing': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
stefanbucur commented 3 years ago

Hmm, that's unexpected! Let me look into it.

stefanbucur commented 3 years ago

Is this the fuzz target that I can try out? https://github.com/eustas/nopb-worker/blob/5a5c5e4e83d0a04a59f5ccecb220ce31ace670f7/fuzz/BUILD

eustas commented 3 years ago

Yup, this one.

fmeum commented 3 years ago

Good catch! I should have cleanly separated the flags in .bazelrc that are only meant for internal use within rules_fuzzing.

@eustas Can you take a look at https://github.com/bazelbuild/rules_fuzzing/pull/165 and remove the lines containing -//fuzzing:cc_engine_sanitizer (without the leading @rules_fuzzing) from your .bazelrc?

eustas commented 3 years ago

Unfortunately, same error report.

fmeum commented 3 years ago

@stefanbucur The error results from the .bazelrc line setting the java_engine without an @rules_fuzzing prefix. This does seem to be a problem for all the cc_engine configs as well though. Maybe the .bazelrc should be split into a part that can be safely copy-pasted into a project's own .bazelrc and an internal part?

@eustas Could you try using

build:jazzer --@rules_fuzzing//fuzzing:java_engine=//fuzzing/engines:jazzer

instead of

build:jazzer --//fuzzing:java_engine=//fuzzing/engines:jazzer

?

stefanbucur commented 3 years ago

@stefanbucur The error results from the .bazelrc line setting the java_engine without an @rules_fuzzing prefix. This does seem to be a problem for all the cc_engine configs as well though. Maybe the .bazelrc should be split into a part that can be safely copy-pasted into a project's own .bazelrc and an internal part?

I don't see the purpose of the .bazelrc file in our own project as copy-pasteable. I would prefer this to be covered in the documentation, so the users don't have to resort to finding and copy-pasting code from our own repository.

stefanbucur commented 3 years ago

@eustas Could you try using

build:jazzer --@rules_fuzzing//fuzzing:java_engine=//fuzzing/engines:jazzer

instead of

build:jazzer --//fuzzing:java_engine=//fuzzing/engines:jazzer

?

Note that both labels (//fuzzing:java_engine and //fuzzing/engines:jazzer) need to be prefixed. This should be: build:jazzer --@rules_fuzzing//fuzzing:java_engine=@rules_fuzzing//fuzzing/engines:jazzer

I have just sent out #166 which clarifies these aspects in the README file. I have followed the tutorial myself in a new repository and "It Works on My Machine". @eustas can you perhaps try it out?

eustas commented 3 years ago

Woo-hoo, works now (following #166 how-to). Thanks.