search

bazel-contrib / rules_fuzzing

Bazel Starlark extensions for defining fuzz tests in Bazel projects
Apache License 2.0
87 stars 20 forks source link

Simple fuzzing project doesn't work (pypi__enum34 issue) #207

Open matta opened 2 years ago

matta commented 2 years ago

Expected Behavior

I've copy-pasted the example from https://github.com/bazelbuild/rules_fuzzing#readme into a trivial project. I should be able to bazel run --config=asan-libfuzzer //:fuzz_test_run but I can't get it to work.

Actual Behavior

bazel run --config=asan-libfuzzer //:fuzz_test_run
Loading: 
Loading: 0 packages loaded
Analyzing: target //:fuzz_test_run (0 packages loaded, 0 targets configured)
ERROR: /home/matt/.cache/bazel/_bazel_matt/d58dd01668b145fbe0a002f4953a11e2/external/fuzzing_py_deps/pypi__absl_py/BUILD:5:11: no such package '@fuzzing_py_deps//pypi__enum34': BUILD file not found in directory 'pypi__enum34' of external repository @fuzzing_py_deps. Add a BUILD file to a directory to mark it as a package. and referenced by '@fuzzing_py_deps//pypi__absl_py:pypi__absl_py'
ERROR: Analysis of target '//:fuzz_test_run' failed; build aborted: 
INFO: Elapsed time: 0.062s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 3 targets configured)
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully (0 packages loaded, 3 targets configured)

Steps to Reproduce the Problem

  1. git clone https://github.com/matta/bazel-fuzz-example
  2. cd bazel-fuzz-example
  3. bazel run --config=asan-libfuzzer //:fuzz_test_run

Specifications

bazel version
Bazelisk version: v1.10.1
Build label: 5.2.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jun 7 16:02:26 2022 (1654617746)
Build timestamp: 1654617746
Build timestamp as int: 1654617746
cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux bookworm/sid"
NAME="Debian GNU/Linux"
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
clang --version
Debian clang version 14.0.6-2
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
stefanbucur commented 2 years ago

Thanks for reporting this issue! Just wanted to first rule out a Bazel compatibility issue.

If you pin the Bazel version to 4.2.0 (use this string in a .bazelversion file), are you still getting this error? (Make sure to bazel clean --expunge first.)

matta commented 2 years ago

Pinning to Bazel 4.2.2 doesn't change the result:

# echo 4.2.2 > .bazelversion

# bazel run --config=asan-libfuzzer //:fuzz_test_run
2022/08/15 09:24:04 Downloading https://releases.bazel.build/4.2.2/release/bazel-4.2.2-linux-x86_64...
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Loading:  (1 packages loaded)
Loading: 1 packages loaded
Analyzing: target //:fuzz_test_run (2 packages loaded, 0 targets configured)
Analyzing: target //:fuzz_test_run (5 packages loaded, 5 targets configured)
Analyzing: target //:fuzz_test_run (19 packages loaded, 68 targets configured)
ERROR: /home/matt/.cache/bazel/_bazel_matt/d58dd01668b145fbe0a002f4953a11e2/external/fuzzing_py_deps/pypi__absl_py/BUILD:5:11: no such package '@fuzzing_py_deps//pypi__enum34': BUILD file not found in directory 'pypi__enum34' of external repository @fuzzing_py_deps. Add a BUILD file to a directory to mark it as a package. and referenced by '@fuzzing_py_deps//pypi__absl_py:pypi__absl_py'
ERROR: Analysis of target '//:fuzz_test_run' failed; build aborted: Analysis failed
INFO: Elapsed time: 7.769s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (24 packages loaded, 74 targets configured)
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully (24 packages loaded, 74 targets configured)

# bazel version
Bazelisk version: v1.10.1
Build label: 4.2.2
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Dec 2 18:15:58 2021 (1638468958)
Build timestamp: 1638468958
Build timestamp as int: 1638468958
matta commented 2 years ago

...same result after a bazel clean --expunge. And with bazel version 4.2.0

stefanbucur commented 2 years ago

Thanks for confirming! Will take a closer look and follow back here.

matta commented 2 years ago

So, I installed Debian 11 (stable) in a VM, installed git, cloned my example repo, installed homebrew, then bazelisk from there, then clang from Debian's repo, then installed Debian's build-essential and python3-distutils packages, and it works!

I suspect that the problem I am having on my original Debian Testing workstation is some interaction with the way Python is installed there.

Any idea how I might set up a Bazel project to effectively use a completely hermetic Python installation?

fmeum commented 2 years ago

rules_python uses a hermetic Python interpreter in its latest versions: https://github.com/bazelbuild/rules_python#toolchain-registration I don't know whether rules_fuzzing works with these versions yet though.

matta commented 2 years ago

For fun I just tried this in a new Fedora 36 install and get the same error that I got with Debian Testing.

matta commented 2 years ago

For fun I just tried this in a VM running a new Ubunu 22.04.1 LTS install. I get the same error I got with Debian Testing.

stefanbucur commented 2 years ago

I've just created #208 to bump up the Python dependencies to the latest version. Can you depend on this revision in particular and let me know if you're still running into the issue?

matta commented 2 years ago

That fixes it, thanks @stefanbucur!

stefanbucur commented 2 years ago

Awesome! I'll cut a new release then once this gets in.

emesterhazy commented 2 years ago

This is causing an issue for me and at least a few others trying to build TCMalloc, and it looks like TCMalloc is actually using the 0.1.1 release of rules_fuzzing.

Is the solution to wait until rules_fuzzing cuts a new release, and then update TCMalloc to reference it?

Cross reference: https://github.com/google/tcmalloc/issues/127

Edit: @matta can you explain what you did to bypass this error? I tried changing the rules_fuzzing dependency in TCMalloc so that it pointed directly at the master branch, but I'm getting the same error.

git_repository(
  name = "rules_fuzzing",
  branch = "master",
  remote = "https://github.com/bazelbuild/rules_fuzzing.git",
)

The error:

➜  ~/c/g/g/tcmalloc git:(master) ✗ bazel test //tcmalloc/...
ERROR: /home/evan/.cache/bazel/_bazel_evan/0a2e6973e40898d99a1f54ccbe9ea23f/external/fuzzing_py_deps/pypi__absl_py/BUILD:5:11: no such package '@fuzzing_py_deps//pypi__enum34': BUILD file not found in directory 'pypi__enum34' of external repository @fuzzing_py_deps. Add a BUILD file to a directory to mark it as a package. and referenced by '@fuzzing_py_deps//pypi__absl_py:pypi__absl_py'
ERROR: Analysis of target '//tcmalloc/internal:profile_builder_fuzz' failed; build aborted:
INFO: Elapsed time: 0.213s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (65 packages loaded, 671 targets configured)
FAILED: Build did NOT complete successfully (65 packages loaded, 671 targets configured)
    currently loading: @com_google_absl//absl ... (2 packages)
➜  ~/c/g/g/tcmalloc git:(master) ✗
matta commented 2 years ago

Edit: @matta can you explain what you did to bypass this error? I tried changing the rules_fuzzing dependency in TCMalloc so that it pointed directly at the master branch, but I'm getting the same error.

Hi @emesterhazy,

First a disclaimer: I am only a novice when it comes to this part of Bazel (third party package downloads, etc.).

Simplest fix would be to get tcmalloc to depend on a newer rules_fuzzing. Both https://bazel.build/docs/external#shadowing-dependencies and https://bazel.build/docs/external#overriding-repositories might let you use a local git clone of the tcmalloc project for debug/test purposes.

I am using the fuzzing rules directly in a small hobby project of mine, not indirectly through the tcmalloc project. Fetching the latest version in my WORKSPACE with git_repository worked for me. I have this:

# For now, depend on a specific git revision of rules_fuzzing (that is
# still a PR in review!) to get newer Python treatment.  See:
# https://github.com/bazelbuild/rules_fuzzing/pull/208
# https://github.com/bazelbuild/rules_fuzzing/issues/207
git_repository(
    name = "rules_fuzzing",
    remote = "https://github.com/bazelbuild/rules_fuzzing.git",
    commit = "77bb2af4ee076e0706bb14fb5d62b108043624e1",
    shallow_since = "1660592161 +0000",
)

Also it looks like https://github.com/bazelbuild/rules_fuzzing/commit/23f9676510995ebf0de2ef3aa7da41045308c171 bumped the rules_python dependency in a call to a maybe rule. It "may be" that this call to maybe isn't taking effect in your project? I say this because https://bazel.build/docs/external#overriding-repositories implies that Bazel will complain if the same dependency is loaded at different versions; you didn't experience this and this maybe might be the reason. E.g. some other dependency in your project could still be pulling in and old version of rules_python.

See this search https://github.com/bazelbuild/rules_fuzzing/search?q=maybe -- hmm... we have this: https://bazel.build/rules/lib/repo/utils#maybe.

I wonder what would happen if you unconditionally loaded the newer rules_python early in your WORKSPACE:

    # Work around https://github.com/bazelbuild/rules_fuzzing/issues/207
    # and https://github.com/google/tcmalloc/issues/127
    http_archive(
        name = "rules_python",
        sha256 = "c03246c11efd49266e8e41e12931090b613e12a59e6f55ba2efd29a7cb8b4258",
        strip_prefix = "rules_python-0.11.0",
        url = "https://github.com/bazelbuild/rules_python/archive/refs/tags/0.11.0.tar.gz",
    )

The idea is; the older rules_fuzzing might see the newer rules_python is already present and skip loading the older rules_python.

matta commented 2 years ago

@stefanbucur FWIW I've confirmed that this fixes things. I'll leave the issue open in case you are using it as a reminder to push a new labeled release.

emesterhazy commented 2 years ago

Thanks @matta. It turns out there were a couple of issues. First, it turns out there was actually an explicit rules_python load in the WORKSPACE file to work around an earlier bug in rules_python:

# This is explicitly added to workaround
# https://github.com/bazelbuild/rules_python/issues/437.
http_archive(
    name = "rules_python",
    urls = ["https://github.com/bazelbuild/rules_python/releases/download/0.1.0/rules_python-0.1.0.tar.gz"],
    sha256 = "b6d46438523a3ec0f3cead544190ee13223a52f6a6765a29eae7b7cc24cc83a0",
)

I ended up fixing my problem by replacing this with rules_python-0.11.0 as you suggested. Unfortunately simply using a newer rules_fuzzing doesn't resolve the problem because an older rules_python gets pulled in by protobuf.

By the way, did you ever figure out the root cause of this python issue? I'm running Ubuntu 22.04.1 LTS and don't remember doing anything in particular to my python installation. I guess Debian 11 is the only distro that worked for you?

matta commented 2 years ago

I guess Debian 11 is the only distro that worked for you?

Yes, in a VM Debian 11 worked. Distros that user newer software broke.

My understanding is that the newer rules_python bundles its own python interpreter, which makes it hermetic with respect to the host's python.

I ended up fixing my problem by replacing this with rules_python-0.11.0 as you suggested. Unfortunately simply using a newer rules_fuzzing doesn't resolve the problem because an older rules_python gets pulled in by protobuf.

Ugh.

https://bazel.build/build/bzlmod aims at making this less of a problem. The problem statement in the bzlmod design doc describes this kind of problem clearly ("diamond dependency problem", with unclear resolution if there is a version conflict).

Until bzlmod matures, I guess a game of whack-a-mole with PRs bumping rules_python versions is the best way forward.