bazelbuild / rules_fuzzing

Bazel Starlark extensions for defining fuzz tests in Bazel projects
Apache License 2.0
82 stars 19 forks source link

OSS-Fuzz build fails because of tar command failure (large size) #185

Open asraa opened 2 years ago

asraa commented 2 years ago

https://github.com/bazelbuild/rules_fuzzing/blob/2548aeae56e911b09b2ea879b94f30ae3a01dc50/fuzzing/private/oss_fuzz/package.bzl#L74

It seems like the tar command in the line linked above is causing a failure in this OSS-Fuzz build (https://oss-fuzz-build-logs.storage.googleapis.com/log-86b8ab3e-bb19-47c2-b07a-ed8ceb2b30d0.txt)

Step #3 - "compile-afl-address-x86_64": INFO: From Generating Descriptor Set proto_library //test/common/buffer:buffer_fuzz_proto:
Step #3 - "compile-afl-address-x86_64": Warning: AFL++ tools might need to set AFL_MAP_SIZE to 121232 to be able to run this instrumented program if this crashes!
Step #3 - "compile-afl-address-x86_64": ERROR: /src/envoy/test/common/http/http2/BUILD:188:19: Action test/common/http/http2/response_header_fuzz_test_oss_fuzz.tar failed: (Exit 2): bash failed: error executing command 
Step #3 - "compile-afl-address-x86_64":   (cd /root/.cache/bazel/_bazel_root/4e9824db8e7d11820cfa25090ed4ed10/execroot/envoy && \
Step #3 - "compile-afl-address-x86_64":   exec env - \
Step #3 - "compile-afl-address-x86_64":   /bin/bash -c '
Step #3 - "compile-afl-address-x86_64":             set -e
Step #3 - "compile-afl-address-x86_64":             declare -r STAGING_DIR="$(mktemp --directory -t oss-fuzz-pkg.XXXXXXXXXX)"
Step #3 - "compile-afl-address-x86_64":             function cleanup() {
Step #3 - "compile-afl-address-x86_64":                 rm -rf "$STAGING_DIR"
Step #3 - "compile-afl-address-x86_64":             }
Step #3 - "compile-afl-address-x86_64":             trap cleanup EXIT
Step #3 - "compile-afl-address-x86_64":             ln -s "$(pwd)/bazel-out/k8-fastbuild-ST-b2add19c87e5/bin/test/common/http/http2/response_header_fuzz_test" "$STAGING_DIR/response_header_fuzz_test"
Step #3 - "compile-afl-address-x86_64":             while IFS= read -r line; do
Step #3 - "compile-afl-address-x86_64":               IFS='\'' '\'' read -r link target <<< "$line"
Step #3 - "compile-afl-address-x86_64":               mkdir -p "$(dirname "$STAGING_DIR/response_header_fuzz_test.runfiles/$link")"
Step #3 - "compile-afl-address-x86_64":               ln -s "$(pwd)/$target" "$STAGING_DIR/response_header_fuzz_test.runfiles/$link"
Step #3 - "compile-afl-address-x86_64":             done <bazel-out/k8-fastbuild/bin/test/common/http/http2/response_header_fuzz_test_oss_fuzz_runfiles
Step #3 - "compile-afl-address-x86_64":             if [[ -n "bazel-out/k8-fastbuild/bin/test/common/http/http2/response_header_fuzz_test_corpus" ]]; then
Step #3 - "compile-afl-address-x86_64":                 pushd "bazel-out/k8-fastbuild/bin/test/common/http/http2/response_header_fuzz_test_corpus" >/dev/null
Step #3 - "compile-afl-address-x86_64":                 zip --quiet -r "$STAGING_DIR/response_header_fuzz_test_seed_corpus.zip" ./*
Step #3 - "compile-afl-address-x86_64":                 popd >/dev/null
Step #3 - "compile-afl-address-x86_64":             fi
Step #3 - "compile-afl-address-x86_64":             if [[ -n "" ]]; then
Step #3 - "compile-afl-address-x86_64":                 ln -s "$(pwd)/" "$STAGING_DIR/response_header_fuzz_test.dict"
Step #3 - "compile-afl-address-x86_64":             fi
Step #3 - "compile-afl-address-x86_64":             if [[ -n "" ]]; then
Step #3 - "compile-afl-address-x86_64":                 ln -s "$(pwd)/" "$STAGING_DIR/response_header_fuzz_test.options"
Step #3 - "compile-afl-address-x86_64":             fi
Step #3 - "compile-afl-address-x86_64":             tar -chf "bazel-out/k8-fastbuild/bin/test/common/http/http2/response_header_fuzz_test_oss_fuzz.tar" -C "$STAGING_DIR" .
Step #3 - "compile-afl-address-x86_64":         ')
Step #3 - "compile-afl-address-x86_64": Execution platform: @local_config_platform//:host
Step #3 - "compile-afl-address-x86_64": tar: bazel-out/k8-fastbuild/bin/test/common/http/http2/response_header_fuzz_test_oss_fuzz.tar: Wrote only 4096 of 10240 bytes
Step #3 - "compile-afl-address-x86_64": tar: Error is not recoverable: exiting now

Maybe -z should be added? If that makes sense, let me know! happy to open a PR

asraa commented 2 years ago

@adisuissa

stefanbucur commented 2 years ago

Adding -z might make sense - the reason it's currently not used is due to the assumption that CPU is a scarcer resource than disk during builds and that uncompressed I/O is faster than compressed I/O. But I haven't validated this assumption anywhere before implementing it :)

If -z fixes the issue in your case, I'd be happy to accept this change in the codebase!