google / oss-fuzz

OSS-Fuzz - continuous fuzzing for open source software.
https://google.github.io/oss-fuzz
Apache License 2.0
10.19k stars 2.17k forks source link

How to add code coverage for compile_native_go_fuzzer? #10916

Open antonmedv opened 10 months ago

antonmedv commented 10 months ago

I see what project expr has no coverage report.

From the documentation looks like go-fuzz should have cov reporting, but not compile_native_go_fuzzer, is that right?

How to add coverage for compile_native_go_fuzzer?

Thanks.

DavidKorczynski commented 10 months ago

@AdamKorcz could you assist here?

manunio commented 10 months ago

Hi, facing same issue with mongo-go-driver, its also using compile_native_go_fuzzer https://github.com/google/oss-fuzz/blob/5f372e6e7928c9a7b78542a7b710a0020b4de4b4/projects/mongo-go-driver/build.sh#L26

Coverage report link is throwing 404, Coverage builds are passing but it shows following lines:

2023/09/12 06:27:30 failed to parse go file: open go.mongodb.org/workspace/out/libfuzzer-coverage-x86_64/mongo-driver/bson/bson_corpus_spec.go: no such file or directory
Step #5: 2023/09/12 06:27:30 failed to parse go file: open go.mongodb.org/workspace/out/libfuzzer-coverage-x86_64/mongo-driver/bson/decoder.go: no such file or directory
Step #5: 2023/09/12 06:27:30 failed to parse go file: open go.mongodb.org/workspace/out/libfuzzer-coverage-x86_64/mongo-driver/bson/encoder.go: no such file or directory
Step #5: 2023/09/12 06:27:30 failed to parse go file: open go.mongodb.org/workspace/out/libfuzzer-coverage-x86_64/mongo-driver/bson/fuzz.go: no such file or directory
Step #5: 2023/09/12 06:27:30 failed to parse go file: open go.mongodb.org/workspace/out/libfuzzer-coverage-x86_64/mongo-driver/bson/marshal.go: no such file or directory
Step #5: 2023/09/12 06:27:30 failed to parse go file: open go.mongodb.org/workspace/out/libfuzzer-coverage-x86_64/mongo-driver/bson/primitive_codecs.go: no such file or directory
antonmedv commented 10 months ago

Seems what coverage for expr still failing. 🤔

antonmedv commented 10 months ago

Looks like code at https://github.com/google/oss-fuzz/blob/d4f443201aa7fef94f597e49ca768cfcabe9e25b/infra/base-images/base-builder/compile_native_go_fuzzer#L52C1-L52C1 selects all go files. In my expr there are two go files. So next cp fails.

AdamKorcz commented 10 months ago

Yes, it might be that it is the -r flag that causes issues. I assume you have two fuzz_test.go?

antonmedv commented 10 months ago
antonmedv commented 10 months ago

I can delete fuzz_coverage_test.go

antonmedv commented 10 months ago

After dropping fuzz_coverage_test file coverage reporting works. But it strange: coverage reported only for one file: https://storage.googleapis.com/oss-fuzz-coverage/expr/reports/20230917/linux/index.html#file0

image

Only fuzz_env.go file in coverage.

AdamKorcz commented 10 months ago

After dropping fuzz_coverage_test file coverage reporting works. But it strange: coverage reported only for one file: https://storage.googleapis.com/oss-fuzz-coverage/expr/reports/20230917/linux/index.html#file0

image

Only fuzz_env.go file in coverage.

Thank you for testing it out @antonmedv; I am happy that it is working. The issue with only showing a single file is in Golang: AFAIK, Golang only includes the packages that have coverage in the report.

Closing, since things are fixed on the OSS-Fuzz side.

antonmedv commented 10 months ago

But why Expr source files were not included in the coverage report? fuzz_text calls expr.Run() but it is not included.

AdamKorcz commented 10 months ago

But why Expr source files were not included in the coverage report? fuzz_text calls expr.Run() but it is not included.

Let me check that.

antonmedv commented 9 months ago

Coverage is still reported only for one file (100%):

Screenshot 2023-10-02 at 22 53 31 Screenshot 2023-10-02 at 22 53 52