jonathanmetzman
commented
1 year ago Thank you for the feedback! This is very helpful. @ochang WDYT about this?
Bug reports contain lots of information that is not relevant for me as an end used, and contain little information I need as an end used to act on the bug.
Looking at (which is also email contents I received): https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=51436
Project: syzkaller
I have only 1 project, so it's always syzkaller for me. + I receive it on the syzkaller mailing list.
Fuzzing Engine: libFuzzer
Not relevant for me as an end user at all.
Job Type: libfuzzer_asan_syzkaller
Not relevant for me as an end user at all.
These are all good points. Though project might be worth keeping for maintainers with multiple.
Platform Id: linux
It's always linux. Not sure if our project is even tested on other OSes. I think not.
This is useful in places like chrome which is fuzzed on OSes other than linux. Not sure we want to special case things.
Crash Address: Crash State:
Not useful.
This is meant to be a summary of the crash. The top 3 stackframes. e.g. (from here)
Crash State:
out of memory
tiff.Decode
tiff.FuzzTiffDecode
_cgoexp_45f247b365c3_LLVMFuzzerTestOneInputSanitizer: address (ASAN)
Looks excessive provided there is "Crash Type".
I disagree knowing the sanitizer is useless for everyone. It is relevant if someone wants to reproduce the crash and doesn't know about sanitizers (e.g. some of our users) this helps them create a build that can reproduce this issue.
Crash Revision: https://oss-fuzz.com/revisions?job=libfuzzer_asan_syzkaller&revision=202209150601
The revision is important for me. But this (1) gives me a link with indirection, (2) the link contents contain irrelevant information (Aflplusplus/Centipede revisions).
The AFLplusplus revision shouldn't appear in new bugs. The same will be true for new bugs and centipede: https://github.com/google/oss-fuzz/pull/8986
Detailed Report: https://oss-fuzz.com/testcase?key=5145981141778432
Such outline page/info is required, but all common info required for end users should be inline in the report, and the "Detailed Report" link should contain uncommon info and better be placed at the bottom, not at the top.
Issue filed automatically.
The constant trailer is needed, but maybe can be tightened somewhat (since it repeated text that appears in every report).
The useful info in the report for me is only:
I'm not sure we can always display the stacktrace however, in many cases it will be quite large. The indirection also allows us to update the stacktrace to the latest build.
Crash Type: Out-of-memory (exceeds 2560 MB) Reproducer Testcase: https://oss-fuzz.com/download?testcase_id=5145981141778432
What's useful for me is and is missing in the report: (1) crash report is the most useful bit of info (2) git revision of my project
A useful report for me would look like:
Crash Type: Out-of-memory (exceeds 2560 MB) Revision: b884348d83ed05e7e82aa997c494252ce0f48687 Reproducer: https://oss-fuzz.com/download?testcase_id=5145981141778432 Crash report: ==1770== ERROR: libFuzzer: out-of-memory (used: 2679Mb; limit: 2560Mb) Live Heap Allocations: 24130426 bytes in 32 chunks; quarantined: 10329 bytes in 23 chunks; 8189 other chunks; total chunks: 8244; showing top 95% (at most 8 unique contexts) 24121112 byte(s) (99%) in 11 allocation(s) #0 0x52ef96 in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 #1 0x4ad997 in operator new(unsigned long) cxa_noexception.cpp:0 #2 0x458342 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 #3 0x7f57659db0b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/libc-start.c:308:16 SUMMARY: libFuzzer: out-of-memory Detailed Report: https://oss-fuzz.com/testcase?key=5145981141778432The github copy of the issue contains no useful information for me (only indirections): google/syzkaller#3385
This is intentional, to avoid leaking sensitive data about vulnerabilities.
evverx
dvyukov
Bug reports contain lots of information that is not relevant for me as an end used, and contain little information I need as an end used to act on the bug.
Looking at (which is also email contents I received): https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=51436
I have only 1 project, so it's always syzkaller for me. + I receive it on the syzkaller mailing list.
Not relevant for me as an end user at all.
Not relevant for me as an end user at all.
It's always linux. Not sure if our project is even tested on other OSes. I think not.
Not useful.
Looks excessive provided there is "Crash Type".
The revision is important for me. But this (1) gives me a link with indirection, (2) the link contents contain irrelevant information (Aflplusplus/Centipede revisions).
Such outline page/info is required, but all common info required for end users should be inline in the report, and the "Detailed Report" link should contain uncommon info and better be placed at the bottom, not at the top.
The constant trailer is needed, but maybe can be tightened somewhat (since it repeated text that appears in every report).
The useful info in the report for me is only:
What's useful for me is and is missing in the report: (1) crash report is the most useful bit of info (2) git revision of my project
A useful report for me would look like:
The github copy of the issue contains no useful information for me (only indirections): https://github.com/google/syzkaller/issues/3385