search

google / clusterfuzz

Scalable fuzzing infrastructure.
https://google.github.io/clusterfuzz
Apache License 2.0
5.27k stars 551 forks source link

Character-based minimization #2294

Open progers opened 3 years ago

progers commented 3 years ago

Clusterfuzz minimizations are still quite difficult to handle and typically take me an hour to minimize by hand. Could the minimizer do character-based minimizations to get closer to the smallest possible repro?

Here is an example of a user using lithium to minimize a clusterfuzz minimized testcase: https://crbug.com/1183879#c7

atotic commented 3 years ago

Anecdotal datapoint: I've been assigned several clusterfuzzes a week for a while now. Each one starts as a ~1000 lines, and I spend one to two hours minimizing it to 5 tags, and 5 lines of js. Having smaller minimizations would be very helpful.

It feels like the current minimizations are not very smart. Often, I am able to remove entire bodies of JS functions and still reproduce the failure. A good first step might be to try making each function a noop?

atotic commented 3 years ago

Just got one today. Unminimized case is 1MB, 7900LOC https://bugs.chromium.org/p/chromium/issues/detail?id=1204348#c2

Clusterfuzz did not minimize it at all. I was also unable to minimize the testcase. DCHECK was only hit on initial content_shell launch, not on reload, making reduction very labor intensive. In addition, most of the minor removals made the bug go away.

Maybe this testcase was just irreducible.

atotic commented 3 years ago

I've just ran into a particularly tough reduction: https://clusterfuzz.com/testcase-detail/6452246536323072

The problem is buildDOM() javascript routine. What it does is randomly create elements and insert them by indexing into document.all. elemTree.push(document.createElement('div')) elemTree[elemTree.length-1].id='id_10'; document.all[2].appendChild(elemTree[elemTree.length-1])

The problem is that removing any element causes buildDOM to change a lot, because all indexes start pointing to a new element.

I've already spent 3 hours reducing this case, and I'll probably spend 3 more. The bug is underinvalidation when an obsolete table property has mutated. If not found by clusterfuzz, this bug would have never been fixed, because the property predates javascript, and I've never seen it used.

Spending 6 hours on a P3 feels like a very unproductive use of developer time.

Suggestion: either prohibit document.all indexing in fuzzing, or implement character-based minimization.