MozillaSecurity / grizzly

A cross-platform browser fuzzing framework
Mozilla Public License 2.0
304 stars 37 forks source link

[reduce] Report count less than expected? #432

Closed pyoor closed 6 months ago

pyoor commented 6 months ago

Seen in https://community-tc.services.mozilla.com/tasks/Jf3JNe3iSN--tgbSwxx9Cw/runs/0/logs/project/fuzzing/private/logs/screen2.log#L121

1716170640.217607 [2024-05-20 02:03:59] Exception during reduction!
1716170640.217608 Traceback (most recent call last):
1716170640.217608   File "/home/worker/grizzly-auto-run/venv/lib/python3.10/site-packages/grizzly/reduce/core.py", line 867, in main
1716170640.217608     return_code = mgr.run(
1716170640.217609   File "/home/worker/grizzly-auto-run/venv/lib/python3.10/site-packages/grizzly/reduce/core.py", line 517, in run
1716170640.217609     results = replay.run(
1716170640.217610   File "/home/worker/grizzly-auto-run/venv/lib/python3.10/site-packages/grizzly/replay/replay.py", line 543, in run
1716170640.217610     sum(x.count for x in reports.values() if x.expected)
1716170640.217610 AssertionError
tysmith commented 6 months ago

This can be triggered when a result that matches the provided signature is triggered by a startup failure. So in this case we added prefs that are intermittently triggering an issue that is unrelated to the provided test case.

tysmith commented 6 months ago

Fixed by #436.