fuzz-format: Timeout in fuzz-format

[mohammed90]

Detailed Report: https://oss-fuzz.com/testcase?key=5806400649363456

Project: caddy Fuzzing Engine: libFuzzer Fuzz Target: fuzz-format Job Type: libfuzzer_asan_caddy Platform Id: linux

Crash Type: Timeout (exceeds 60 secs) Crash Address: Crash State: fuzz-format

Sanitizer: address (ASAN)

Regressed: https://oss-fuzz.com/revisions?job=libfuzzer_asan_caddy&range=202401210622:202401220608

Reproducer Testcase: https://oss-fuzz.com/download?testcase_id=5806400649363456

Issue on oss-fuzz tracker: Issue 66641

Minimized reproducer test case: clusterfuzz-testcase-minimized-fuzz-format-5806400649363456.txt

[bbaa-bbaa]

I can't reproduce the timeout with minimized testcase.

$ time ./caddy fmt clusterfuzz-testcase-minimized-fuzz-format-5806400649363456.txt > /dev/null
Executed in  609.03 millis    fish           external
   usr time  721.92 millis  127.15 millis  594.77 millis
   sys time   64.97 millis   26.36 millis   38.60 millis

Even if I limit the cpu frequency to 0.4Ghz, it can be done in 3 secs.

$ sudo cpupower frequency-set -u 0.4G
$ time ./caddy fmt clusterfuzz-testcase-minimized-fuzz-format-5806400649363456.txt > /dev/null
________________________________________________________
Executed in    2.27 secs    fish           external
   usr time    2.23 secs    1.19 millis    2.23 secs
   sys time    0.12 secs    2.08 millis    0.11 secs

Can you provide any other testcases?

[mohammed90]

Can you provide any other testcases?

Sure. There's an un-minimized test case. I'll share it later.

[mohammed90]

Can you provide any other testcases?

Here it is, 234KB Caddyfile: clusterfuzz-testcase-fuzz-format-5806400649363456.txt

Note this part from the oss-fuzz FAQ:

How do you handle timeouts and OOMs?

If a single input to a [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target) requires more than ~25 seconds or more than 2.5GB RAM to process, we report this as a timeout or an OOM (out-of-memory) bug

What are the specs on your machines?

Fuzzing machines only have a single core and fuzz targets should not use more than 2.5GB of RAM.

Those 2 caveats may play a big part in the finding.

[bbaa-bbaa]

Here it is, 234KB Caddyfile: clusterfuzz-testcase-fuzz-format-5806400649363456.txt

Test platform: AMD Ryzen 7 6800H with Radeon Graphics

Before #6117

$ time ./caddy fmt clusterfuzz-testcase-fuzz-format-5806400649363456.txt > /dev/null
________________________________________________________
Executed in    7.34 secs    fish           external
   usr time    7.30 secs  612.00 micros    7.30 secs
   sys time    0.03 secs  931.00 micros    0.03 secs

After #6117

$ time ./caddy fmt clusterfuzz-testcase-fuzz-format-5806400649363456.txt > /dev/null
________________________________________________________
Executed in   62.86 millis    fish           external
   usr time   59.54 millis   72.00 micros   59.47 millis
   sys time   21.77 millis  972.00 micros   20.80 millis

$ /usr/bin/time -v ./caddy fmt ./clusterfuzz-testcase-fuzz-format-5806400649363456.txt > /dev/null
Error: ./clusterfuzz-testcase-fuzz-format-5806400649363456.txt:5: Caddyfile input is not formatted; Tip: use '--overwrite' to update your Caddyfile in-place instead of previewing it. Consult '--help' for more options
Command exited with non-zero status 1
        Command being timed: "./caddy fmt ./clusterfuzz-testcase-fuzz-format-5806400649363456.txt"
        User time (seconds): 0.05
        System time (seconds): 0.02
        Percent of CPU this job got: 132%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.06
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 75092
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 0
        Minor (reclaiming a frame) page faults: 1523
        Voluntary context switches: 308
        Involuntary context switches: 21
        Swaps: 0
        File system inputs: 0
        File system outputs: 0
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 1

It takes about 8 seconds before patch #6117，and after #6117 it only takes about 70ms and about 80 MB of memory. I think PR #6117 already has enough speed without hitting the timeout