search

caddyserver / caddy

Fast and extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS
https://caddyserver.com
Apache License 2.0
58.29k stars 4.03k forks source link

[FUZZIT] Crash at fuzzing target parse-network-address #2851

Closed fuzzit-dev[bot] closed 5 years ago

fuzzit-dev[bot] commented 5 years ago

A new crash was discovered for fuzzing target parse-network-address. Here is a snippet of the log:

S: 1 CrossOver-
FUZZER: #1712   REDUCE ft: 875 corp: 72/783b lim: 4 exec/s: 0 rss: 66Mb L: 2/20 MS: 2 CopyPart-EraseBytes-
FUZZER: #1749   NEW    ft: 877 corp: 73/801b lim: 4 exec/s: 1749 rss: 66Mb L: 18/20 MS: 2 CopyPart-CrossOver-
FUZZER: #1780   REDUCE ft: 877 corp: 73/797b lim: 4 exec/s: 1780 rss: 66Mb L: 4/20 MS: 1 EraseBytes-
FUZZER: #1832   REDUCE ft: 878 corp: 74/805b lim: 4 exec/s: 1832 rss: 66Mb L: 8/20 MS: 2 PersAutoDict-CopyPart- DE: "\xff\xff"-
FUZZER: #1854   REDUCE ft: 878 corp: 74/803b lim: 4 exec/s: 1854 rss: 66Mb L: 6/20 MS: 2 ShuffleBytes-EraseBytes-
FUZZER: #1879   REDUCE ft: 878 corp: 74/802b lim: 4 exec/s: 1879 rss: 66Mb L: 6/20 MS: 5 ChangeBinInt-ChangeBit-EraseBytes-ChangeBit-ChangeBinInt-
FUZZER: #1970   REDUCE ft: 878 corp: 74/801b lim: 4 exec/s: 1970 rss: 66Mb L: 9/20 MS: 1 EraseBytes-
FUZZER: #2036   NEW    ft: 881 corp: 75/820b lim: 4 exec/s: 2036 rss: 66Mb L: 19/20 MS: 1 CopyPart-
FUZZER: #2097   NEW    ft: 882 corp: 76/837b lim: 4 exec/s: 349 rss: 454Mb L: 17/20 MS: 1 ShuffleBytes-
FUZZER: #2103   REDUCE ft: 882 corp: 76/834b lim: 4 exec/s: 350 rss: 454Mb L: 6/20 MS: 1 EraseBytes-
FUZZER: #2118   REDUCE ft: 882 corp: 76/833b lim: 4 exec/s: 353 rss: 454Mb L: 10/20 MS: 5 ChangeBit-ChangeBit-ChangeByte-ShuffleBytes-EraseBytes-
FUZZER: #2139   REDUCE ft: 882 corp: 76/831b lim: 4 exec/s: 356 rss: 454Mb L: 10/20 MS: 1 EraseBytes-
FUZZER: #2320   REDUCE ft: 882 corp: 76/830b lim: 4 exec/s: 154 rss: 573Mb L: 15/20 MS: 1 CrossOver-
FUZZER: #2342   REDUCE ft: 882 corp: 76/829b lim: 4 exec/s: 156 rss: 573Mb L: 10/20 MS: 2 ChangeBinInt-EraseBytes-
FUZZER: #2393   REDUCE ft: 882 corp: 76/828b lim: 4 exec/s: 159 rss: 573Mb L: 2/20 MS: 1 CrossOver-
FUZZER: #2394   REDUCE ft: 882 corp: 76/827b lim: 4 exec/s: 159 rss: 573Mb L: 2/20 MS: 1 EraseBytes-
FUZZER: #2397   REDUCE ft: 890 corp: 77/840b lim: 4 exec/s: 159 rss: 573Mb L: 13/20 MS: 3 ShuffleBytes-ChangeByte-ChangeBit-
FUZZER: #2424   NEW    ft: 891 corp: 78/857b lim: 4 exec/s: 96 rss: 649Mb L: 17/20 MS: 2 ChangeByte-PersAutoDict- DE: "\xff\xff"-
FUZZER: #2485   REDUCE ft: 891 corp: 78/855b lim: 4 exec/s: 99 rss: 649Mb L: 4/20 MS: 1 EraseBytes-
FUZZER: #2491   NEW    ft: 892 corp: 79/872b lim: 4 exec/s: 83 rss: 649Mb L: 17/20 MS: 1 ChangeASCIIInt-
FUZZER: #2557   REDUCE ft: 892 corp: 79/871b lim: 4 exec/s: 73 rss: 670Mb L: 12/20 MS: 1 EraseBytes-
FUZZER: #2568   NEW    ft: 893 corp: 80/879b lim: 4 exec/s: 73 rss: 670Mb L: 8/20 MS: 1 CopyPart-
FUZZER: #2570   REDUCE ft: 893 corp: 80/877b lim: 4 exec/s: 73 rss: 670Mb L: 4/20 MS: 2 CopyPart-EraseBytes-
FUZZER: #2677   NEW    ft: 894 corp: 81/894b lim: 4 exec/s: 58 rss: 755Mb L: 17/20 MS: 2 ChangeBit-ShuffleBytes-
FUZZER: #2684   NEW    ft: 895 corp: 82/902b lim: 4 exec/s: 58 rss: 755Mb L: 8/20 MS: 2 ShuffleBytes-ShuffleBytes-
FUZZER: #2695   REDUCE ft: 895 corp: 82/901b lim: 4 exec/s: 58 rss: 755Mb L: 12/20 MS: 1 EraseBytes-
FUZZER: #2844   NEW    ft: 898 corp: 83/919b lim: 4 exec/s: 45 rss: 755Mb L: 18/20 MS: 4 ShuffleBytes-ShuffleBytes-ChangeBit-PersAutoDict- DE: "\x00\x00\x00\x00\x00\x00\x00\x00"-
FUZZER: #2850   REDUCE ft: 898 corp: 83/916b lim: 4 exec/s: 45 rss: 755Mb L: 10/20 MS: 1 CrossOver-
FUZZER: #2909   REDUCE ft: 898 corp: 83/914b lim: 4 exec/s: 39 rss: 755Mb L: 4/20 MS: 4 ChangeByte-ChangeBinInt-ChangeByte-EraseBytes-
FUZZER: #2941   REDUCE ft: 898 corp: 83/910b lim: 4 exec/s: 40 rss: 755Mb L: 4/20 MS: 2 CopyPart-EraseBytes-
FUZZER: #3104   REDUCE ft: 899 corp: 84/914b lim: 4 exec/s: 30 rss: 755Mb L: 4/20 MS: 3 ShuffleBytes-ChangeByte-ShuffleBytes-
FUZZER: #3189   REDUCE ft: 899 corp: 84/911b lim: 4 exec/s: 26 rss: 756Mb L: 3/20 MS: 5 ChangeBinInt-ChangeBit-ChangeBit-CopyPart-CrossOver-
FUZZER: #3285   NEW    ft: 960 corp: 85/928b lim: 4 exec/s: 26 rss: 773Mb L: 17/20 MS: 1 ShuffleBytes-
FUZZER: #3511   NEW    ft: 1020 corp: 86/945b lim: 4 exec/s: 27 rss: 773Mb L: 17/20 MS: 1 ChangeASCIIInt-
FUZZER: #3596   NEW    ft: 1022 corp: 87/963b lim: 4 exec/s: 25 rss: 773Mb L: 18/20 MS: 5 PersAutoDict-ChangeBit-ChangeByte-ChangeByte-CrossOver- DE: "\x01\x00"-
FUZZER: #3636   NEW    ft: 1025 corp: 88/976b lim: 4 exec/s: 25 rss: 773Mb L: 13/20 MS: 5 EraseBytes-EraseBytes-ChangeBit-ChangeByte-InsertByte-
FUZZER: ==23== ERROR: libFuzzer: out-of-memory (used: 1985Mb; limit: 1984Mb)
FUZZER:    To change the out-of-memory limit use -rss_limit_mb=<N>
FUZZER: 
FUZZER: MS: 2 ChangeASCIIInt-CopyPart-; base unit: fe75feba60d9f051d75977c81fd564e0c76bb2ba
FUZZER: 0x6c,0xff,0xff,0x21,0x3a,0x37,0x2d,0x36,0x34,0x32,0x36,0x34,0x32,0x33,0x37,0x37,0x30,
FUZZER: l\xff\xff!:7-6426423770
FUZZER: artifact_prefix='./'; Test unit written to ./artifact
FUZZER: Base64: bP//ITo3LTY0MjY0MjM3NzA=
FUZZER: SUMMARY: libFuzzer: out-of-memory
FUZZER: stat::number_of_executed_units: 3733
FUZZER: stat::average_exec_per_sec:     22
FUZZER: stat::new_units_added:          153
FUZZER: stat::slowest_unit_time_sec:    0
FUZZER: stat::peak_rss_mb:              1987
2019/11/01 06:29:18 process finished with error = exit status 76
2019/11/01 06:29:18 Exit Status: 76
2019/11/01 06:29:18 uploading crash...

More details can be found here

Cheers, Fuzzit Bot

mohammed90 commented 5 years ago

Crasher input: l\xff\xff!:7-6426423770

mohammed90 commented 5 years ago

We've had other OOM crashers of similar type:

This is one of the log messages of one of the crashers libFuzzer: out-of-memory (used: 1996Mb; limit: 1984Mb). It's worth mentioning that this is relatively low (=249.5 MB) for users who do want to listen on such large range of addresses (~2^32), but it's still kind of wasteful.

Looking at the code that uses this func, I think the fix is simple. This func is used in 8 places all around the project (at commit fb06c041c4be4eb32f18d54e8e7feff8dd76b0e9 at the time of writing). They are:

I think we can optimize this by doing away with the slice, and replace it with a struct with fields for protocol (e.g. tcp, unix, udp, etc.) as string, host as string, port range start as int, and port range end as int. We can do away with the loop in ParseNetworkAddress and just return an instance of this struct. Funcs that use only the first address can go ahead withe using the protocol+host+port range start. Funcs that loop over the addresses can change their loop condition to start the index from the port range start to port range end inclusive. If both of these numbers are equal, then it'll go through the loop once, otherwise it'll loop just as many as needed.

How does this sound?

mholt commented 5 years ago

It's worth mentioning that this is relatively low (=249.5 MB) for users who do want to listen on such large range of addresses (~2^32), but it's still kind of wasteful.

This is amazing :joy:

And yes you are right. Great analysis!

I do like your idea a lot, it reduces memory and still allows a large port range.

Regardless, I think we should limit the port range to, say, 64K. That should solve the problems even without needing to use a struct. I'd be happy to review a struct implementation though.

But first, let's at least limit the port range to 64K (can ports go much higher anyway?) before you go into the extra effort. We can discuss the struct in more detail at that time.

mohammed90 commented 5 years ago

According to Wikipedia (too lazy to find the actual RFC):

A port number is a 16-bit unsigned integer, thus ranging from 0 to 65535.

We can limit it to that. I'll send the PR that covers both later today.

mohammed90 commented 5 years ago

The fuzzer been running for more than an hour with no crash yet, as opposed to previous runs that ran into crasher within second~minutes. Considering this resolved.