Input formats of the supported benchmarks - Request for help

[thuanpv]

Hi,

I have been trying to collect the input formats information of the supported benchmarks in FuzzBench. The information could be helpful for structure-aware fuzzers such as Libprotobuf-mutator (LPM), AFLSmart, Nautilus, and Superion. However, the information that I have collected so far, as shown in the following table, is not complete yet because I am not familiar with some benchmarks. It would be great if someone could help to update the table in the editable shared spreadsheet at https://docs.google.com/spreadsheets/d/1KBgiZrHLGKu2hsHIp6IMsbxjCrgYDC0aQVQyEU4Ql_c/edit?usp=sharing.

Benchmark	Currently Supported (File) Formats	Reference/Notes
bloaty_fuzz_target	ELF, Mach-O, WebAssembly
curl_curl_fuzzer_http	?	Supports network protocols
freetype2-2017	TTF, OTF, WOFF
harfbuzz-1.3.2	TTF, OTF, TTC	https://harfbuzz.github.io/ch01s03.html
jsoncpp_jsoncpp_fuzzer	JSON
lcms-2017-03-21	ICC profile
libjpeg-turbo-07-2017	JPEG
libpcap_fuzz_both	PCAP
libpng-1.2.56	PNG
libxml2-v2.9.2	XML
mbedtls_fuzz_dtlsclient	?	Supports network protocols
openssl_x509	?
openthread-2019-12-23	?	Supports Thread networking protocol
php_php-fuzz-parser	PHP
proj4-2017-08-14	?
re2-2014-12-09	?	Regular expression library
sqlite3_ossfuzz	SQLITE, DB?
systemd_fuzz-link-parser	?
vorbis-2017-12-11	OGG
woff2-2016-05-06	WOFF
zlib_zlib_uncompress_fuzzer	?

This is related to Issue #471

Thuan

[vanhauser-thc]

openssl: DER certificates zlib: zlib compressed data the others I dont know but would guess its their own specific protocol data.

[thuanpv]

Thanks @vanhauser-thc. I have updated the Google Spreadsheet accordingly.

[jonathanmetzman]

Thanks @thuanpv and @vanhauser-thc I'll see if I know/can figure out the formats of the other benchmarks. We could also ask the developers of these targets to confirm what they do since the targets are from OSS-Fuzz.

[jonathanmetzman]

Curl: HTTP server responses. Seed files here.

Systemd: Systemd (network) link files. The page describes it as an ini style text file. Seeds

re2: 2 bytes for options followed by a regular expression. Looks like it matches itself which doesn't seem great (we might want to switch to taking a string to match on and a regex). The options will make grammar aware fuzzing harder as I see one of them turns on posix style regexes (instead of perl I assume).

proj4: This builds standard_fuzzer.cpp. Here is the current copy.. It takes a format that looks somewhat custom but involves coordinates. There's probably more details on that here, but I'll try to get some lower hanging fruit for now.

zlib: zlib compressed files (see fuzz target source code)

sqlite3: Uses 3 (?) bytes for configuration followed by SQL to execute (the rest of the data becomes zsql).

mbedtls: I had a hard time figuring out what this does from reading the code but it gets set as the p_bio field of an mbedtls_ssl_context struct . The comment for this field says it is "context for I/O operations" (not very helpful :-( The seed file isn't too helpful either, but I would strongly guess this is the data read by the client during a tls connection. There are some more hints provided by the implementations of some of the fuzzer-specific callbacks they pass to the API functions they are testing: https://github.com/ARMmbed/mbedtls/blob/6abc20e0e3c89d790be355c922cf3c07cf0480c8/programs/fuzz/common.c#L35

Probably worth asking about this one.

[jonathanmetzman]

Oh forgot openthread: It's using the ip6-send-fuzzer (source code). It looks like it uses a byte for configuration and then the rest of the data gets passed to otIp6Send. The docs for that function says "This function sends an IPv6 datagram via the Thread interface." So I guess it is an IPv6 packet?

Once https://github.com/google/fuzzbench/pull/509 lands it should be easy for anyone to add OSS-Fuzz projects as benchmarks so if any of the benchmarks seem crappy, they can be replaced.

[inferno-chromium]

Reopen if needed - see https://google.github.io/fuzzbench/reference/benchmarks/