ossf / fuzz-introspector

Fuzz Introspector -- introspect, extend and optimise fuzzers
https://fuzz-introspector.readthedocs.io
Apache License 2.0
377 stars 54 forks source link

use caching in compiler plugin to reduce resources used #195

Open DavidKorczynski opened 2 years ago

DavidKorczynski commented 2 years ago

Many projects on OSS-Fuzz are killed when we are collecting data about all of the functions. We can improve the performance in this area a lot as it currently duplicates a ton of work for many functions.

evverx commented 2 years ago

FWIW FI appears to get killed by the OOM killer locally sometimes as well:

Aug 26 14:23:40 H kernel: Out of memory: Killed process 411425 (ld.gold) total-vm:1692920kB, anon-rss:1613104kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:3344kB oom_score_adj:0
Aug 26 14:23:40 H systemd[1]: user@1000.service: A process of this unit has been killed by the OOM killer.
Aug 26 14:23:40 H systemd[736]: libpod-498855f30a8282abcc71721286f426b0310b43a5642bd63086619f2941e0c473.scope: A process of this unit has been killed by the OOM killer.
Aug 26 14:23:41 H admiring_benz[409266]: [393/400] clang++  -o fuzz-dhcp-client fuzz-dhcp-client.p/src_libsystemd-network_fuzz-dhcp-client.c.o -L/usr/lib/clang/15.0.0/lib/linux -Wl,--as-needed -Wl,--allo>
Aug 26 14:23:41 H admiring_benz[409266]: FAILED: fuzz-dhcp-client
Aug 26 14:23:41 H admiring_benz[409266]: clang++  -o fuzz-dhcp-client fuzz-dhcp-client.p/src_libsystemd-network_fuzz-dhcp-client.c.o -L/usr/lib/clang/15.0.0/lib/linux -Wl,--as-needed -Wl,--allow-shlib-un>
Aug 26 14:23:41 H admiring_benz[409266]: We are now in the FuzzIntrospector module pass
Aug 26 14:23:41 H admiring_benz[409266]: [Log level 1] : 14:20:48 : Fuzz introspector is running
Aug 26 14:23:41 H admiring_benz[409266]: [Log level 1] : 14:20:48 : Running introspector on ld-temp.o
Aug 26 14:23:41 H admiring_benz[409266]: [Log level 1] : 14:20:48 : This is a fuzzer, performing analysis
Aug 26 14:23:41 H admiring_benz[409266]: [Log level 2] : 14:20:48 : Using default configuration
Aug 26 14:23:41 H admiring_benz[409266]: [Log level 1] : 14:20:48 : Logging next yaml tile to /src/fuzzerLogFile-0-fpVbMGb2BQ.data.yaml
Aug 26 14:23:41 H admiring_benz[409266]: [Log level 1] : 14:20:48 : Wrapping all functions
Aug 26 14:23:41 H admiring_benz[409266]: clang-15: error: unable to execute command: Killed
Aug 26 14:23:41 H admiring_benz[409266]: clang-15: error: linker command failed due to signal (use -v to see invocation)