Expose interface to libfuzzer's extra counters

[fitzgen]

IIUC, this allows you to maximize another metric other than source code coverage (e.g. malloc sizes to try and trigger OOMs). Would be particularly useful for JITs that want to get coverage of the JITed code.

It seems like the interface is to define a custom section though, which isn't great: https://cs.github.com/aosp-mirror/platform_system_core/blob/8c8df2284bf81e4e2fe0a1d641983ff9b77b43d3/trusty/fuzz/counters.cpp?q=__libfuzzer_extra_counters#L37

[jameysharp]

I don't think the custom section is a libFuzzer requirement. I think it's just hard to get the linker to do what you need without one. In particular, IIRC you need a symbol for both the start and end of the array of counters, which means you need the linker to place one immediately after the other.

[fitzgen]

Yeah, I wish we could dynamically register multiple sets of custom counters.

[fitzgen]

Thinking about this a little more, and given the constraints of the interface, I think it may make sense to actually put this functionality in its own crate.

I'm thinking this crate would:

• internally define the __libfuzzer_extra_counters section (need to double check that this does connect the counters to libfuzzer or whether we also need to specify an export name for the static):

  #[link_section = "__libfuzzer_extra_counters"]
  static mut EXTRA_COUNTERS: [u8; 65_536] = [0; 65_536];

• expose a function to increment the ith counter (and wrap i around since we have a limited number of actual counters, not sure what the best approach is here since the total number of actual counters we have is somewhat arbitrary, but this generates tighter, branch-free assembly; although this might also suffer from harmonics so we might want to consider some more options here):

  pub fn inc(i: usize) {
    unsafe {
        let i = i % EXTRA_COUNTERS.len();
        let counter = &EXTRA_COUNTERS[i] as *const _ as *const AtomicU8;
        let count = (*counter).load(Ordering::Relaxed);
        let (count, overflowed) = count.overflowing_add(1);
        (*counter).store(count + (overflowed as u8), Ordering::Relaxed);
    }
  }

• use the links field in Cargo.toml to avoid creating multiple conflicting extra counters sections

Thoughts?

[fitzgen]

• although this might also suffer from harmonics so we might want to consider some more options here

I guess the answer here is recognizing that what we are doing here is hashing and using a better hash function than i % len :-p

[fitzgen]

I guess the crate would also want to mark EXTRA_COUNTERS as pub as well so that it could be used from JIT code.

[fitzgen]

Oh also: the reason for adding the overflow carry back to the counter is to ensure that the counter is never zero again after its been incremented at all, which could otherwise accidentally hide information from the fuzzer if there were harmonics of 256 involved. This is taken from AFL++. They found that this was superior to doing a saturating add. See section 3.3 of https://www.usenix.org/system/files/woot20-paper-fioraldi.pdf

[jameysharp]

I spent a while today trying to figure out how other projects manage to use the extra counters. Then I found that Go used to use them but has switched to a different mechanism that's supposed to work on all LLVM-supported platforms. So maybe we want to study https://github.com/golang/go/commit/7ec6ef432a85a390365f2daed788f0d14c830c73. I notice they did do the never-zero thing with their counters.

Looking around more, I think we should be able to call __sanitizer_cov_8bit_counters_init(uint8_t *, uint8_t *) any number of times to add memory ranges for libFuzzer to check. That's way nicer than defining weird symbols in custom sections, and means we can just make it an optional API from libfuzzer-sys. See libfuzzer/FuzzerTracePC.cpp for details.

[fitzgen]

https://clang.llvm.org/docs/SanitizerCoverage.html has more docs, specifically https://clang.llvm.org/docs/SanitizerCoverage.html#inline-8bit-counters

Leaning towards implementing a sancov-sys and sancov pair of crates.

[fitzgen]

I've started on an implementation of those crates over here: https://github.com/rust-fuzz/sancov

Help most welcome!

[fitzgen]

Going to close this issue, work can continue in that repo.