Don't require entries in the corpus

[langston-barrett]

When writing a generative (e.g., black-box, grammar-based) fuzzer that doesn't do mutations, I use ConstFeedback::False as the feedback, because I don't need to save any inputs that don't cause crashes. But I can't just leave the corpus empty. In particular, this fuzzer panics:

use libafl::feedbacks::ConstFeedback;
use libafl::monitors::SimpleMonitor;
use libafl::{
    corpus::InMemoryCorpus,
    events::SimpleEventManager,
    executors::{inprocess::InProcessExecutor, ExitKind},
    feedbacks::CrashFeedback,
    fuzzer::{Fuzzer, StdFuzzer},
    generators::RandPrintablesGenerator,
    inputs::BytesInput,
    schedulers::QueueScheduler,
    stages::generation::GenStage,
    state::StdState,
};
use libafl_bolts::{current_nanos, rands::StdRand, tuples::tuple_list};

pub fn main() {
    let mut harness = |_input: &BytesInput| ExitKind::Ok;
    let mut feedback = ConstFeedback::False;
    let mut objective = CrashFeedback::new();
    let mut state = StdState::new(
        StdRand::with_seed(current_nanos()),
        InMemoryCorpus::new(),
        InMemoryCorpus::new(),
        &mut feedback,
        &mut objective,
    )
    .unwrap();

    let mon = SimpleMonitor::new(|s| println!("{s}"));
    let mut mgr = SimpleEventManager::new(mon);
    let scheduler = QueueScheduler::new();
    let mut fuzzer = StdFuzzer::new(scheduler, feedback, objective);
    let mut executor = InProcessExecutor::new(&mut harness, (), &mut fuzzer, &mut state, &mut mgr)
        .expect("Failed to create the Executor");
    let generator = RandPrintablesGenerator::new(32);
    // state
    //     .generate_initial_inputs(&mut fuzzer, &mut executor, &mut generator, &mut mgr, 1)
    //     .expect("Failed to generate the initial corpus");
    let mut stages = tuple_list!(GenStage::new(generator));
    fuzzer
        .fuzz_loop(&mut stages, &mut executor, &mut state, &mut mgr)
        .expect("Error in the fuzzing loop");
}

thread 'main' panicked at src/main.rs:43:10:
Error in the fuzzing loop: Empty("No entries in corpus. This often implies the target is not properly instrumented.", ErrorBacktrace)
stack backtrace:
   0: rust_begin_unwind
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/std/src/panicking.rs:645:5
   1: core::panicking::panic_fmt
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/core/src/panicking.rs:72:14
   2: core::result::unwrap_failed
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/core/src/result.rs:1654:5
   3: core::result::Result<T,E>::expect
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/core/src/result.rs:1034:23
   4: no_entries::main
             at ./src/main.rs:41:5
   5: core::ops::function::FnOnce::call_once
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Note that the error message is also misleading. This panics because I didn't do generate_initial_inputs{,_forced}, not because of instrumentation. However, I shouldn't need to generate_initial_inputs_forced, because I don't need them in my corpus!

Describe the solution you'd like Don't panic in this case

Describe alternatives you've considered Status quo

[tokatoka]

addison suggested that we can move this check into the startup of most of the schedulers. and then we can have NopScheduler that doesn't have this check, and you can use NopScheduler

[langston-barrett]

Sounds good to me! How would you suggest dealing with this call to Scheduler::next in StdFuzzer::fuzz_one? If there are no test cases, there will be no indices to return. Could return a "dummy" index, but that feels like a hack.

[domenukk]

technically we could change fuzz_one to return an Optional corpus_idx? Or maybe we should just have another fuzz function that is not fuzz_one (since we're not really fuzzing one testcase?), such as fuzz_generative?

[langston-barrett]

After having attempted this (#2199), it occurs to me that the "proper" fix is probably more wide-ranging. Here are a few notes from my reading of the code.

Ideally, I wouldn't be using StdState for my fuzzer, because I don't need a corpus (not even an InMemoryCorpus or NopCorpus), nor a corresponding feedback (not even ConstFeedback::False). Furthermore, my implementation of HasCurrentCorpusIdx could be trivial (always return None). However, implementing a custom State type is tricky, because State does a lot of different things.

Additionally, as noted in the comments above, the Fuzzer trait is not well-suited to handle generative fuzzers. It assumes a non-empty corpus, so that it can pick and set a "current testcase". We could add methods like fuzz_generative, but these wouldn't make sense for other impls, and would require a whole tree of alternative methods (fuzz_loop_generative, fuzz_loop_for_generative). Perhaps these are better suited to a different trait entirely, e.g. GenFuzzer?

The StdFuzzer chooses a "current testcase" because several other stages end up using it (obviously not all of them, i.e., GenStage). It seems like only these stages need the HasCurrentCorpusIdx bound, it doesn't necessarily need to be part of State itself.

[langston-barrett]

For now, I've worked around this problem in my fuzzers by replacing my previous combination of generate_initial_inputs_forced+fuzz_loop{,_for} with a basic inlined version of fuzz_loop_for:

            for _ in 0..gas {
                stages
                    .perform_all(
                        &mut fuzzer,
                        &mut executor,
                        &mut state,
                        &mut mgr,
                        CorpusId::from(0u64),
                    )?;
            }

[EDIT]: As it turns out, I still needed to have an entry in the corpus. Can at least avoid evaluating it with the fuzzer by doing:

            let input = generator.generate(&mut state)?;
            let testcase = Testcase::from(input);
            let corpus_id = state.corpus_mut().add(testcase)?;
            // ...
            for _ in 0..gas - iters {
                stages
                    .perform_all(&mut fuzzer, &mut executor, &mut state, &mut mgr, corpus_id);
            }

[tokatoka]

We could add methods like fuzz_generative, but these wouldn't make sense for other impls, and would require a whole tree of alternative methods (fuzz_loop_generative, fuzz_loop_for_generative). Perhaps these are better suited to a different trait entirely, e.g. GenFuzzer

I think this is the right way to do it

[tokatoka]

also in my opinion as long as State has a Corpus, it should have HasCurrentCorpusIdx

[domenukk]

I think we should be able to have a fuzzer that does generative sometimes, and mutational otherwise. Should we get rid of the corpus_id for perform_all?

[tokatoka]

to me if you want to have a state without corpus, then making another state is the right way. because stdstate does have the corpus. (and as long as it has corpus, it should have access to corpus.) and fuzz_one assumes the existence an corpus to mutate from and you can't use it for generative fuzzing, then making another method is the right way to do it.