Introduce ``AFL_EXIT_ON_SEED_ISSUES`` for LibAFL

[R9295]

Is your feature request related to a problem? Please describe. For some of our projects, we generate seeds based on the source code and initialize the fuzzer with them. It would be great to have the possibility to exit on any initial seeds leading to a solution; similar to AFL_EXIT_ON_SEED_ISSUES.

Describe the solution you'd like Modify the load_initial_inputs API to include a boolean flag to allow solutions and pass them down to load_file Something like:

    fn load_file<E, EM, Z>(
        ..
        allow_if_solution: bool, // here
    ) -> Result<(), Error>
    where
        ...
    {
      ..
        if forced {
          ..
        } else {
            let (res, _) = fuzzer.evaluate_input(self, executor, manager, input.clone())?;
            if res == ExecuteInputResult::Solution && !allow_if_solution { // here
                return Err(Error::corpus_error("Solutions are not allowed in the initial corpus")); // here
            }
        }
        Ok(())
    }

Describe alternatives you've considered I could check if there are any solutions after state.must_load_initial_inputs is done but then I would have to wait for all seeds to be processed.

Additional context None

[R9295]

Or we could add an option to State, which would retain the current API and just add a field on the struct and a builder function.

[domenukk]

How about returning an enum from load_file that's either LoadResult::Interesting, LoadResult::Ignored LoadResult::Forced, or LoadResult::Solution ?

[R9295]

What about something like this? Since we already have ExecuteInputResult, might be better to keep it

diff --git a/libafl/src/state/mod.rs b/libafl/src/state/mod.rs
index a8e4001d..3ecf7a9a 100644
--- a/libafl/src/state/mod.rs
+++ b/libafl/src/state/mod.rs
@@ -219,6 +219,10 @@ pub struct StdState<I, C, R, SC> {
     named_metadata: NamedSerdeAnyMap,
     /// `MaxSize` testcase size for mutators that appreciate it
     max_size: usize,
+    #[cfg(feature = "std")]
+    /// Allow an initial input to be a solution;
+    /// if not, will raise an error when loading initial inputs.
+    allow_initial_input_solution: bool,
     /// Performance statistics for this fuzzer
     #[cfg(feature = "introspection")]
     introspection_monitor: ClientPerfMonitor,
@@ -686,10 +690,35 @@ where
             let _: CorpusId = fuzzer.add_input(self, executor, manager, input)?;
         } else {
             let (res, _) = fuzzer.evaluate_input(self, executor, manager, input.clone())?;
-            if res == ExecuteInputResult::None {
+            self.process_evaluate_initial_input(fuzzer, path, input, res)?;
+        }
+
+        Ok(())
+    }
+    fn process_evaluate_initial_input<E, EM, Z>(
+        &mut self,
+        fuzzer: &mut Z,
+        path: &PathBuf,
+        input: I,
+        res: ExecuteInputResult,
+    ) -> Result<(), Error>
+    where
+        I: Input,
+        E: UsesState<State = Self>,
+        EM: EventFirer<State = Self>,
+        Z: Evaluator<E, EM, State = Self>,
+    {
+        match res {
+            ExecuteInputResult::None => {
                 fuzzer.add_disabled_input(self, input)?;
                 log::warn!("input {:?} was not interesting, adding as disabled.", &path);
             }
+            ExecuteInputResult::Corpus => {}
+            ExecuteInputResult::Solution => {
+                if !self.allow_initial_input_solution {
+                    return Err(Error::corpus_error(format!("initial input {} let to a solution; this is not allowed since allow_initial_input_solution is false", path.display())));
+                }
+            }
         }
         Ok(())
     }
@@ -1047,6 +1076,8 @@ where
             phantom: PhantomData,
             #[cfg(feature = "std")]
             multicore_inputs_processed: None,
+            #[cfg(feature = "std")]
+            allow_initial_input_solution: true,
         };
         feedback.init_state(&mut state)?;
         objective.init_state(&mut state)?;
diff --git a/libafl_bolts/src/lib.rs b/libafl_bolts/src/lib.rs
index 26271a8a..7f605bf0 100644
--- a/libafl_bolts/src/lib.rs
+++ b/libafl_bolts/src/lib.rs
@@ -322,6 +322,8 @@ pub enum Error {
     OsError(io::Error, String, ErrorBacktrace),
     /// Something else happened
     Unknown(String, ErrorBacktrace),
+    // Error with corpora
+    CorpusError(String, ErrorBacktrace),
 }

 impl Error {
@@ -438,6 +440,14 @@ impl Error {
     {
         Error::Unknown(arg.into(), ErrorBacktrace::new())
     }
+    /// Error with corpora
+    #[must_use]
+    pub fn corpus_error<S>(arg: S) -> Self
+    where
+        S: Into<String>,
+    {
+        Error::CorpusError(arg.into(), ErrorBacktrace::new())
+    }
 }

 impl Display for Error {
@@ -498,6 +508,10 @@ impl Display for Error {
                 write!(f, "Unknown error: {0}", &s)?;
                 display_error_backtrace(f, b)
             }
+            Self::CorpusError(s, b) => {
+                write!(f, "Corpus error: {0}", &s)?;
+                display_error_backtrace(f, b)
+            }
         }
     }
 }

[tokatoka]

i agree with domenukk's suggestion

[R9295]

Where would the LoadInputResult be handled though?

[domenukk]

Something like fn load_initial_inputs vs fn load_initial_inputs_fail_fast?

I don't think we should add an additional variable to the state just for this, it'll be stored and reloaded everytime, and it's really only important early on / rather nieche

[domenukk]

I guess we can reuse ExecuteInputResult here, yes