I am testing GNU Scientific Library (GSL) with AFL. Mathematical functions in GSL usually expect inputs to be within certain ranges. For example, a square root function expects nonnegative inputs. If the input goes out of the expected range, GSL invokes an error handler which prints out error messages and then invokes the abort() function.
How can I ask AFL to ignore crashes that are due to these kinds of crashes triggered by expected invalid inputs? With GSL, one can write a customized error handler. So I am trying to come up with an error handler that looks like this:
<some_type> gsl_error_handler(){
// ask afl to not consider the following abort() as a unique crash
abort();
}
The question is how to fill the part with "//" above. Note that if I simply create an error handler that does nothing would be problematic too due to other issues, so the abort() above has to be there in the error handler.
I am testing GNU Scientific Library (GSL) with AFL. Mathematical functions in GSL usually expect inputs to be within certain ranges. For example, a square root function expects nonnegative inputs. If the input goes out of the expected range, GSL invokes an error handler which prints out error messages and then invokes the abort() function.
How can I ask AFL to ignore crashes that are due to these kinds of crashes triggered by expected invalid inputs? With GSL, one can write a customized error handler. So I am trying to come up with an error handler that looks like this:
The question is how to fill the part with "//" above. Note that if I simply create an error handler that does nothing would be problematic too due to other issues, so the abort() above has to be there in the error handler.