stedolan / crowbar

Property fuzzing for OCaml
MIT License
180 stars 31 forks source link

"No instrumentation detected" after adding a particular simple property #71

Open remexre opened 6 months ago

remexre commented 6 months ago

I have the following property tests:

open Crowbar

let env (type a) (value : a gen) : (string * a) list gen =
  map
    [ list (pair bytes value) ]
    (List.fold_left (fun env (name, value) -> (name, value) :: env) [])

let evenp x = x mod 2 = 0
let sign i = if i < 0 then `Negative else `Positive

let () =
  add_test ~name:"example true property" [ int ] (fun i ->
      Crowbar.check_eq (evenp (i - 1)) (evenp (i + 1)));
  add_test ~name:"example false property" [ int ] (fun i ->
      Crowbar.check (i = -1 || sign i = sign (i + 1)));

  if true then
    add_test
      [ pair (pair bytes int) (pair bytes int); env int ]
      (fun ((l, v), (l', v')) s ->
        check_eq
          (List.assoc_opt l' ((l, v') :: (l, v) :: s))
          (List.assoc_opt l' ((l, v') :: s)))

I build this with the following dune file:

(executable
 (libraries crowbar)
 (name example_proptest)
 (ocamlopt_flags (-afl-instrument)))

When I run this under AFL or AFL++, I get PROGRAM ABORT : No instrumentation detected, which of course seems like it's a problem with my configuration. However, if I change the if true to if false (or otherwise remove the last test), it works perfectly fine under either (and they find the bug in the second property in seconds).

I can reproduce this against OCaml 5.0.0 or 4.14.1, both running on NixOS 23.11 (Linux) on an x86_64 machine.

Is there something particularly weird about that property, or broken with that test, or is this a broader bug?