Count is ignored, test stops on first failure

vincent-hugot / qtest

Inline (Unit) Tests for OCaml

GNU General Public License v3.0

67 stars 8 forks source link

Count is ignored, test stops on first failure #38

Closed copy closed 7 years ago

copy commented 8 years ago

In the following example, I'd expect that 10000000 examples are generated and then the smallest one (0) is printed, however it seems to stop after the first test:

(*$Q & ~count:10000000
  (Q.make ~print:string_of_int ~small:(fun i -> i) (fun s -> Random.int 1000)) (fun i -> i = i + 1)
*)

Maybe I'm misunderstanding the documentation?

c-cube commented 8 years ago

Indeed, that seems to be a bug. I have to confess I mostly use shrinking (which is really awesome) but sometimes it's too complicated to write. I will look into it!

c-cube commented 8 years ago

After a more thorough look, the current situation with max_fail, max_gen, small and shrink is a bit too complicated. I think we should force a choice between shrinking and generating a number of cases within [count, max_gen] (and maybe keep the smallest one(s) if small is provided).

c-cube commented 7 years ago

Please re-open if you think there still is a bug! With the latest release it should only print "0".