discussion: merge QCheck with the quickcheck module of ITeml

[c-cube]

I don't use qcheck anymore because I tend to prefer to write the tests inline. However, the work I did on the Quickcheck module here could be ported to qcheck, in order to unify both. What do you think?

[vincent-hugot]

Unification is good (assuming qtest does not end up depending on another project; I don't think that's what you had in mind, but, just in case...).

Of course generally speaking I'm still perfectly willing to merge changes -- even ambitious ones -- under the common-sense assumptions previously outlined in https://github.com/vincent-hugot/iTeML/pull/19#issuecomment-140164682.

What changes would that unification entail for qtest, in terms of new features and potential compatibility breaks? Is it simply new generators, like the nice "fuel" thing etc, or something more?

(Note: After this answer I'll probably be inactive on github until tomorrow)

[c-cube]

Note (mostly) to myself: since 'a Quickcheck.arbitrary is already an object, I could probably add a module ExhaustiveGen for smallcheck tests and an optional method exhaustive: 'a ExhaustiveGen.t, where:

ExhaustiveGen : sig
  type +'a t
  type size = int
  val run : size -> 'a Iter.t

  val int : int t
  val list : 'a t -> 'a list t
  (* ... *)
end

This might be a nice complement to random testing, and combinators should not be much harder to write.

[c-cube]

Ok, I plan to do this real soon. Qcheck would become deprecated, and QTest2Lib would become the new qcheck, with some additional features to make it possible and convenient to use QTest2Lib without generated tests. In other words, qtest would become two relatively orthogonal components:

• a library for writing random tests, random generators (+ printers, shrinkers...), and running those tests. This library should be usable as is, for people who want to write their tests in a separate file;
• a tool that generates code that relies upon the library, from tests written in comments.

In the future, a third component could be added, that does the same as the tool but from ppx annotations instead of comments.

[vincent-hugot]

Great. This might be as good an opportunity as any to rename "QTest2Lib" into something more conventional -- I recall some mild trouble occasioned by the capitalisation: cf. #22.

[c-cube]

Well, I'd propose to rename it QCheck, but you might prefer something still related to qtest?

edit: or just Qtestlib or Libqtest or Qtest (since there is no possible conflict with the tool)…

[vincent-hugot]

From #22, the issue was with capitalisation.

QTest2Lib is the only library not restricting it's [sic] name to lowercase characters.

So I'd go with something lowercase, like qcheck or qtest or libqtest/check,... whichever is the most consistent with the way libraries are usually named in practice.

Given the intention to have a self-contained testing library, then naming it qcheck seems good, otherwise there might be confusion between the tool (qtest) and the lib.

So I'd say the choice is between qcheck and libqcheck. I'd vote for the first, unless most libraries on opam are named lib*.

(For really major changes, I had planned an eventual migration to the name iteml "Inline (Unit) Tests for OCaml", which is more searchable -- there are already plenty of "qtest"s on the net.)

[c-cube]

About "iteml", I'm really skeptical (it doesn't contain "test" in its name, and I'm used to "qtest"). Otherwise I think I'll go for Qcheck (it's a merge, after all).

[vincent-hugot]

Qcheck

Was the upper-case intentional? I juste linked twice to #22, mentioning that other library names are all lowercase (at least for the purposes of opam; of course it can be capitalised in writing).

About "iteml", I'm really skeptical

No argument there. If I really liked the name, I would not have waited to make the change. When I uncoupled qtest from Batteries and created the repo, I intended to make a complete, compatibility-breaking rewrite, and needed a name other than qtest to avoid confusion. Iteml was the best I could come up with at the time. In the absence of major compatibility-breaking changes, this is moot.

[c-cube]

Oh sorry, I misunderstood. The ocamlfind name would be qcheck, I think — the module itself, Qcheck. Would you object to put the current Quickcheck module into what is called qcheck (on opam and github) and have qtest (the tool that extracts tests from comments) depend on it, the same way it depends on ounit? The point is that opam names would correspond to module names (mostly), and the separation between tool and library would be clear.

[vincent-hugot]

When you say "on github", do you mean on a different repository ?

I would not object.

That said I am just not entirely convinced with the idea*, however, because qtest is joined with its test lib -- let's retroactively call it qcheck for the sake of this discussion -- much more intimately than with oUnit. The syntax and scope of the tests actually written by the user depend on it (Unless you eventually intend to make qtest lib-agnostic, which I don't think would be good idea.). oUnit is a backend which could be changed without the user noticing anything.

It is quite possible I am biased because I wrote lib and extractor together; it's true that the extractor has been stable for a while, and most changes go to qcheck, so I may overestimate the difficulty of having two repositories.

In the end, you're the one who'll take the brunt of maintaining and documenting the result (the latter being the most energy-consuming in my experience), so it's your call.

A difficulty I have in this discussion is that I have no personal convincing use-case for a test lib without extractor; like you said,

I don't use qcheck anymore because I tend to prefer to write the tests inline.

The quantity of effort that should go into making qcheck standalone depends on those "people who want to write their tests in a separate file": how strongly do they prefer that way and how many are there? I have no idea.

(*) this is not a euphemism for "I hate it". Then I'd say "I hate it".

[c-cube]

Well, several people have asked me about a standalone lib on IRC. But it's not a problem to bundle them with qtest, it just needs to be well documented (and you are right that documenting in one place is much simpler). I'll keep you informed anyway, I should go and merge the two projects, and worry about module naming later.

[c-cube]

About the license: some worry about GPL, but it looks like quickcheck.ml is LGPL (according to its small header). Qcheck is BSD, I've asked the few contributors if they agree to move it to LGPL. So, it looks like it's already GPL for the tool and LGPL for the library; is it ok for you if I make this clearer?

[vincent-hugot]

is it ok for you if I make this clearer?

It is.

[c-cube]

Merge is almost done at febe5b4c31c3c76daa08407701ca393aca412a6f.

[c-cube]

Ok, I think the merge is complete. I will fix a few more issues and release in a few days.