vincent-hugot / qtest

Inline (Unit) Tests for OCaml
GNU General Public License v3.0
67 stars 8 forks source link

small bugfix release 2.1.1 #29

Closed c-cube closed 8 years ago

c-cube commented 8 years ago

I think we should make a release, because the last merges are actually useful for batteries (tests don't build on < 4.02 because the dependency on bytes is not enforced at link time).

vincent-hugot commented 8 years ago

Ok; I'll do it on March 10.

vincent-hugot commented 8 years ago

There is a problem with the self tests in the current version:

Σ tests ./testfoo.sh                                                     ~/Desktop/iTeML/qtest/Σ 14:38 vhugot@Sita master:⚡
Target file: `footest.ml'. Extraction : `foo.ml' 
Warning: likely qtest syntax error: `(* $T bar' at foo.ml:122. Done.
Finished, 7 targets (0 cached) in 00:00:00.
random seed: 8590450
 [36 1! / 38] >even>foo.ml:232                                                 

does not terminate in reasonable time (or, possibly, at all).

c-cube commented 8 years ago

Ah, right, I vaguely remember a failing test. Was it added by me?

vincent-hugot commented 8 years ago

The test is supposed to be this one:

(*$T & 6 \
  & =
  2*3
*)

which is an old test that's been here for years about a corner case of the syntax; I don't remember which, exactly.

Either way, something's broken that certainly wasn't in the last release, as I wouldn't have published the package without running that script first.

c-cube commented 8 years ago

Ah yes, I remember. The bug is actually lower, in the last test of the file, which tests:

(Q.small_int_corners ()) (fun n-> odd (abs n+3) = even (abs n))

The problem is that among the corner cases, there is max_int, and abs n + 3 overflows on max_int.

Edit: actually the problem is that even takes a long time to compute on max_int anyway.

vincent-hugot commented 8 years ago

It would be best to have a more sensible test instead. I am probably to blame for the even / odd functions but I don't recall what purpose they served.

I'll replace the test by something else.

vincent-hugot commented 8 years ago

And now I seem to have broken my Opam 11 ways to sunday in a previous attempts to try other versions, and can't even compile my tests. It's depressing.

Would you like to take over maintainership for qtest? I simply haven't really programmed in years and, tough I would enjoy getting back on that horse, I have no professional reason to do so and neither time nor energy left for anything non-job related. (Hopefully that may change sometime, but I have 0 visibility; that's part of the problem).

I think it would make more sense for you to have direct access to qtest rather than needing to wait for me to catch up.

c-cube commented 8 years ago

Oh. Well, I can take over, but I might do changes my way (e.g. moving files in the repo, renaming stuff, etc.). Would that be okay then?

vincent-hugot commented 8 years ago

Of course, that's implied by "taking over maintainership". I'm currently too far from the action to sustain strong opinions on how things should be done.

I added you as a "collaborator", which should at least give you push access to the repository. Let me know if there are other settings I need to deal with to give you fuller access.

And thanks!

c-cube commented 8 years ago

Ok, thanks for your trust :-) I have access, I just pushed a small change. The new test seems to work fine, but another one fails (I suppose it's designed to fail anyway since the property looks false to me):

Failure("law foo.ml:164: arb_tree (fun t -> (size t > 1) ==> (t = rev t)) failed on 100 cases. Ex: Node(282, 59) (after 6 shrink steps)")

Anyway, I will make a release soon, if it's fine by you; if you have specific steps for making a release I'd be glad to hear them (and I would write them into some howto file).

vincent-hugot commented 8 years ago

Yeah, some tests are supposed to fail, of course. Perhaps the expected behaviour should be better documented, at least in the comments of foo.ml -- it grew quite organically and without direction. The aim is mainly to catch regressions when changing the syntax, so the only hard requirement is that those tests should run relatively fast.

For the releases:

I roughly followed https://opam.ocaml.org/doc/Packaging.html , and last time I used opam-publish.

opam pin add qtest.2.1.1 . -n on the repository; then opam-publish prepare qtest.2.1.1 https://github.com/vincent-hugot/iTeML/archive/v2.1.1.tar.gz -- I pushed a version tag for that; you'll need to move it --; this step should generate the metadata files by fetching the info from the current repository, which I updated last time. You might need to manually add a version: "2.1.1" to the generated opam file. Of course then you have to test the package; install, remove etc. Then theoretically opam-publish submit qtest.2.1.1 should submit a pull request on the opam repo. For some reason the command I used last time was opam publish submit ./qtest.2.1.0; I don't recall why the ./... I'm not at all clear on where opam pin edit qtest fetches its info, exactly, so watch out for that.

It's all a bit of a headache to me because the metadata feels like something that ought to be in the repo, and apparently several opam commands like "pin to directory" actually look for metadata in the repo, but it can't be, because the metadata should also contain a checksum of the repo archive (which would then include itself). You can't have a file containing its own checksum...

Having to force push to move tags is also a bit awkward, and you'll have to do it each time you test the package and realise you need to change something in the repo.

I managed to make it work thrice, but never quite got comfortable with the process; perhaps if you're more familiar with opam's architecture, you'll have an easier time than I had.

c-cube commented 8 years ago

Ok, I have used opam-publish already. I think I will manage, thanks!

c-cube commented 8 years ago

https://github.com/ocaml/opam-repository/pull/5935 should do the trick, hopefully.