Open avsm opened 6 years ago
in opam2, we have with-test to run basic tests
with-test
is not to run test, it is to indicate to build the tests.
running benchmarks is also highly desirable for some packages, and this currently has no place in the test/doc flags.
I once opened this https://github.com/ocaml/opam/issues/2278, but eventually closed as I'm quite sure it's not such a good idea (see message there).
I'd like not to conflate the (valid, as far as I can tell) reasons for not doing this for benchmarking, with whether it's reasonable for the general case of testlike things that take a long time to run.
in opam2, we have
with-test
to run basic tests, but we are now running into some limitations that I wanted to discuss before the opam2 RC (in the interest of minimising opam format changes)test/doc
flags.Any thoughts on how to incorporate these sorts of things into the current
with-test
andwith-doc
mechanism? Something slightly more general would let us expose fuzz testing and benchmarking more easily.cc @yomimono @stedolan @gemmag who were in the discussion about wanting this feature.