It should be possible for a package to include inline tests but only depend on `ppx_inline_test` as a `with-test` dependency

[gridbugs]

Desired Behavior

I want to use ppx_inline_test to test some internal APIs of a library, but in the opam package containing the library I want ppx_inline_test to be a with-test dependency. That is:

My code has:

let%test _ = Int.equal (41 + 1) 42

My dune file has:

(library
 ...
 (inline_tests)
 (preprocess (pps ppx_inline_test)))

My opam file has:

depends: [
  ...
  "ppx_inline_test" {with-test}
]

Currently this doesn't work as if ppx_inline_test is not installed the preprocess operation will fail as it runs every time the library is built regardless of whether it's being built in development or as part of opam install, and fair enough something needs to preprocess the file to either add or remove the tests. When ppx_inline_test is a test dependency then it won't be installed when installing the package with opam install <package> and then the package will then fail to build due to the missing dependency on ppx_inline_test.

This means that if I want to write unit tests of internal APIs either:

• people who use my library need to install ppx_inline_test as a dependency but I don't want to do this as ppx_inline_test has many dependencies of its own and it would bloat my package.
• I create a test-only executable/library and put all my private api inline tests there and have that be the only thing which depends on ppx_inline_test, though this would move the test definitions away from the code that they are testing, and I think having tests right next to the code is one of the great things about ppx_inline_test.

Neither solution is great and both present a hurdle to writing tests which I suspect will mean that people just won't write tests of their internal APIs. ppx_inline_test seems to be the blessed default testing library and all the dune docs I can find about inline tests require that the package containing the tests depends on ppx_inline_test (as a non-test dependency). Coming from rust where it is easy to test internal APIs without affecting the released artifact I expected that ppx_inline_test would work similarly in OCaml and was surprised to learn that what I see as the default use case for inline tests isn't (ergonomically) supported.

One possible solution might be to write a preprocessor which interprets the same directives as ppx_inline_test (e.g. let%test <name> = <test>) and replaces them with nothing (ie. just remove all the test directives) and ship this as part of dune and have dune run that preprocessor instead if ppx_inline_test is not installed? That way if I depend on ppx_inline_test as a test-only dependency then it will be available and dune will use it to run inline tests if my package is installed with opam install <package> --with-test and in CI where I make sure to install ppx_inline_test or when developing the package locally.

[Alizter]

The issue is that without ppx_inline_test, how will we separate the file to ignore the ppx? The common solution is to keep the test files separate but this can cause issues if you don't have access to certain functions in the API.

I guess one way to solve the latter issue is to implement an ability to link tests to a library or executable without the mli's getting compiled. That way you can have tests access everything as if they were in the same file.

[gridbugs]

Makes sense. Rust-style inline tests defined right next to the code under test are only possible in rust because the build system and preprocessor are tightly coupled so it can identify and remove testing code when building in a non-testing context. Since OCaml preprocessing is completely based on plugins (ppx's) this isn't possible or at least I can't think of a way to have it without sacrificing generality.