Defining two packages from the same package description

[ghost]

Some motivation for this: I'd like to setup running tests in the various Jane Street packages for various reason. However it is pretty much always the case that tests have more dependencies than the library code and adding these dependencies would create circular dependencies. For instance bin_prot tests are using core. Breaking these circular dependencies would be a huge amount of work for very little benefit. Moreover it is natural to want more things for tests.

Right now, we could solve this problem by creating _test packages for all our packages: core_test, async_test, ... They would use the exact same archive with a slightly different opam file. However maintaining this two parallel set of packages with exactly the same versions and sharing most of the opam file and other patches would be a pain, plus it would pollute the opam repo.

[AltGr]

Thanks for sharing an interesting use-case. I don't know if you had a more generic way of doing this in mind, but maybe it would be possible to have the build-test / depends {test] fields define a separate testing instance of the package instead of changing options of the base package. By doing opam install foo --test you would actually require this specific instance to be built/installed rather than affecting the normal package installation. It's just an idea

[ghost]

I guess that would work yes. Would that allow to do this: have the tests in core depends on core_extended and core_extended depend on core?

I though of another use case for this feature, which I think could benefit a lot of packages in opam. Right now, it is often the case that optional dependencies are used to build independent part of a package. A typical example is lwt, and this was recently added to bin_prot as well (janestreet/bin_prot#11).

In the case of bin_prot for instance, it make sense to have bin_prot and bin_prot_xen in the same repo, and to have them use the same version numbers given that the bin_prot_xen depends on the internals of bin_prot. However whether bin_prot_xen is installed or not doesn't affect at all users of just bin_prot.

At the moment we have this frustrating situation:

• one installs async, which will install bin_prot as a dependency
• later they install mirage-xen-ocaml, this will trigger the recompilation of bin_prot and all its reverse dependencies, including core_kernel, core, async* even though this is pointless as the bin_prot library itself didn't change

If we could define several packages from the same opam entry, maybe the opam file for bin_prot could look like something like this and bin_prot and bin_prot.xen would end up being independent packages for users:

opam-version: "1.2"
maintainer: "opensource@janestreet.com"
authors: ["Jane Street Group, LLC <opensource@janestreet.com>"]
homepage: "https://github.com/janestreet/bin_prot"
bug-reports: "https://github.com/janestreet/bin_prot/issues"
dev-repo: "https://github.com/janestreet/bin_prot.git"
license: "Apache-2.0"
build: [
  ["./configure" "--prefix" prefix]
  [make]
]
depends: [
  "ocamlbuild"     {build}
  "oasis"          {build & >= "0.4"}
  "ocamlfind"      {build & >= "1.3.2"}
  "js-build-tools" {build & >= "200.42.00" & < "200.43.00"}
]
available: [ ocaml-version = "4.02.3" ]

package "xen" [
  build: [
    ["./configure" "--prefix" prefix "--enable-xen" "--use-installed-bin_prot"]
    [make "bin_prot_xen"]
  ]
  depends: [
    "ocamlbuild"     {build}
    "oasis"          {build & >= "0.4"}
    "ocamlfind"      {build & >= "1.3.2"}
    "js-build-tools" {build & >= "200.42.00" & < "200.43.00"}
    "bin_prot" {same version}
  ]
]

This would require some work on the build system as well so that you can build bin_prot.xen using the installed bin_prot, but it shouldn't be too hard to do.

Technically what I'm proposing is just a convenient way to merge two directories in packages/.

[AltGr]

I guess that would work yes. Would that allow to do this: have the tests in core depends on core_extended and core_extended depend on core?

it's the point, core+tests would have the dependencies marked as {test}, but without affecting the normal core package.

For the bin_prot issue, I am right now trying to split opam-lib into several opam packages (to provide e.g. the parsers without requiring dose), and am confronted with part of the problems you describe — the main one being the build system, that must be able to compile using a sub-library either from the included source, or from an already installed version, since there is no conditional sub-packages in my case.

On allowing more flexibility with package definitions, I will see the different options. Duplicating the whole package into bin_prot and bin_prot_xen may be quite heavy, but on the other hand, having a "flat" repository also has a lot of advantages.

[AltGr]

I still don't have a good story on this, but there might be a way using the new {post} dependencies, which are designed to avoid cyclic dependencies.

So having bin_prot define depends: "core" {version & with-test & post} could be a meaningful way to break the cycle. At the moment, it wouldn't work ; but if running the tests could be delayed until after the post dependencies have been installed, it could.

There may be two issues with this idea, though:

• it means the package gets installed before the tests are run (while the currently expected behaviour is to abort before installing if the tests fail !) Rolling back the installation would mean uninstalling core as well and becomes a bit too involved...
• build-test: was redundant and has been deprecated, we now have build: with uses of the {with-test} filter, and run-test:. So in this case only run-test: would be able to use the {post & with-test} dependency, they wouldn't be able for the actual build: part.