Allow optional libraries without explicit use

[toots]

Desired Behavior

In Liquidsoap, optional features are historically registered via a global hash, typically:

root.ml

let registered_features = Hashtbl.create 10

optional_feature.ml

let () =
  Hashtbl.add Root.registered_features optional_feature

This means that, in order to compile optional features, we just need to pass the corresponding module to the linker.

In the process of migrating to dune, our approach so far has been to define a main library, an optional library and use them to build the final executable:

(library
 (name root)
 (modules root))

(library
  (name optional_feature)
  (libraries some-dependency)
  (optional)
  (modules optional_feature))

However, in order to make sure that the optional module is linked we have to do a couple of annoying tricks:

 (executable
 (name application)
 (modules application)
 (link_flags -linkall)
 (libraries
  root
  (select
   noop.ml
   from
   (optional_feature -> noop.dep.ml)
   (-> noop.dep.ml))))

Here we:

• Force all modules to be linked
• Use an empty noop.ml file to take advantage of alternative dependencies.

Is there an existing better way to do this? Alternatively, could it be possible for dune to support linking an optional library without having to specify a file using it?

[nojb]

I must be missing something, but why not simply pass optional_feature as a libraries dependency of application?

[toots]

I must be missing something, but why not simply pass optional_feature as a libraries dependency of application?

Because that'd make it a hard dependency:

Error: Library "optional_feature" in _build/default/src is hidden (optional with unavailable dependencies).

[ghost]

Force all modules to be linked

You can add (library_flags -linkall) to the library stanza of optional_feature, and this will have the same effect as passing optional_feature.cmx rather than optional_feature.cmxa to ocamlopt. Libraries that are plugin should generally be linked with -linkall. That's what we do for ppx rewriters for instance, where (kind ppx_rewriter) implies -linkall.

Is there an existing better way to do this?

Not that I know of. Apart from making optional_feature always exist and provide a dummy implementation when it cannot be provided.

Alternatively, could it be possible for dune to support linking an optional library without having to specify a file using it?

That seems fine to me.

[toots]

That makes a lot of sense, thanks.

Alternatively, could it be possible for dune to support linking an optional library without having to specify a file using it?

That seems fine to me.

This would be great! Until then, we have a work-around.

[bobot]

The way optional features are used seems very similar to plugins. Do you tried this approach?

[toots]

The way optional features are used seems very similar to plugins. Do you tried this approach?

We really want a single binary to be produced. It's already pretty complex to install the program. We've had support for plugins in the past and it was too complex to deal on top of everything else.