How to generate a shared libray from C and ML code?

cogumbreiro commented 6 years ago

Hi, everyone,

I am a bit confused when trying to generate a dynamic library (shared object).

I expected this to be done by creating a .clib file listing the object files (one per line). When I do ocamlbuild libproj.so I get the following error message:

Backtrace:
                      - Failed to build the target lib/libproj.so
                          - Building lib/libproj.so:
                              - Failed to build all of these:
                                  - Building lib/libproj.cmxa:
                                      - Failed to build all of these:
                                          - Building lib/libproj.cmx:
                                              - Failed to build all of these:
                                                  - Building lib/libproj.ml:
                                                      - Failed to build all of these:
                                                          - Building lib/libproj.mly
                                                          - Building lib/libproj.mll
                                                  - Building lib/libproj.mlpack
                                          - Building lib/libproj.mllib
                                  - Building lib/libproj.mldylib

Should I be generating a shared library with a .clib file? Any help on what I am missing?

gasche commented 6 years ago

To build a dynamic library, the target-naming convention is that you should build dllproj.so (using dll rather than lib as for static libraries). (This is documented in the manual, but I admit this is a bit confusing.)

If you find yourself confused about ocamlbuild build rules, you could consider using ocamlbuild -documentation; grepping for clib there shows you the shape of the build rule that your ocamlbuild version supports.

cogumbreiro commented 6 years ago

Thanks for the help!

The advanced targets section could be improved (I'm happy to offer a pull-request if you welcome contributions).

Table 6 is a bit confusing to me

Table 6 is confusing to browse because it appears to be listing sometimes the inputs and sometimes the outputs (aka targets) of OCB (contrast this to Table 4 that only lists the outputs). I finding listing the outputs more intuitive, as it is usually what one is looking for (How do I invoke OCB?)

Suggestions

For consistency, I would suggest only listing output extensions.
Additionally, maybe we could make the examples stand out a little, rather than being in the middle of the text.

Examples of inconsistencies follow.

In the row

.c
.{o,obj}

The extension .c is the input and .{o,obj} is the output of the tool.

The row

.clib

Lists the input and does not list the output.

In row

.mltop
.top

We have one input .mltop and one output .top.

The documentation for clib files in `ocb -documentation` has a bug

I think that there are two problems:

While completely unambiguous for a program, prods it's not very obvious, in this case, for a human (especially because there are some branching and negation going on). So as a user of -documentation, doc is our best bet here.
The doc fragment is wrong, as it says "to build a dynamic library you should request libfoo.so" and it should say "you should request dllfoo.so".

rule "ocaml C stubs: clib & (o|obj)* -> (a|lib) & (so|dll)"
  ~deps:[ %(path)lib%(libname).clib ]
  ~prods:[ %(path:<**/>)lib%(libname:<*> and not <*.*>).a;
           %(path:<**/>)dll%(libname:<*> and not <*.*>).so ]
  ~doc:"OCamlbuild can create a C library by calling ocamlmklib with the file
        paths listed (one per line) in libfoo.clib. To build a static library
        from libfoo.clib, you should request libfoo.a (or libfoo.lib on
        Windows), and to build a dynamic library you should request libfoo.so
        (or libfoo.dll on Windows). Finally, any file listed in the .clib
        with name 'bar/baz.o' will link 'bar/baz.obj' instead on Windows.
        This means that using the .o extension will give portable clib files,
        even if it is not the object file extension on your system."
  <fun>

cogumbreiro commented 6 years ago

Again, @gasche, I am happy to submit patches for either problem.

And very importantly, let me also thank you for the wonderful work you've put in the tool and in the documentation. In my opinion, OCB is a great example of an OCaml project done right, especially because of its great documentation.

gasche commented 6 years ago

Thanks! Those are excellent comments, and (of course ?) I would be happy to receive contributions.

Regarding documentation output:

thanks for catching the ~doc mistake; this should be fixed indeed (a PR is welcome, otherwise I can fix it)
I realize that prods is not obvious to read for someone that is not familiar with ocamlbuild rule declarations, but I don't see right now how to do better without considerable effort. The output of -documentation is produced mechanically from rule declarations, and I see no easy way to print something better here. (Maybe one possibility would be to provide "examples" of environment values (path and libname) at rule-declaration time, to show deps/prods instantiation of those examples to the user. That sounds doable but still a couple hours of hacking -- I don't think we have the machinery to instantiate variables inside patterns for now -- rather than a minor change.)

Regarding the manual: I hadn't thought about it this way (about the confusion between "inputs" and "outputs"), that's a great perspective! I see that other figures in 2.2 are guilty of this as well. (For example Table 4 lists mllib and .cma at the same level.)

Let me first explain how the current state came to be. Mixing inputs and outputs makes some sense from a conceptual point of view because to ocamlbuild there is no difference, they are all "targets". A rule can build a target from another target (so from the rule perspective one is output and one is output), but users can add new rules to turn things that were previously inputs into outputs; for example, a .ml file may be generated from a .ml4, .mlp or .ml.cppo cases in many user build setups (but not with the default build rules).

On the other hand, I agree with you that this is confusing to the user. We could try to point out more explicitly which targets are generally outputs and which are generally inputs, and also not mix inputs and outputs of a rule together without saying anything. It is not completely clear to me what's a good way to give this information (or hint at this common structure) in a way that clears the confusion away without being too heavy-handed. Do you have suggestions?

ocaml / ocamlbuild