Failed linking when trying to use zarith.h

[disteph]

I'm trying to convert between Z.t and the mpz_ptr C type reified in OCaml by Ctypes, following following my own feature request. My plan was to use zarith's C interface zarith.h since, at the ocaml level, Z.t is abstract. I'm failing to link

+ ocamlfind ocamlopt -linkpkg -g -package ctypes -package ctypes.foreign -package yices2_bindings -package sexplib -package containers -package ppx_deriving.std -cclib -lyices -ccopt -L/Users/e29546/.opam/4.10.0+flambda/lib/zarith/ -ccopt -I/Users/e29546/.opam/4.10.0+flambda/lib/zarith/ -cclib -lunix -cclib -lasmrun -cclib -lcamlrun -I src_tests src_tests/context_test.cmx src_tests/error_test.cmx src_tests/yices_runtime.cmx -o src_tests/yices_runtime.native
Undefined symbols for architecture x86_64:
  "_ml_z_from_mpz_ml", referenced from:
      _camlYices_runtime__entry in yices_runtime.o
      _camlYices2_high__x_26 in yices2_bindings.a(yices2_high.o)
      _camlZbindings__anon_fn$5bzbindings$2eml$3a9$2c0$2d$2d66$5d_24 in yices2_bindings.a(zbindings.o)
      _camlZbindings in yices2_bindings.a(zbindings.o)
  "_ml_z_mpz_init_set_z_ml", referenced from:
      _camlYices_runtime__entry in yices_runtime.o
      _camlYices2_high__x_26 in yices2_bindings.a(yices2_high.o)
      _camlZbindings__anon_fn$5bzbindings$2eml$3a6$2c0$2d$2d85$5d_17 in yices2_bindings.a(zbindings.o)
      _camlZbindings in yices2_bindings.a(zbindings.o)
  "_ml_z_mpz_set_z_ml", referenced from:
      _camlZbindings__anon_fn$5bzbindings$2eml$3a3$2c0$2d$2d75$5d_10 in yices2_bindings.a(zbindings.o)
      _camlZbindings in yices2_bindings.a(zbindings.o)

My understanding is that the implementation of the three functions offered in zarith.h is not found and, when I see the content of findlib's zarith directory, indeed I see zarith.h and only OCaml's compiled files, so I'm not sure how to link C code with them. As you can see in the command, I've tried many combinations of the C options I gathered from here and there.

Is this related to issue https://github.com/ocaml/Zarith/pull/8? I do not have a dllzarith.* file anywhere...

[antoinemine]

These functions are not in Zarith. There is a Zarith C function called ml_z_from_mpz (and similarly for the two others). It is defined in caml_z.c and should be available in all the compiled Zarith library, whatever format your link chooses (.a, .so, etc.). I understand that you use Ctypes to generate the call to ml_z_from_mpz. Are the missing functions actually supposed to be generated by Ctypes ? Moreover, I don't see the Zarith library in your ocamlfind command line. Maybe there's a -package missing in your build as well. I don't think this is related to #8, nor to zarith.h.

[disteph]

Thanks for the quick response! Indeed, I don't know where the names with the initial underscore come from. I try to call the zarith.h API from the following C stubs:

#include <stdlib.h>
#include <stdint.h>
#include <gmp.h>
#include <zarith.h>
#include <caml/mlvalues.h>
#include <caml/memory.h>
#include <caml/alloc.h>

/* sets rop to the value in op (limbs are copied) */
CAMLprim value ml_z_mpz_set_z_ml(value rop, value op) {
  CAMLparam2(rop, op);
  mpz_ptr z = (mpz_ptr) (Data_custom_val(rop));
  /* void ml_z_mpz_set_z(mpz_t rop, value op); */
  ml_z_mpz_set_z(z, op);
  CAMLreturn(Val_unit);
}

/* inits and sets rop to the value in op (limbs are copied) */
CAMLprim value ml_z_mpz_init_set_z_ml(value rop, value op) {
  CAMLparam2(rop, op);
  mpz_ptr z = (mpz_ptr) (Data_custom_val(rop));
  /* void ml_z_mpz_init_set_z(mpz_t rop, value op); */
  ml_z_mpz_init_set_z(z, op);
  CAMLreturn(Val_unit);
}

/* returns a new z objects equal to op (limbs are copied) */
CAMLprim value ml_z_from_mpz_ml(value rop) {
  CAMLparam1(rop);
  mpz_ptr z = (mpz_ptr) (Data_custom_val(rop));
  /* value ml_z_from_mpz(mpz_t op); */
  CAMLreturn(ml_z_from_mpz(z));
}

and then in a file zbindings.ml I have

open Gmp

external ml_z_mpz_set_z_ml : mpz_ptr -> Z.t -> unit
  = "ml_z_mpz_set_z_ml"

external ml_z_mpz_init_set_z_ml : mpz_ptr -> Z.t -> unit
  = "ml_z_mpz_init_set_z_ml"

external ml_z_from_mpz_ml : mpz_ptr -> Z.t 
  = "ml_z_from_mpz_ml"

I've added the -package zarith to the command but it doesn't change anything...

[antoinemine]

It seems to me that all of this is not relevant de Zarith nor zarith.h. If the functions that are missing (ml_z_mpz_set_z_ml and co.) are defined in your stubs; you must ensure that your own C file is compiled and linked. I don't see any Zarith symbol missing (it may happen later...).

Are you on a MacOS X ? Old (pre 64-bit) compilers, and apparently MacOS X for some reason, like to add underscore prefixes.

[disteph]

Yes, you are totally right, it's the implementation of my stubs that's not found. Even though I've got a .o and .so which I think contains them. And yes, I'm on MacOS X (and new to Mac...). This has nothing to do with zarith, apologies...

[disteph]

That being said, while I can compile the stub above with gcc, as a file zbindings.o, it doesn't seem to link well with zarith. I'd typically want to link to zarith dynamically, and I'm not sure how to complete the command for my own stub

gcc zbindings.o -shared -I/Users/e29546/.opam/4.10.0+flambda/lib/zarith/ -I/Users/e29546/.opam/4.10.0+flambda/lib/ocaml

to refer to the zarith files in the findlib directory. Should I have a a libzarith.so / dllzarith.so and have a -lzarith option? At the moment the above command produces

Undefined symbols for architecture x86_64:
  "_Caml_state", referenced from:
      _ml_z_mpz_set_z_ml in zbindings.o
      _ml_z_mpz_init_set_z_ml in zbindings.o
      _ml_z_from_mpz_ml in zbindings.o
  "_ml_z_from_mpz", referenced from:
      _ml_z_from_mpz_ml in zbindings.o
     (maybe you meant: _ml_z_from_mpz_ml)
  "_ml_z_mpz_init_set_z", referenced from:
      _ml_z_mpz_init_set_z_ml in zbindings.o
     (maybe you meant: _ml_z_mpz_init_set_z_ml)
  "_ml_z_mpz_set_z", referenced from:
      _ml_z_mpz_set_z_ml in zbindings.o
     (maybe you meant: _ml_z_mpz_set_z_ml)

and unlike my original issue, these look like the zarith functions from zarith.h with the prefix underscore, plus a Caml_state function I didn't know I was using. Sorry, it's probably a generic C/Ocaml interfacing issue that I'm missing, but the learning curve is steep...

[xavierleroy]

Creating shared libraries in macOS is a black art. You may need one or several of the following options: -flat_namespace -undefined suppress -Wl,-no_compact_unwind.

This said, if your objective is to create an OCaml library that integrates some C stub code, ocamlmklib automates much of the process, including the magic macOS options.

Finally, after 6 messages I still don't have a clear idea of what you're trying to do, so I'm going to close this report pretty soon.

[disteph]

Thanks for the tips. I'll try that. What I'm trying to do is simply to have Ocaml conversion functions between Zarith's Z.t and the OCaml representation of mpz_t or mpz_ptr via Ctypes, but the discussion on the ctypes issue clearly shows that it's more complicated than I thought... What I was more precisely trying to do above, as part of that effort, was to dynamically bind the implementation of zarith.h, but yes, you can close the issue. Thanks!

[disteph]

For the record, https://github.com/fdopen/ctypes-zarith completed what I wanted to do here.