Build fails on systems with LTO (link time optimization)

[houseofsuns]

On my Gentoo machine I saw the build fail to generate the file src/ctypes/ctypes_primitives.ml with the log at the bottom of this post.

The fundamental issue was that LTO removed the information searched for by gen_c_primitives. I tried to disable LTO in the build environment (I confirmed this), but the build system must have some auto-magic lookup of the global system configuration which sneaked LTO back in.

The following patch fixed the issue for me (it is sadly not portable I think).

diff --git a/Makefile b/Makefile
index 41a4f0f..2ca38ee 100644
--- a/Makefile
+++ b/Makefile
@@ -128,7 +128,7 @@ src/ctypes-foreign/dl_stubs.c: src/ctypes-foreign/dl_stubs.c$(OS_ALT_SUFFIX)

 src/ctypes/ctypes_primitives.ml: src/configure/extract_from_c.ml src/configure/gen_c_primitives.ml
    $(HOSTOCAMLFIND) ocamlc -o gen_c_primitives -package str -strict-sequence -linkpkg $^ -I src/configure
-   ./gen_c_primitives > $@ 2> gen_c_primitives.log || (rm $@ && cat gen_c_primitives.log || false)
+   CFLAGS="-fno-lto" ./gen_c_primitives > $@ 2> gen_c_primitives.log || (rm $@ && cat gen_c_primitives.log || false)

 src/ctypes-foreign/libffi_abi.ml: src/configure/extract_from_c.ml src/configure/gen_libffi_abi.ml
    $(HOSTOCAMLFIND) ocamlc -o gen_libffi_abi -package str -strict-sequence -linkpkg $^ -I src/configure

./gen_c_primitives > src/ctypes/ctypes_primitives.ml 2> gen_c_primitives.log || (rm src/ctypes/ctypes_primitives.ml && cat gen_c_primitives.log || false)
ocamlfind ocamlc -verbose -ccopt '-fPIC' -ccopt '-Wall' -ccopt '-g' -ccopt '-I/usr/lib64/libffi/include' -ccopt '"-I/usr/lib64/ocaml/integers"' -ccopt '-I' -ccopt '"/var/tmp/portage/dev-ml/ocaml-ctypes-0.20.0/work/ocaml-ctypes-0.20.0/src/ctypes"' -ccopt '-I' -ccopt '"/var/tmp/portage/dev-ml/ocaml-ctypes-0.20.0/work/ocaml-ctypes-0.20.0/tests"' '/var/tmp/portage/dev-ml/ocaml-ctypes-0.20.0/temp/ctypes_libffi_configdc3eb9.c' -c 1>&2
Effective set of compiler predicates: autolink,byte
+ ocamlc.opt -verbose -ccopt -fPIC -ccopt -Wall -ccopt -g -ccopt -I/usr/lib64/libffi/include -ccopt "-I/usr/lib64/ocaml/integers" -ccopt -I -ccopt "/var/tmp/portage/dev-ml/ocaml-ctypes-0.20.0/work/ocaml-ctypes-0.20.0/src/ctypes" -ccopt -I -ccopt "/var/tmp/portage/dev-ml/ocaml-ctypes-0.20.0/work/ocaml-ctypes-0.20.0/tests" -c /var/tmp/portage/dev-ml/ocaml-ctypes-0.20.0/temp/ctypes_libffi_configdc3eb9.c
+ x86_64-pc-linux-gnu-gcc -O2 -fno-strict-aliasing -fwrapv -pthread -fPIC -O2 -pipe -march=native -flto=auto -floop-nest-optimize -D_FILE_OFFSET_BITS=64    -c  -fPIC -Wall -g -I/usr/lib64/libffi/include "-I/usr/lib64/ocaml/integers" -I "/var/tmp/portage/dev-ml/ocaml-ctypes-0.20.0/work/ocaml-ctypes-0.20.0/src/ctypes" -I "/var/tmp/portage/dev-ml/ocaml-ctypes-0.20.0/work/ocaml-ctypes-0.20.0/tests"  -I'/usr/lib64/ocaml' '/var/tmp/portage/dev-ml/ocaml-ctypes-0.20.0/temp/ctypes_libffi_configdc3eb9.c'
Fatal error: exception Failure("sizeof: Char")

[atupone]

Hi maybe you need to recompile without LTO all the toolchain (findlib, dune, maybe other)

[houseofsuns]

Hi maybe you need to recompile without LTO all the toolchain (findlib, dune, maybe other)

gen_c_primitives does some configuration tests via compiling sample code and inspecting the resulting binary for introspecting the used toolchain. LTO caused the introspection to fail as it optimized away the markers injected via the sample code.

I don't see what recompiling the toolchain will change about that.

[atupone]

Well, the context of my comment was "the compilation flags that are applied are probably coming from dune or findlib"

[houseofsuns]

Well, the context of my comment was "the compilation flags that are applied are probably coming from dune or findlib"

Ah, that makes sense. However I would call this behavior a bug in those tools.

[atupone]

Looking at /usr/lib64/ocaml/Makefile.config it seems that all setting are stored in ocaml

[houseofsuns]

Hm, I can no longer reproduce the problem. So closing I'm closing this issue for now.