Error building if directory path to opam switch contains "host" somewhere in it

[pablo-meier]

Edit: diagnosed the problem here, original description left for posterity, but the real description of the issue is in the linked comment:

===

Hi! 👋 I'm on an M1 Mac, and opam install luv is throwing me some errors:

pablo:hosted-tweet-reader/ $ opam install luv
The following actions will be performed:
  ∗ install luv 0.5.11

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
⬇ retrieved luv.0.5.11  (cached)
[ERROR] The compilation of luv.0.5.11 failed at "dune build -p luv -j 9".

#=== ERROR while compiling luv.0.5.11 =========================================#
# context     2.1.3 | macos/arm64 |  | https://opam.ocaml.org#dfad10f2
# path        ~/projects/hosted-tweet-reader/_opam/.opam-switch/build/luv.0.5.11
# command     ~/.opam/opam-init/hooks/sandbox.sh build dune build -p luv -j 9
# exit-code   1
# env-file    ~/.opam/log/luv-13645-28472b.env
# output-file ~/.opam/log/luv-13645-28472b.out
### output ###
# c_generated_functions.c:1704:36: warning: passing 'unsigned char *' to parameter of type 'char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not [-Wpointer-sign]
# [...]
#                                    ^~~~~
# vendor/libuv/include/uv.h:1260:52: note: passing argument to parameter 'buffer' here
# UV_EXTERN int uv_os_getenv(const char* name, char* buffer, size_t* size);
#                                                    ^
# c_generated_functions.c:1737:29: warning: passing 'unsigned char *' to parameter of type 'char *' converts between pointers to integer types where one is of the unique plain 'char' type and the other is not [-Wpointer-sign]
#    int x1340 = luv_os_uname(x1339);
#                             ^~~~~
# ./helpers.h:186:24: note: passing argument to parameter 'buffer' here
# int luv_os_uname(char *buffer);
#                        ^
# 15 warnings generated.

I don't see -Werror in the output file that 👆 generates, but I do see them in the project directory sources:

pablo:my-project/ $ rg '\-Werror' _opam/.opam-switch

_opam/.opam-switch/build/luv.0.5.11/src/c/vendor/configure/configure
4063:      # or remarks (even with -Werror).  So we grep stderr for any message
4113:     CFLAGS="$CFLAGS -Werror"
4130:  cc_cv_werror=-Werror
13032:        # that build with -Werror.  So if the active version of Clang has
13054:                CFLAGS="-Werror -Wunknown-warning-option $ax_pthread_try -pthread $ax_pthread_save_CFLAGS"

_opam/.opam-switch/build/luv.0.5.11/src/c/vendor/libuv/configure.ac
21:AM_INIT_AUTOMAKE([-Wall -Werror foreign subdir-objects] UV_EXTRA_AUTOMAKE_FLAGS)

_opam/.opam-switch/sources/ocaml-base-compiler.5.0.0/aclocal.m4
85:  CFLAGS="-Werror -fno-tree-vrp $CFLAGS"
107:  CFLAGS="-Werror $CFLAGS"

_opam/.opam-switch/sources/ocaml-base-compiler.5.0.0/configure
13543:  warn_error_flag='-Werror'
15164:  CFLAGS="-Werror -fno-tree-vrp $CFLAGS"
15214:  CFLAGS="-Werror $CFLAGS"
18358:        # that build with -Werror.  So if the active version of Clang has
18382:                CFLAGS="-Werror -Wunknown-warning-option $ax_pthread_try -pthread $ax_pthread_save_CFLAGS"

_opam/.opam-switch/sources/ocaml-base-compiler.5.0.0/configure.ac
609:  warn_error_flag='-Werror'

_opam/.opam-switch/sources/ocaml-base-compiler.5.0.0/build-aux/ax_pthread.m4
382:        # that build with -Werror.  So if the active version of Clang has
399:                CFLAGS="-Werror -Wunknown-warning-option $ax_pthread_try -pthread $ax_pthread_save_CFLAGS"

I'll keep digging to see if I can disable the flags, but wanted to know if you knew a better remedy, and/or call it to your attention if you were curious and developing on Linux or something. Hope you're well! 😄

[pablo-meier]

These lines looks promising:

(cd _build/default/src/c/vendor/libuv && /bin/bash -e -u -o pipefail -c 'sh configure --host `ocamlc -config | awk '\''/host/ {print $NF}'\''` '\''CC=cc -O2 -fno-strict-a
liasing -fwrapv -pthread'\'' CFLAGS=-DNDEBUG --silent --enable-silent-rules')
configure: WARNING: you should use --build, --host, --target
configure: WARNING: invalid host type: /Users/pablo/projects/my-project/_opam/lib/ocaml
configure: WARNING: you should use --build, --host, --target                                                                                                              Invalid configuration `/Users/pablo/projects/my-project/_opam/lib/ocaml': machine `/Users/pablo/projects/my-project' not recognized
configure: error: /bin/sh ./config.sub /Users/pablo/projects/hosted-tweet-reader/_opam/lib/ocaml failed
(cd _build/default/src/c && /usr/bin/cc -O2 -fno-strict-aliasing -fwrapv -pthread -D_FILE_OFFSET_BITS=64 -O2 -fno-strict-aliasing -fwrapv -pthread -g -I /Users/pablo/proj
ects/hosted-tweet-reader/_opam/lib/ocaml -I /Users/pablo/projects/my-project_opam/lib/bigarray-compat -I /Users/pablo/projects/hosted-tweet-reader/_opam/lib/byt
es -I /Users/pablo/projects/my-project/_opam/lib/ctypes -I /Users/pablo/projects/my-project/_opam/lib/integers -I /Users/pablo/projects/hosted-tweet-rea
der/_opam/lib/ocaml/threads -I /Users/pablo/projects/my-project/_opam/lib/ocaml/unix -I /Users/pablo/projects/my-project/_opam/lib/stdlib-shims -I . -I
vendor/libuv/include -o c_generated_functions.o -c c_generated_functions.c)

Line singled out:

Invalid configuration `/Users/pablo/projects/my-project/_opam/lib/ocaml': machine `/Users/pablo/projects/my-project' not recognized

Seems to be using my directory name as an architecture option? May be an M1 thing?

[pablo-meier]

Ahhhh I got it, it's pretty funny.

In the lines I've pasted above, I subbed out the name of my project to my-project to make it easier to generalize. The actual name of my project was hosted-tweet-reader. So this line

cd _build/default/src/c/vendor/libuv && /bin/bash -e -u -o pipefail -c 'sh configure --host `ocamlc -config | awk '\''/host/ {print $NF}'\''` '\''CC=cc -O2 -fno-strict-a
liasing -fwrapv -pthread'\'' CFLAGS=-DNDEBUG --silent --enable-silent-rules'

Uses awk to set the host flag:

sh configure --host `ocamlc -config | awk '\''/host/ {print $NF}'\''`

but that was picking up the directory name, /Users/pablo/projects/hosted-tweet-reader/..., thus setting the host to an invalid value.

Updating the title. I changed the name of the project, haha.

[aeghost]

Hi, I have the same error using BuildRoot to cross compile OCaml 5.0.0 with Eio for our embedded product.

The problem : BuildRoot seems to create a folder with target HOST type name in it,

Example of typical ocamlc path, after rebuilding OPAM and initializing switch_name with Ocaml 5.0.0 : /project/_build/vm-name-dev/per-package/host-opam/host/x86_64-product-linux-gnu/sysroot/opamroot/switch_name/bin/ocamlc

And it seems /project/_build/vm-name-dev/per-package/host-opam/host/x86_64-product-linux is not a recognized machine.

This is pretty dark to have this type of problem with a lib, built in sandboxed environment.

I will recommend to rewrite your machine detection code that seems to be an horrible hack instead of a secure way to get host name or host type; Or document a way a force it. (Maybe it already exists, but I didn't find it)

What is making me really sad is that even if Eio_main needs Eio_luv that needs Luv, the libluv is never linked in any of our binaries. But this is an Eio problem not yours. (I use zmq-lwt to manage most clients/servers (legacy), and Lwt_eio to convert program to Eio, then Eio_main.run, so why should I need Luv...)

Wishing you a very good day.

[aeghost]

Proposed fix

--- a/src/c/dune
+++ b/src/c/dune
@@ -72,7 +72,7 @@ let () = Jbuild_plugin.V1.send @@ {|
   (bash "cp -r vendor/configure/* vendor/libuv/")
   (chdir vendor/libuv (progn
    (bash
-    "sh configure --host `ocamlc -config | awk '/host/ {print $NF}'` \
+    "sh configure --host `ocamlc -config-var host` \
       'CC=%{cc}' CFLAGS=-DNDEBUG --silent --enable-silent-rules")
    (ignore-outputs (bash
     "$([ '%{os_type}' = Unix ] && echo %{make} || echo make) V=0 -j 4 \

But ocaml -config-var host is available only with ocaml 4.08+ "ocaml" {>= "4.02.0"} should be updated to "ocaml" {>= "4.08.0"}

[aantron]

Thanks @pablo-meier and @aeghost! The above commit, c7b3b6d02121e97760e71c2bfa7bec9ece98986c, changes the awk pattern from /host/ to /^host:/, which appears to fix the immediate problem. I tested this by building in a path that contained "ghost".

I don't think ocaml -config-var host is worth dropping support for 4.02-4.07. If you can think of a more correct and safe but still brief way of extracting the host field from the toolchain config, I will be happy to adopt that instead of /^host:/.

[aantron]

This will be released in 0.5.12.