Build issue with opam in llvm-15.0.0

[tanapthetimid]

The build fails with The script must be executed from its current directory in setup.sh. I added cd $(dirname $0) to the top of the file and removed the exit inside the check and it builds fine. Not sure if this is an issue with my setup.

[alan-j-hu]

I am capable of running ./setup.sh llvm-config-15 from the root of the repository and the script works fine. I am also able to install this package from opam.

How are you installing it? Is this happening when you are installing the package from opam?

[tanapthetimid]

This happens when I'm installing normally from opam (2.1.2).

[alan-j-hu]

Can you please run opam install llvm --verbose and post the output?

For what it's worth, I apparently have opam 2.1.5 installed.

[tanapthetimid]

Here is the output. I am inside a podman container with ubuntu 22.04 image if that makes any difference.

The following actions will be performed:
  - install conf-bash         1            [required by conf-llvm]
  - install dune              3.10.0       [required by llvm]
  - install conf-llvm         15           [required by llvm]
  - install stdlib-shims      0.3.0        [required by integers]
  - install csexp             1.5.2        [required by dune-configurator]
  - install bigarray-compat   1.1.0        [required by ctypes]
  - install integers          0.7.0        [required by ctypes]
  - install dune-configurator 3.10.0       [required by ctypes]
  - install ctypes            0.21.1       [required by llvm]
  - install llvm              15.0.7+nnp-2
===== 10 to install =====
Do you want to continue? [Y/n] y

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
-> retrieved bigarray-compat.1.1.0  (cached)
-> retrieved csexp.1.5.2  (cached)
-> retrieved ctypes.0.21.1  (cached)
+ /usr/bin/bash "--version" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/conf-bash.1)
- GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
- Copyright (C) 2020 Free Software Foundation, Inc.
- License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
- 
- This is free software; you are free to change and redistribute it.
- There is NO WARRANTY, to the extent permitted by law.
- GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
- Copyright (C) 2020 Free Software Foundation, Inc.
- License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
- 
- This is free software; you are free to change and redistribute it.
- There is NO WARRANTY, to the extent permitted by law.
-> compiled  conf-bash.1
-> installed conf-bash.1
-> retrieved integers.0.7.0  (cached)
+ /usr/bin/bash "configure.sh" "15" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/conf-llvm.15)
- + version=15
- + hash brew
- + shopt -s nullglob
- + for llvm_config in llvm-config-$version llvm-config-${version}.0 llvm-config${version}0 llvm-config-mp-$version llvm-config-mp-${version}.0 llvm${version}-config llvm-config-${version}-32 llvm-config-${version}-64 llvm-config $brew_llvm_config
- ++ llvm-config-15 --version
- + llvm_version=15.0.7
- + case $llvm_version in
- + echo 'config: "llvm-config-15"'
- + echo 'version: "15.0.7"'
- + exit 0
- + version=15
- + hash brew
- + shopt -s nullglob
- + for llvm_config in llvm-config-$version llvm-config-${version}.0 llvm-config${version}0 llvm-config-mp-$version llvm-config-mp-${version}.0 llvm${version}-config llvm-config-${version}-32 llvm-config-${version}-64 llvm-config $brew_llvm_config
- ++ llvm-config-15 --version
- + llvm_version=15.0.7
- + case $llvm_version in
- + echo 'config: "llvm-config-15"'
- + echo 'version: "15.0.7"'
- + exit 0
-> compiled  conf-llvm.15
-> installed conf-llvm.15
-> retrieved dune.3.10.0  (cached)
-> retrieved dune-configurator.3.10.0  (cached)
-> retrieved stdlib-shims.0.3.0  (cached)
-> retrieved llvm.15.0.7+nnp-2  (https://opam.ocaml.org/cache)
+ /home/tanapoom/.opam/test-switch/bin/ocaml "boot/bootstrap.ml" "-j" "15" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/dune.3.10.0)
- ocamlc -output-complete-exe -w -24 -g -o .duneboot.exe -I boot -I +unix unix.cma boot/libs.ml boot/duneboot.ml
- ./.duneboot.exe -j 15
- 
+ /home/tanapoom/.opam/test-switch/.opam-switch/build/dune.3.10.0/./_boot/dune.exe "build" "dune.install" "--release" "--profile" "dune-bootstrap" "-j" "15" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/dune.3.10.0)
-> compiled  dune.3.10.0
-> installed dune.3.10.0
+ /home/tanapoom/.opam/test-switch/bin/dune "build" "-p" "stdlib-shims" "-j" "15" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/stdlib-shims.0.3.0)
-> compiled  stdlib-shims.0.3.0
-> installed stdlib-shims.0.3.0
+ /home/tanapoom/.opam/test-switch/bin/dune "build" "-p" "csexp" "-j" "15" "@install" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/csexp.1.5.2)
-> compiled  csexp.1.5.2
-> installed csexp.1.5.2
+ /home/tanapoom/.opam/test-switch/bin/dune "build" "-p" "bigarray-compat" "-j" "15" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/bigarray-compat.1.1.0)
-> compiled  bigarray-compat.1.1.0
-> installed bigarray-compat.1.1.0
+ /home/tanapoom/.opam/test-switch/bin/dune "build" "-p" "integers" "-j" "15" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/integers.0.7.0)
-> compiled  integers.0.7.0
-> installed integers.0.7.0
+ /usr/bin/rm "-rf" "vendor/csexp" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/dune-configurator.3.10.0)
+ /usr/bin/rm "-rf" "vendor/pp" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/dune-configurator.3.10.0)
+ /home/tanapoom/.opam/test-switch/bin/dune "build" "-p" "dune-configurator" "-j" "15" "@install" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/dune-configurator.3.10.0)
-> compiled  dune-configurator.3.10.0
-> installed dune-configurator.3.10.0
+ /home/tanapoom/.opam/test-switch/bin/dune "build" "-p" "ctypes" "-j" "15" "--promote-install-files=false" "@install" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/ctypes.0.21.1)
+ /home/tanapoom/.opam/test-switch/bin/dune "install" "-p" "ctypes" "--create-install-files" "ctypes" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/ctypes.0.21.1)
-> compiled  ctypes.0.21.1
-> installed ctypes.0.21.1
+ /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/./setup.sh "llvm-config-15" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2)
- + dirname /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/./setup.sh
- + test /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/. != .
- + echo The script must be executed from its current directory.
- The script must be executed from its current directory.
- + exit 1
[ERROR] The compilation of llvm.15.0.7+nnp-2 failed at "./setup.sh llvm-config-15".

#=== ERROR while compiling llvm.15.0.7+nnp-2 ==================================#
# context     2.1.2 | linux/x86_64 | ocaml-base-compiler.5.0.0 | https://opam.ocaml.org#ba404223
# path        ~/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2
# command     ~/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/./setup.sh llvm-config-15
# exit-code   1
# env-file    ~/.opam/log/llvm-510774-e2da2d.env
# output-file ~/.opam/log/llvm-510774-e2da2d.out
### output ###
# + dirname /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/./setup.sh
# + test /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/. != .
# + echo The script must be executed from its current directory.
# The script must be executed from its current directory.
# + exit 1

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+- The following actions failed
| - build llvm 15.0.7+nnp-2
+- 
+- The following changes have been performed
| - install bigarray-compat   1.1.0
| - install conf-bash         1
| - install conf-llvm         15
| - install csexp             1.5.2
| - install ctypes            0.21.1
| - install dune              3.10.0
| - install dune-configurator 3.10.0
| - install integers          0.7.0
| - install stdlib-shims      0.3.0
+- 

The former state can be restored with:
    /usr/bin/opam switch import
"/home/tanapoom/.opam/test-switch/.opam-switch/backup/state-20230915082615.export"
'opam install llvm --verbose' failed.

[alan-j-hu]

Thank you for providing the output. This seems to be the difference between your --verbose and my --verbose:

Yours:

+ /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/./setup.sh "llvm-config-15" (CWD=/home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2)
- + dirname /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/./setup.sh
- + test /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/. != .
- + echo The script must be executed from its current directory.
- The script must be executed from its current directory.
- + exit 1

Mine:

+ /home/alan/.opam/opam-init/hooks/sandbox.sh "build" "./setup.sh" "llvm-config-15" (CWD=/home/alan/.opam/5.1.0/.opam-switch/build/llvm.15.0.7+nnp-2)
- + dirname ./setup.sh
- + test . != .
- + test 1 -ne 1

dirname just chops off the directory portion of the path and doesn't do any normalization of the path. This is the reason why the script is brittle.

I'm curious about the different ways in which opam is invoking the script; /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/./setup.sh vs /home/alan/.opam/opam-init/hooks/sandbox.sh "build" "./setup.sh". Would this have to do with opam 2.1.2 vs 2.1.5, or opam being run in a container? Nevertheless, I don't feel that this difference in invoking the script should be something that breaks it.

[tanapthetimid]

I see, this makes a lot of sense now. opam uses bubblewrap to sandbox which doesn't work and is disabled in containers. It seems like the script is invoked from a different directory when not executed inside the sandbox.

I don't think this should break it either, but the script explicitly exits if the directory in which opam invokes the script is not the same as that of the script. To make sure that nothing breaks I added cd $(dirname $0) to the script and removed the exit condition and everything works fine.

[alan-j-hu]

Maybe the difference in invocations has to do with bubblewrap, but I'm not sure if the script is even being invoked from a different directory. According to your verbose output, the script is being invoked as /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/./setup.sh and your CWD is /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2. The problem is simply that dirname /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/./setup.sh is /home/tanapoom/.opam/test-switch/.opam-switch/build/llvm.15.0.7+nnp-2/., which isn't equal to . according to test even if they may be logically equal paths.

[tanapthetimid]

You're right. Even so, I don't see any reason for the test to be there. If the script needs to run in a specific directory it can just cd there. And if the test really needs to be there maybe something like checking if pwd is equal to dirname $(readlink -f $0)

[alan-j-hu]

Could you please test https://github.com/kit-ty-kate/llvm-dune/pull/17 and see if it fixes the problem? I am not a Bash expert, but am getting the answer from https://stackoverflow.com/questions/33980224/how-to-check-if-two-paths-are-equal-in-bash.

[tanapthetimid]

Yup, it works now. Thanks!

[alan-j-hu]

Do you think this fix should warrant a new release on opam? Or should I try to investigate the other issue you opened about violation of the 2-bit alignment assumption (I'm not sure how long it will take), and then put out a new release after resolving that issue as well?

[tanapthetimid]

Personally I don't need it in a hurry since I'm working off of a fork.