Transitive configure_make builds are broken on Darwin

[nomis52]

Description

A chain of 3 or more libraries fails to build on Darwin. Imagine we have the following:

libvolume -> libarea -> liblength

Then on Darwin, libvolume fails to build with:

EXT_BUILD_DEPS=/var/folders/qh/c65tnjdn605_7hw2_85jxrtw0000gn/T/tmp.bIJ95LxY
../libtool: line 8985: cd: ${EXT_BUILD_DEPS}/liblength/lib: No such file or directory
libtool: warning: cannot determine absolute directory name of '${EXT_BUILD_DEPS}/liblength/lib'
grep: ${EXT_BUILD_DEPS}/liblength/lib/liblength.la: No such file or directory
sed: ${EXT_BUILD_DEPS}/liblength/lib/liblength.la: No such file or directory

Cause

From what I can tell this is a result of the difference in the libtool .la files built on each platform.

After the build step for libarea completes, the libarea.la file contains:

Linux: dependency_libs=' -lstdc++ -lm -L${EXT_BUILD_DEPS}/liblength/lib'

Darwin: dependency_libs=' ${EXT_BUILD_DEPS}/liblength/lib/liblength.la -lstdc++ -lm -L${EXT_BUILD_DEPS}/liblength/lib'

When the libvolume build runs and we try to link, libtool (via ltmain.sh) executes the following

      for deplib in $dependency_libs; do
        path=
        case $deplib in
        -L*) path=$deplib ;;
        *.la)
          func_resolve_sysroot "$deplib"
          deplib=$func_resolve_sysroot_result
          func_dirname "$deplib" "" "."
         dir=$func_dirname_result                          # Line  8985 
         # We need an absolute path.
         case $dir in
            [\\/]* | [A-Za-z]:[\\/]*) absdir=$dir ;;

This doesn't understand env vars, in this case $EXT_BUILD_DEPS for *.la libs.

Steps to Reproduce

Use the configure branch of https://github.com/nomis52/rules_foreign_cc

cd examples/configure_transitive/
./build_tarballs.sh . # This will build tarballs for liblength, libarea & libvolume & put them in /tmp
bazel build -s --sandbox_debug  //configure_transitive:libvolume 

[nomis52]

@irengrig any ideas?

[ktf]

I've a similar problem while building svn (needed as mesos dependency):

libtool: warning: library '/var/folders/gg/xldx68351cd9k1rpb_0bhbc80000gn/T/tmp.inuzzJon/apr-util/lib/libaprutil-1.la' was moved.
/var/folders/gg/xldx68351cd9k1rpb_0bhbc80000gn/T/tmp.cBvwqZVn/libtool: line 8999: cd: ${EXT_BUILD_DEPS}/apr/lib: No such file or directory
libtool: warning: cannot determine absolute directory name of '${EXT_BUILD_DEPS}/apr/lib'
grep: ${EXT_BUILD_DEPS}/apr/lib/libapr-1.la: No such file or directory
sed: ${EXT_BUILD_DEPS}/apr/lib/libapr-1.la: No such file or directory
libtool:   error: '${EXT_BUILD_DEPS}/apr/lib/libapr-1.la' is not a valid libtool archive

Notice the files are actually there...

[wuzangsama]

same issue when I build eXosip depends on libosip2.a. libosip2.a is existing in copy_osip2 directory, but show 'was moved' error.

Platform: MacOS 10.15.2 bazel info release: 1.2.1

WORKSPACE:

    if "osip2" not in native.existing_rules():
        http_archive(
            name = "osip2",
            build_file_content = all_content,
            strip_prefix = "libosip2-5.1.0",
            urls = ["https://download.savannah.nongnu.org/releases/osip/libosip2-5.1.0.tar.gz"],
        )

    if "eXosip2" not in native.existing_rules():
        http_archive(
            name = "eXosip2",
            build_file_content = all_content,
            strip_prefix = "libexosip2-5.1.0",
            urls = ["https://download.savannah.nongnu.org/releases/exosip/libexosip2-5.1.0.tar.gz"],
        )

BUILD:

configure_make(
    name = "osip2",
    configure_env_vars = {
        "AR": "",
    },
    lib_source = "@osip2//:all",
    out_lib_dir = "lib",
    static_libraries = [
        "libosip2.a",
        "libosipparser2.a",
    ],
)

configure_make(
    name = "eXosip2",
    # configure_env_vars = {
    #     "AR": "",
    # },
    lib_source = "@eXosip2//:all",
    out_lib_dir = "lib",
    static_libraries = [
        "libexosip2.a",
    ],
    deps = [":osip2"],
)

[jheaff1]

I am also getting the libtool "library x was moved" error. I am trying to build log4cxx, which depends on apr and apr-util. apr-util itself depends on apr. I am using the configure_make rule and I'm getting the following error:

/bin/grep: ${EXT_BUILD_DEPS}/apr/lib/libapr-1.la: No such file or directory /bin/sed: can't read ${EXT_BUILD_DEPS}/apr/lib/libapr-1.la: No such file or directory

[whs-dot-hk]

Hi, I get it to work with make_commands below

https://github.com/whs-dot-hk/bazel_libass_test/blob/061ce3f1e19ead96bc3e0484ab1eb69f4f28d9f7/BUILD.libass#L31

[flaviut]

@whs-dot-hk That seems to work for me too, but would you mind explaining exactly what it does? I'm having a lot of trouble finding an explanation elsewhere.

[whs-dot-hk]

Hi, @flaviut , I think the line

"##define_absolute_paths## $$EXT_BUILD_DEPS$$ $$EXT_BUILD_DEPS$$"

is needed.

It was removed in #324. (I am very happy to put it back with another pr :smiley:)

On, linux, define_absolute_paths https://github.com/bazelbuild/rules_foreign_cc/blob/74b146dc87d37baa1919da1e8f7b8aafbd32acd9/tools/build_defs/shell_toolchain/toolchains/impl/linux_commands.bzl#L148 will only run replace_in_files https://github.com/bazelbuild/rules_foreign_cc/blob/74b146dc87d37baa1919da1e8f7b8aafbd32acd9/tools/build_defs/shell_toolchain/toolchains/impl/linux_commands.bzl#L55 which the sed of that command is the "magic" here...

Without define_absolute_paths the file uuid.pc is a symbolic link

└── libuuid
    ├── include
    │   └── uuid
    │       └── uuid.h -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/36/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/include/uuid/uuid.h
    ├── lib
    │   ├── libuuid.a -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/36/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/lib/libuuid.a
    │   ├── libuuid.la -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/36/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/lib/libuuid.la
    │   └── pkgconfig
    │       └── uuid.pc -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/36/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/lib/pkgconfig/uuid.pc
    ├── logs
    │   ├── Configure.log -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/36/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/logs/Configure.log
    │   └── Configure_script.sh -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/36/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/logs/Configure_script.sh

But a actual copy with define_absolute_paths

└── libuuid
    ├── include
    │   └── uuid
    │       └── uuid.h -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/37/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/include/uuid/uuid.h
    ├── lib
    │   ├── libuuid.a -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/37/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/lib/libuuid.a
    │   ├── libuuid.la
    │   └── pkgconfig
    │       └── uuid.pc
    ├── logs
    │   ├── Configure.log -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/37/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/logs/Configure.log
    │   └── Configure_script.sh -> /home/whs/.cache/bazel/_bazel_whs/2f08497982d73fd6bce8e85656ee6cfe/sandbox/linux-sandbox/37/execroot/__main__/bazel-out/k8-fastbuild/bin/external/util_linux/copy_libuuid/libuuid/logs/Configure_script.sh

And the .pc files will be like below, respectively

find /tmp/tmp.F9HLluKA3r -name "*.pc" -exec cat {} +
prefix=${EXT_BUILD_DEPS}/libuuid
exec_prefix=${EXT_BUILD_DEPS}/libuuid
libdir=${EXT_BUILD_DEPS}/libuuid/lib
includedir=${EXT_BUILD_DEPS}/libuuid/include

Name: uuid
Description: Universally unique id library
Version: 2.36.0
Requires:
Cflags: -I${includedir}/uuid
Libs: -L${libdir} -luuid
prefix=${EXT_BUILD_DEPS}/libuuid
exec_prefix=${EXT_BUILD_DEPS}/libuuid
libdir=${EXT_BUILD_DEPS}/libuuid/lib
includedir=${EXT_BUILD_DEPS}/libuuid/include

Name: uuid
Description: Universally unique id library
Version: 2.36.0
Requires:
Cflags: -I${includedir}/uuid
Libs: -L${libdir} -luuid
# fribidi.pc.  Generated from fribidi.pc.in by configure.

prefix=${EXT_BUILD_DEPS}/library
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: GNU FriBidi
Description: Unicode Bidirectional Algorithm Library
Version: 1.0.10
Libs: -L${libdir} -lfribidi
Cflags: -I${includedir}/fribidi
CFLAGS.private: -DFRIBIDI_LIB_STATIC
prefix=${EXT_BUILD_DEPS}/library
exec_prefix=${prefix}
libdir=${exec_prefix}/lib64
includedir=${prefix}/include

Name: expat
Version: 2.2.9
Description: expat XML parser
URL: http://www.libexpat.org
Libs: -L${libdir} -lexpat
Cflags: -I${includedir}
prefix=${EXT_BUILD_DEPS}/library
exec_prefix=${prefix}
libdir=${prefix}/lib64
includedir=${prefix}/include

Name: FreeType 2
URL: https://freetype.org
Description: A free, high-quality, and portable font engine.
Version: 23.2.17
Requires:
Requires.private: zlib
Libs: -L${libdir} -lfreetype
Libs.private: 
Cflags: -I${includedir}/freetype2

find /tmp/tmp.UkENFqTU8c -name "*.pc" -exec cat {} +
prefix=/tmp/tmp.UkENFqTU8c/libuuid
exec_prefix=/tmp/tmp.UkENFqTU8c/libuuid
libdir=/tmp/tmp.UkENFqTU8c/libuuid/lib
includedir=/tmp/tmp.UkENFqTU8c/libuuid/include

Name: uuid
Description: Universally unique id library
Version: 2.36.0
Requires:
Cflags: -I${includedir}/uuid
Libs: -L${libdir} -luuid
prefix=/tmp/tmp.UkENFqTU8c/libuuid
exec_prefix=/tmp/tmp.UkENFqTU8c/libuuid
libdir=/tmp/tmp.UkENFqTU8c/libuuid/lib
includedir=/tmp/tmp.UkENFqTU8c/libuuid/include

Name: uuid
Description: Universally unique id library
Version: 2.36.0
Requires:
Cflags: -I${includedir}/uuid
Libs: -L${libdir} -luuid
# fribidi.pc.  Generated from fribidi.pc.in by configure.

prefix=/tmp/tmp.UkENFqTU8c/library
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: GNU FriBidi
Description: Unicode Bidirectional Algorithm Library
Version: 1.0.10
Libs: -L${libdir} -lfribidi
Cflags: -I${includedir}/fribidi
CFLAGS.private: -DFRIBIDI_LIB_STATIC
prefix=/tmp/tmp.UkENFqTU8c/library
exec_prefix=${prefix}
libdir=${prefix}/lib64
includedir=${prefix}/include

Name: FreeType 2
URL: https://freetype.org
Description: A free, high-quality, and portable font engine.
Version: 23.2.17
Requires:
Requires.private: zlib
Libs: -L${libdir} -lfreetype
Libs.private: 
Cflags: -I${includedir}/freetype2
prefix=/tmp/tmp.UkENFqTU8c/library
exec_prefix=${prefix}
libdir=${exec_prefix}/lib64
includedir=${prefix}/include

Name: expat
Version: 2.2.9
Description: expat XML parser
URL: http://www.libexpat.org
Libs: -L${libdir} -lexpat
Cflags: -I${includedir}

[slsyy]

@jheaff1 Did you find the solution? I want to build exactly the same stack

[jheaff1]

Hi @slsyy,

Yes, thanks to @whs-dot-hk’s comment (https://github.com/bazelbuild/rules_foreign_cc/issues/344#issuecomment-691472066)

I just needed to set the make_commands option for the apr and apr-util build actions as below:

make_commands = [
    "##define_absolute_paths## $$EXT_BUILD_DEPS$$ $$EXT_BUILD_DEPS$$",
    "make",
    "make install",
  ],

[whs-dot-hk]

Hi,

I proposed a solution to fix this issue. #447

[attilaolah]

Even after #447 merged, I still encounter this issue plenty of times. At this point, I'm not sure whether the configure scripts are broken, or the generated pkg-config files are broken, or if it is just a bad behaviour of mixing various build systems (CMake, configure/make, etc.)

Specifically, when building libraries with configure_make(), and adding transitive dependencies that are built with cmake_external(), the link flags for transitive dependencies are all missing.

As a concrete example: I to build gdal or grass with configure_make(), with dependencies tiff, zstd or proj, which are all built with cmake_external(), the link flags for downstream dependencies are missing, e.g. -lpthread, -lzstd, -lwebp, etc.

In some rare cases, this seems to be a quirk of the configure script, where it seems to plain ignore LDFLAGS when trying to detect a library. In these cases I found that patching the configure script is the only way to go (but: yuck!).

In other cases, I can work around the issue by passing configure_env_vars = {"LDFLAGS": "…"}, listing all transitive dependencies. Which is rather annoying, but at least it works.

I'd be interested to know if this is a known limitation, or if maybe I'm doing something wrong somewhere.

Note that passing e.g. linkopts to the direct dependencies, that include downstream dependencies, doesn't seem to help either.

[attilaolah]

Also note that I'm generating static archives (all the way down to libc and friends). This seems to be a quirk in dealing with static archives: passing link flags of all downstream dependencies seems to be a requirement.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_foreign_cc!

[UebelAndre]

I believe this is fixed, at least as of 0.5.1? Can someone confirm that their issues is resolved?

[jheaff1]

@UebelAndre I am working on a PR to add log4cxx to the examples. This should serve as a regression test to catch this issue in the future.

[jheaff1]

@UebelAndre Please see PR #762

[jsharpe]

Closing as this is now tested via the log4cxx build.