Meson not fully respecting -I/-L inside CLFAGS/LDFLAGS

It's not unusual for projects to have a custom contrib tree that contains bundled third party dependencies (+ their dependencies) to build these inside a custom tree (example). This is usually done by adding extra cflags and ldflags to set the correct -L and -I so that projects can find their own dependencies.

With autotools, for example, this works fine - doing the equivalent of cc.find_library() or cc.get_define() in autotools respects the set -L and -I set via e.g. the environment. With meson, however, this same approach does not work. Even with something like LDFLAGS="-L/path/to/contrib/build/lib" CFLAGS="$LDFLAGS -I/path/to/contrib/build/include" meson --prefix /path/to/contrib/build, meson fails to find a simple library or header, as in the following example:

$ cd /path/to/contrib && tree
.
├── bar.autotools
│   └── configure.ac
├── bar.meson
│   └── meson.build
└── build
    ├── include
    │   └── foo.h
    └── lib
        └── libfoo.so

5 directories, 4 files

bar.meson/meson.build:

project('bar', 'c')

cc = meson.get_compiler('c')
foo = cc.find_library('foo', required: true)
ver = cc.get_define('FOO_VERSION', prefix: '#include <foo.h>').to_int()

To work around the issue in this example, I could do something like this:

project('bar', 'c')

prefix = get_option('prefix')
prefix_lib = join_paths(prefix, get_option('libdir'))
prefix_inc = include_directories(join_paths(prefix, get_option('includedir')))

foo = cc.find_library('libfoo.so', dirs: prefix_lib, required: true)
ver = cc.get_define('FOO_VERSION',
    prefix: '#include <foo.h>',
    include_directories: prefix_inc,
).to_int()

Alternatively, I could expose the equivalent of prefix_lib and prefix_inc directly as string options that the user can provide.

However both of these approaches rely on either patching the meson.build of every project, or somehow communicating a change like this upstream to every one of the projects. An ideal solution would consist of some way to explicitly add extra lib/include search paths at configure time, either via CFLAGS/LDFLAGS as before, explicitly via some meson built-in option that we can set via -D, or implicitly via --prefix.

Okay, I've decided that the best solution to this is to have meson properly parse CLFAGS and LDFLAGS to get information about -I and -L provided by the environment, and use these for everything where it matters. Renaming the issue.

I've also tested that current meson already successfully finds libfoo.so, I just had an error in my example script. So the first line that still causes problems with current master is the cc.get_define.

Okay, I've decided that the best solution to this is to have meson properly parse CLFAGS and LDFLAGS to get information about -I and -L provided by the environment, and use these for everything where it matters.

And just how do can you guarantee that you always determine that last bit correctly? I'd argue that's in brittle AI land...

I just ran into this example of meson trying to be too smart in a situation where I need to inject an additional, compatibility library on a particular host, in order to be able to build a rather large and complex 3rd party bit of software. The standard approach with autoconf and cmake would be to specify that library via the LIBS or LDFLAGS env. variables. It would get added to every product generated by the linker, but that's probably what I'd want. Meson seems to presume that I'd be willing and able to dive into the build system in order to figure out exactly which targets require the library and then how to add the dependency.

And quite frankly, the only reason I use meson is because software I would like to use forces me to. I'm not interested in learning more about it than strictly necessary or to do development with/on it - and I'm pretty certain that I'm not the only consumer-type user who's been banging his head against the mesonry wall or "here, let me decide this for you"... :-/

The correct way to handle the initial issue is with what we call a "binary wrap", in other words, a small meson subproject that does something like:

project('bundled lib', 'c')

cc = meson.get_compiler('c')
lib = cc.find_library('mylocallib', dirs : ['.'])
inc = include_directorires('include')
bundled_dep = declare_dependency(
  dependencies : lib,
  include_directories : inc,
)

Then in your parent you would do something like:

libdep = dependency('libineed', fallback : ['bundledlibdir', 'bundled_dep'])

Which meson would query pkg-config, cmake, and whatever other methods are relevant for that method, then fall back to using the bundled lib if it can't find it on the system. Doing this search then fall-back will make linux distro maintainers happy, since they generally don't want your bundled libs.

@RJVB, your issue is actually a lot harder, because what you're trying to do is not use a bundled third party lib, but fix a problem with the upstream project definitions. The fact that a compatibility library is needed on an OS and the upstream project doesn't check for it seems more like a bug in their build scripts than in meson itself.

@RJVB, your issue is actually a lot harder

Actually it's not ... just don't do any second-guessing of what the operator tells you. As an analogy: CMake has a feature in which it replaces a full-path specification of a link library (/path/to/libfoo.ext) with the -lfoo short version when it finds the library in a standard location. I'm not entirely certain why, but it can cause very annoying problems if your intention is to link to that specific instance of the library. Fortunately CMake controls such things with policies most of which can be set on the commandline (or else, via a simple patch of the toplevel CMakeLists.txt).

, because what you're trying to do is not use a bundled third party lib, but fix a problem with the upstream project definitions. The fact that a compatibility library is needed on an OS and the upstream project doesn't check for it seems more like a bug in their build scripts than in meson itself.

No, I would argue against that. The need for a compatibility library can be the result of the operator wishing to use the project on an OS or OS version that is not (explicitly) supported by the project. I would rather consider it a mistake (aka design bug) to move to a build system that obliges you to foresee and work around every ideosyncrasy of every possible deployment environment that you do not intend to exclude. That's just as bad as having to litter your code with #ifdefs. Sometimes you have to, but the consensus in the circles where I move is to let the user fix such things with simple compiler or linker arguments if that's all it takes.

To put this into context: I ran into this when trying to build the current Enlightenment Foundation Libraries through MacPorts on an older OS X version that doesn't yet provide the ubiquitous Linux clock functions. MacPorts provides a LegacySupport library for such missing functions, which it injects via CPPFLAGS, Cxx_INCLUDE_PATH and LDLAGS. Somehow I got the build to complete by adding -L/opt/local/lib -lMacPortsLegacySupport to LDFLAGS instead of only -lMacPortsLegacySupport. Go figure why that would be necessary; /opt/local/lib is already added to the linker search path elsewhere because just about every non-OS library is taken from there.

Another case in point: libintl. I build a lot of stuff for deployment for my Linux rigs via a "LinuxPorts" adaptation of the MacPorts environment (which could become an actual thing btw). That means I build against a gettext set-up with an external libintl, which isn't the standard on Linux and typically obliges me to set LIBS=-lintl in the environment for building autoconf-based projects. I saw yesterday that meson figures this out by itself, which is nice ... but seems to me like an additional set of hoops that you couldn't be expected to have to jump through.

mesonbuild / meson

Meson not fully respecting -I/-L inside CLFAGS/LDFLAGS #4597