Make Meson aware of include directories and library paths

[jtojnar]

Boost does not support pkg-config, so to link against it, we rely on CC wrapper passing -isystem ${boost.dev}/include to the compiler and LD wrapper passing -L${boost}/lib to the linker to make them find Boost.

Meson 0.54.0 no longer uses the compiler to check for the presence of the Boost libraries so it cannot find them. Meson would need to know the include paths but extracting them from the compiler is not very easy.

It would be nice to let Meson know about the available include paths using either environment variables like CPATH (clang, gcc) and LIBRARY_PATH (gcc), or preferably a field in the cross file.

cc @Ericson2314

[jtojnar]

Quoting @mensinda from IRC:

To summarize, if you break up the boost packages any further, meson will never be able to find them (or only a part of them). We also need some reliable way to know where the header and library packages are installed.

This would preferably be the mentioned BOOST_ variables, but I am also open to other, sane, alternatives. So no extracting them from a big list that is hard to get (like compiler/linker flags) and has to be post-processed, but loading a NixOS specific .json file or calling a NixOS specific command that directly returns the wanted directories is fine.

I replied:

thanks, I will look into that. Would want to make it so that it is useful for others besides NixOS

[Ericson2314]

Can we use BOOST_INCLUDEDIR BOOST_LIBRARYDIR for everything? or does that assume it's all together?

[mensinda]

Setting BOOST_INCLUDEDIR and BOOST_LIBRARYDIR works, only BOOST_ROOT requires everything to be in a common root dir.

[jtojnar]

I meant this as more generic issue of making Meson aware so it does not step on our toes with stuff like rpath clearing so we do not need to use complicated patches like https://github.com/NixOS/nixpkgs/blob/69711a2f5ffe8cda208163be5258266172ff527f/pkgs/development/tools/build-managers/meson/fix-rpath.patch.

With https://github.com/mesonbuild/meson/pull/7103, we can make the patch much simpler:

--- a/mesonbuild/backend/backends.py
+++ b/mesonbuild/backend/backends.py
@@ -453,6 +453,21 @@ class Backend:
                 args.extend(self.environment.coredata.get_external_link_args(target.for_machine, lang))
             except Exception:
                 pass
+
+        nix_ldflags = os.environ.get('NIX_LDFLAGS', '').split()
+        next_is_path = False
+        # Try to add rpaths set by user or ld-wrapper
+        # https://github.com/NixOS/nixpkgs/blob/69711a2f5ffe8cda208163be5258266172ff527f/pkgs/build-support/bintools-wrapper/ld-wrapper.sh#L148-L177
+        for flag in nix_ldflags:
+            if flag == '-rpath' or flag == '-L':
+                next_is_path = True
+            elif next_is_path or flag.startswith('-L/'):
+                if flag.startswith('-L/'):
+                    flag = flag[2:]
+                if flag.startswith('@storeDir@'):
+                    dirs.add(flag)
+                next_is_path = False
+
         for arg in args:
             if arg.startswith('-Wl,-rpath='):
                 for dir in arg.replace('-Wl,-rpath=','').split(':'):

But if we come up with a better way to signal the library paths to Meson than having it parse NIX_LDFLAGS, it might be upstreamable.

[Ericson2314]

Ah yes that makes sense. I previously thought of trying to get Nixpkgs to provide pkg-config files packages that do not, so we don't need to rely on the wrapper scripts as much, but this is far far far less work and achieves much of the same benefit of the distro telling the build tool high level information rather than smuggling primitive information with wrappers.

[dankegel]

You've probably experimented with setting -L etc. in LDFLAGS; meson would honor those, I think. Is a meson change still needed now that https://github.com/mesonbuild/meson/pull/7103 has landed?

[jtojnar]

@dankegel We are using our own NIX_LDFLAGS variable and our compilers are wrapped to support that. I am not actually sure why we do not use regular LDFLAGS – perhaps some build tools filter that out but surely they would filter our own environment variable too.

I do not think there is much Meson can do with regards to Nixpkgs support unti we decide which way we want to go:

• Continue using NIX_LDFLAGS and patch Meson downstream with the patch from above – not really upstreamable, more of a hack.
• Switch to LDFLAGS as you suggest – no patching of meson should be needed but did not test.
• Come up with some other way (e.g. listing the dependency paths in the cross file) – not sure if this has any benefits compared to LDFLAGS solution.

Thanks for your work on https://github.com/mesonbuild/meson/pull/7103 by the way.

[Ericson2314]

I am not actually sure why we do not use regular LDFLAGS

Yes, I have wondered that myself. I suspect it is due to a fear of various make-based build systems overwriting it, but we should probably give it a shot. We do use PKG_CONFIG_PATH after all.

[dankegel]

Could you set LDFLAGS right before invoking meson, and thereby not affect other build systems?

Also, you're welcome :-) Gotta do something with this involuntary time off!

[Ericson2314]

Yes, but I soon as well use LDFLAGS everywhere :).

[stale[bot]]

I marked this as stale due to inactivity. → More info