Meson no longer able to pick up Boost

emilazy commented 4 years ago

(originally about pulseeffects build)

Broken on Hydra for a few days now; going by the log it seems to be a boost issue:

Run-time dependency Boost (missing: filesystem, system) found: NO 

src/meson.build:97:0: ERROR: Dependency "boost" not found

but removing the boost version override introduced in 470b916556a800147fce11aef05fc503663cbdba didn't fix it (and https://github.com/wwmm/pulseeffects/issues/645 suggests it's required anyway). Unfortunately I'm not very familiar with boost or meson so I'm not sure what to poke at next, but I'll update this issue if I figure anything out.

cc @jtojnar @matthiasbeyer

jtojnar commented 4 years ago

Possibly a meson bump changed boost detection?

jtojnar commented 4 years ago

Yeah, looks like it touched boost: https://github.com/mesonbuild/meson/compare/0.53...0.54.0 And https://github.com/mesonbuild/meson/compare/0.54.0...0.54.1 did too so maybe we just need to update.

emilazy commented 4 years ago

Makes sense. Bisecting the build to see exactly what nixpkgs commit broke things; would try with a bumped meson afterwards but I guess that would need a stdenv rebootstrap?

jtojnar commented 4 years ago

I think creating a directory containing meson.build file with

project(
    'pulseeffects',
    'c',
    'cpp',
    default_options : ['cpp_std=c++17','buildtype=debugoptimized'],
    version: '4.7.2',
    meson_version: '>= 0.40.0'
)
dependency('boost', version: '>=1.65', modules:['system','filesystem'])

and changing the pulseeffects src to point to that and removing all dependencies except for meson, ninja and boost from the pulseeffects expression might avoid that.

Or you can clone meson, update that and use it for pulseeffects. That should limit the fallback even more (if boost or ninja depend on meson transitively).

emilazy commented 4 years ago

Unfortunately, same error even with:

diff --git a/pkgs/applications/audio/pulseeffects/default.nix b/pkgs/applications/audio/pulseeffects/default.nix
index 2972849a2d4..a93fb6c48fe 100644
--- a/pkgs/applications/audio/pulseeffects/default.nix
+++ b/pkgs/applications/audio/pulseeffects/default.nix
@@ -32,6 +32,7 @@
 , rubberband
 , mda_lv2
 , lsp-plugins
+, python3Packages
 }:

 let
@@ -56,7 +57,15 @@ in stdenv.mkDerivation rec {
   };

   nativeBuildInputs = [
-    meson
+    (meson.overrideAttrs (_: rec {
+      pname = "meson";
+      version = "0.54.1";
+      name = "${pname}-${version}";
+      src = python3Packages.fetchPypi {
+        inherit pname version;
+        hash = "sha256-L3b7RXJ2K+E+5HkpJhAJG0UJr1eIvM6zkf4iK80Cltw=";
+      };
+    }))
     ninja
     pkgconfig
     libxml2

Not entirely sure this is enough to get the right version of meson used, though.

jtojnar commented 4 years ago

Yeah, that should be enough.

Hmm. That is unfortunate. Reverting back to 0.53.2 seems to fix the issue:

--- a/pkgs/applications/audio/pulseeffects/default.nix
+++ b/pkgs/applications/audio/pulseeffects/default.nix
@@ -1,5 +1,6 @@
 { stdenv
 , fetchFromGitHub
+, pkgs
 , meson
 , ninja
 , pkgconfig
@@ -56,7 +57,14 @@ in stdenv.mkDerivation rec {
   };

   nativeBuildInputs = [
-    meson
+    (let
+      nixpkgs = pkgs.fetchzip {
+        name = "nixpkgs-with-meson-0.53.2";
+        url = "https://github.com/NixOS/nixpkgs/archive/9073a0cb8b2f419785a60969e11e96733f29b200%5E.tar.gz";
+        hash = "sha256-oC94cOmB7dxpWzSXlLNt2OyhEt20qX9murBWUh52HLo=";
+      };
+    in
+      pkgs.callPackage "${nixpkgs}/pkgs/development/tools/build-managers/meson/default.nix" {})
     ninja
     pkgconfig
     libxml2

We will need to analyze the commits that touched boost and fix them.

emilazy commented 4 years ago

I'm guessing https://github.com/mesonbuild/meson/commit/08224dafcba1b694fb624553e7d84deb565aae22 is the place to start looking.

jtojnar commented 4 years ago

Looking at https://github.com/mesonbuild/meson/pull/6602/files#diff-69d98b8292053a53f88d3015eab2d5daL87, they no longer seem to use compiler.find_library() and instead try to find boost installation roots.

We pass -isystem /nix/store/zyrg127fd6sk40dx576mcsr4a2jxcxqq-boost-1.72.0-dev/include to the compiler and -L/nix/store/97smy1b58mq6n9kaha4icfz7gqq8pyx6-boost-1.72.0/lib to the linker so compiler.find_library() was able to find boost but the new mechanism no longer subscribes to that.

jtojnar commented 4 years ago

Setting the following environment variables fixes the Meson’s Boost detection:

  # Meson is no longer able to pick up Boost automatically.
  # https://github.com/NixOS/nixpkgs/issues/86131
  BOOST_INCLUDEDIR = "${stdenv.lib.getDev boost}/include";
  BOOST_LIBRARYDIR = "${stdenv.lib.getLib boost}/lib";

Not sure if we should do that in Boost setup hook or make Meson support find_library again.

emilazy commented 4 years ago

The setup hook seems like a good solution to me. Weirdly, setting BOOST_ROOT doesn't seem to work (maybe because of the split packages setup?).

jtojnar commented 4 years ago

Weirdly, it seems to work without the variables when the project does not check for specific modules:

jtojnar commented 4 years ago

For now, I have added the variables to pulseeffects.

emilazy commented 4 years ago

Thanks!

Maybe we should ask meson upstream what the preferred way to handle this would be?

jtojnar commented 4 years ago

Yeah, I am chatting with the PR author on IRC right now.

jtojnar commented 4 years ago

They do not want to extract the paths from compiler/linker so it was suggested to me to add a generic method for letting meson know where headers/libraries are stored in lieu of FHS. I will track that in https://github.com/NixOS/nixpkgs/issues/86159.

In the short term, we should add a setup hook to boost, that sets the two environment variables.

wucke13 commented 4 years ago

Setting the two environment variables still seems to be not enough:

Run-time dependency Boost (found: chrono, filesystem, program_options, system, thread | missing: python3) found: NO

despite having

    boost = boost.override { enablePython = true; };

Any idea how to fix this?

jtojnar commented 4 years ago

Boost currently defaults to Python 2, you would also need to override python argument (see calamares in all-packages.nix) or better, use python3.pkgs.boost.