Open agurtovoy opened 5 years ago
I don't have a .pc from my boost install on Linux, and looking at their trac, they don't have pkg-config support. Is conan generating the .pc file for you?
@dcbaker Yep, together with .pc files for all my other dependencies. The boost.pc itself looks fine to me (this one is on Mac):
prefix=/Users/xxx/.conan/data/boost/1.70.0/conan/stable/package/2439fa14ae114198bd340332be9c9f3e28aa8c03
libdir=${prefix}/lib
includedir=${prefix}/include
Name: boost
Description: Boost provides free peer-reviewed portable C++ source libraries
Version: 1.70.0
Libs: -L${libdir} -lboost_wave -lboost_container -lboost_contract -lboost_exception -lboost_graph -lboost_iostreams -lboost_locale -lboost_log -lboost_program_options -lboost_random -lboost_regex -lboost_serialization -lboost_wserialization -lboost_coroutine -lboost_context -lboost_timer -lboost_thread -lboost_chrono -lboost_date_time -lboost_atomic -lboost_filesystem -lboost_system -lboost_type_erasure -lboost_log_setup -lboost_math_c99 -lboost_math_c99f -lboost_math_c99l -lboost_math_tr1 -lboost_math_tr1f -lboost_math_tr1l -lboost_stacktrace_addr2line -lboost_stacktrace_basic -lboost_stacktrace_noop -lboost_unit_test_framework -Wl,-rpath,"${libdir}"
Cflags: -I${includedir} -DBOOST_USE_STATIC_LIBS
Requires: zlib bzip2
...it's just that the code in _build_external_dependency_list
I quoted above finds and returns the built-in BoostDependency
class before checking if there is a .pc, and the BoostDependency
class doesn't pick up the .pc configuration either.
@dcbaker What if instead of unconditionally returning the built-in dependency detector (when present), _build_external_dependency_list
returned a list of candidate detectors that included the built-in dependency?
Have the same issue, showstopper, please fix this.
Does it also fail if you set the method explicitly:
boost_dep = dependency('boost', method: 'pkg-config')
@jpakkane Yep:
src/meson.build:15:0: ERROR: Unsupported detection method: pkg-config, allowed methods are auto and auto
That was actually the first thing I tried; I should have mentioned it in the original report.
I think this is fairly easy to fix: meson supports a priority order of different dependency methods. We can always check for boost using pkg-config first.
Except that Boost upstream does not provide .pc files. They are provided by other people. Thus there is no guarantee that they work or that they even have the same names on different platforms (as they can be provided by different groups).
@jpakkane If they exists, I'd assume that they work, otherwise how/why would they even be there?
Also, Conan is pretty big nowadays; its official Boost package has been downloaded 27,000 in the last 30 days. Granted, most people use it with CMake, but I've been using Conan+Meson for a couple of months now (on Mac and Windows), and I'm loving the ease of use, ease of comprehension, and hackability of both (it helps that both are written in Python).
I should note that, when using Boost with Conan, this issue only surfaces when you invoke Meson directly on the build dir; when you invoke Meson through conan build ..
, Conan also sets up BOOST_ROOT, thus masking this issue. Personally, I think the overall situation is super misleading, since all the other dependencies are picked up through their .pc
files, and w/o knowing that dependency('boost')
is special-cased in Meson to ignore the .pc
, you can bang you head against the wall for a while trying to figure out why this sometimes works and sometimes doesn't.
Re: the .pc
file name, Conan dependency is called "boost" and generates boost.pc
.
There are plenty of upstream projects that don't provide .pc files, but some distro patches it in for them. I would consider Conan to be another one of those for this case.
Same issue. Would love a fix.
How do the various projects that provide Boost with .pc files do their naming? If you use, say, Boost Graph, what is its pkg-config name? boost-graph
? Boost-Graph
? boostgraph
? Something else? Is it the same for all providers or is it named differently in different pc files?
As a workaround, if you know that on your platform the dependency is called boost-something
, you can get it to work by doing dependency('boost-something')
. The special casing code only kicks in if the dependency name is exactly boost
.
@jpakkane WRT Conan, if you use granular Boost packages, e.g. boost_multi_index/1.69.0@bincrafters/stable
, boost_range/1.69.0@bincrafters/stable
, etc., the .pc files will be named after the package: boost_multi_index.pc
, boost_range.pc
, etc.
Personally, I wouldn't expect dependency('boost')
to pick these up; I'd just mirror my Conan dependencies in meson.build
the same way I do it for everything else: dependency('boost_multi_index')
, dependency('boost_range')
, etc.
The problem with (Conan) granular packages is that Boost's dependency graph isn't really that clean. You think you'd gain if you just pulled in the packages that you use, but in practice it ends up being a pessimization. AFAIK the majority of people use the "monolith" boost
package.
Personally, I wouldn't expect dependency('boost') to pick these up; I'd just mirror my Conan dependencies in meson.build the same way I do it for everything else: dependency('boost_multi_index'), dependency('boost_range'), etc.
Actually, scratch that; given the Boost dependency graph, doing the above manually is impractical. For example, pulling in just boost_multi_index
, boost_range
and boost_utility
will also bring in all of the following:
boost_algorithm.pc boost_function.pc boost_proto.pc
boost_array.pc boost_function_types.pc boost_range.pc
boost_assert.pc boost_fusion.pc boost_ratio.pc
boost_atomic.pc boost_integer.pc boost_rational.pc
boost_bind.pc boost_intrusive.pc boost_regex.pc
boost_chrono.pc boost_io.pc boost_serialization.pc
boost_concept_check.pc boost_iterator.pc boost_smart_ptr.pc
boost_config.pc boost_lambda.pc boost_static_assert.pc
boost_container.pc boost_lexical_cast.pc boost_system.pc
boost_container_hash.pc boost_locale.pc boost_throw_exception.pc
boost_conversion.pc boost_math.pc boost_tokenizer.pc
boost_core.pc boost_move.pc boost_tti.pc
boost_cycle_group_a.pc boost_mpl.pc boost_tuple.pc
boost_cycle_group_b.pc boost_multi_index.pc boost_type_index.pc
boost_cycle_group_c.pc boost_numeric_conversion.pc boost_type_traits.pc
boost_detail.pc boost_optional.pc boost_typeof.pc
boost_endian.pc boost_phoenix.pc boost_unordered.pc
boost_exception.pc boost_pool.pc boost_utility.pc
boost_filesystem.pc boost_predef.pc boost_variant.pc
boost_foreach.pc boost_preprocessor.pc boost_winapi.pc
Which is why people stick to the monolith package. IMHO it still doesn't mean that Meson's dependency('boost')
needs to pick up these. The issue here is not so much about Boost dependencies as it's about transitive dependencies. For example, the above also pulls in bzip2.pc
. Being able to automagically pass all the transitive dependencies from Conan to Meson would solve this and a bunch of other use cases, which is why I'm eyeing https://github.com/mesonbuild/meson/pull/5209.
My current workaround:
export PKG_CONFIG_PATH=$(realpath ../build/)
export BOOST_ROOT=$(pkg-config --variable=prefix boost)
meson ../build
Being able to automagically pass all the transitive dependencies from Conan to Meson would solve this
This is what Pkg-config already does if you set it up correctly. If your pc files contain the correct transitive dependencies, everything will just work.
Or, in other words, if in this case dependency('boost_something')
returns a dependency that does not work but requires manually adding dependency('bzip2')
, then the .pc configuration is set up incorrectly.
Or, in other words, if in this case dependency('boost_something') returns a dependency that does not work but requires manually adding dependency('bzip2'), then the .pc configuration is set up incorrectly.
Thanks for the clarification. Conan does generate .pc files for Boost correctly (see below); I'll double-check my specific use case that was giving me trouble.
boost_array.pc
:
prefix=/Users/xxx/.conan/data/boost_array/1.69.0/bincrafters/stable/package/5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
libdir=${prefix}/array/lib
includedir=${prefix}/array/include
Name: boost_array
Description: Shared python code used in other Conan recipes for the Boost libraries
Version: 1.69.0
Libs: -L${libdir} -Wl,-rpath,"${libdir}"
Cflags: -I${includedir} -DBOOST_ALL_NO_LIB=1
Requires: boost_assert boost_config boost_core boost_static_assert boost_throw_exception
Is there an update for this?
I think that the best initial solution would be to just respect explicitly given method
arguments, and error when unsuccessfull:
boost_dep = dependency('boost', method: 'pkg-config')
boost_dep = dependency('boost', method: 'cmake')
@Randshot I agree that these two should definitely work, but IMHO it's not sufficient. dependency('anyotherlib')
works as expected if anyotherlib.pc
is present in the build dir for, well, any library other than Boost. I don't see a good reason why I'd have to spell this particular one as dependency('boost', method: 'pkg-config')
to make it work. It's inconsistent and IMHO just weird:
deps = [
dependency( 'fmt' ),
dependency( 'gsl_microsoft' ),
dependency( 'range-v3' ),
dependency( 'boost', method: 'pkg-config' ) # ???
]
That is true, but as there isn't a consensus on how to handle the module names yet, I thought that there should at least be an initial fix for this. This is really bugging me TBH.
@Randshot Could you clarify where is your .pc for Boost is coming from and what does it look like?
@agurtovoy I, like you, am using Conan for generating the pkg-config files. It's really the best option on Windows for getting the Boost libraries.
The generated file looks like this (in the case of the monolithic package):
prefix=C:/.conan/13e461/1
libdir=${prefix}/lib
includedir=${prefix}/include
Name: boost
Description: Boost provides free peer-reviewed portable C++ source libraries
Version: 1.69.0
Libs: -L${libdir} -llibboost_wave -llibboost_container -llibboost_contract -llibboost_exception -llibboost_graph -llibboost_iostreams -llibboost_locale -llibboost_log -llibboost_program_options -llibboost_random -llibboost_regex -llibboost_serialization -llibboost_wserialization -llibboost_coroutine -llibboost_fiber -llibboost_context -llibboost_timer -llibboost_thread -llibboost_chrono -llibboost_date_time -llibboost_atomic -llibboost_filesystem -llibboost_system -llibboost_type_erasure -llibboost_log_setup -llibboost_math_c99 -llibboost_math_c99f -llibboost_math_c99l -llibboost_math_tr1 -llibboost_math_tr1f -llibboost_math_tr1l -llibboost_stacktrace_noop -llibboost_stacktrace_windbg -llibboost_stacktrace_windbg_cached -llibboost_unit_test_framework -lbcrypt
Cflags: -I${includedir} -DBOOST_USE_STATIC_LIBS -DBOOST_ALL_NO_LIB
Requires: zlib bzip2
And manually setting the BOOST_ROOT
env var also doesn't work for my setup, since it looks for the ABI specific library names:
LINK : fatal error LNK1104: cannot open file 'libboost_date_time-vc141-mt-gd-x64-1_69.lib'
The separate module packages available from Bincrafters might be a solution, but the current lzma package requires an older Windows SDK than I can install for VS2019, and I don't want to install VS2017 again, so yeah.
Yeah, I wouldn't recommend those anyway. They take much longer to install and the Boost dependency graph is too intermingled to make it worth the extra slowness and complexity.
I should mention that if you use Conan to build your project, it should still work as per https://github.com/mesonbuild/meson/issues/5438#issuecomment-516050754.
It does work, if I use Conan to invoke meson, but that somewhat defeats the purpose of using meson in my opinion.
On the note of setting BOOST_ROOT
, like I wrote above, that doesn't seem to work for my setup, as meson is looking for ABI specific library names:
LINK : fatal error LNK1104: cannot open file 'libboost_date_time-vc141-mt-gd-x64-1_69.lib'
@agurtovoy Ok, I think I have a way of using Boost on Windows with Meson and Conan. It is not optimal, but it kinda works.
I am using both the pkg_config
and virtualenv
generators in the conan file.
The generated activate.bat
/activate.ps1
will set the matching BOOST_ROOT
env var.
I also had to make sure that I add the following build argument -DBOOST_AUTO_LINK_SYSTEM
.
This will prevent the linking error in my post above.
Like I said, it is not an optimal solution, but it works for now.
@Randshot Thanks for sharing! I plan on looking into a fix for this, but it's probably going to be another couple of weeks before I get to it. I'll be delighted if someone else beats me to it ;)
I've also been running into this. In case it helps I created a sample project for reproducing the problem here.
This is also inconsistent with other custom meson dependencies: gtest
dependency allows to specify a search method (i.e. works with pkg-config
).
In 0.54.0, the internal BoostDependency
was completely rewritten, so that it automatically picks the correct libraries (with ABI tags, etc.) in a given BOOST_ROOT
, so the linking problem from @Randshot should be resolved. If not please open an issue.
Adding *.pc
support is definitely possible, but I would rather try fixing the internal dependency since supporting all the boost_*
looks like a huge mess and the boost.pc
adds everything in boost. In short the Conan *.pc
files are far from ideal IMHO.
I have only skimmed through this issue, but from what I can tell it looks like this is more or less a Conan specific problem correct?
Does Conan have a predictable directory layout or are there some environment variables that would help meson automatically compute the BOOST_ROOT
? Alternatively, where is the boost.pc
file locate relative to the BOOST_ROOT
? If this is a known location meson could use PKG_CONFIG_PATH
(with the existence of the boost.pc
) to calculate BOOST_ROOT
.
I have never used Conan, so sorry if these are trivial questions.
In short the Conan *.pc files are far from ideal IMHO.
Yep. Currently conan
does not have a way to specify intra-package dependencies (they are working on that though). So if a user wants to link only a single library from multi-library package, they'll have to extract that lib manually by using find_library(get_variable(pkgconfig: 'libdir') + '/' + libname)
in meson.
Does Conan have a predictable directory layout or are there some environment variables that would help meson automatically compute the BOOST_ROOT?
Conan uses a standard directory layout:
.pc
files generated by conan contain the following variables (if the package has corresponding artifacts):
prefix
: points to dep_rootlibdir
: points to dep_root/libincludedir
: points to dep_root/includeAlternatively, where is the boost.pc file locate relative to the BOOST_ROOT? If this is a known location meson could use PKG_CONFIG_PATH (with the existence of the boost.pc) to calculate BOOST_ROOT.
.pc
files are generated to a conan build directory (not documented and should not be relied upon). They are passed to meson via PKG_CONFIG_PATH
.
Dependencies itself are stored in internal conan cache directory, and can be only accessed through .pc
files in meson (since those paths are dynamic).
Note: I've not tried using boost
conan package in meson on Windows, so I don't know if the names of libraries provided by conan package are consistent with the expected names in meson. But it just works(c) on *nix after specifying BOOST_ROOT
.
So, running pkg-config --variable=prefix boost
would be the most reliable way for getting BOOST_ROOT
?
So, running pkg-config --variable=prefix boost would be the most reliable way for getting BOOST_ROOT?
Yes, I thinks so (at least on *nix).
@mensinda , but IMO, it would still require the addition of method:pkg-config
, since otherwise there is no way to ensure that the proper dependency is fetched (e.g. a pkgconfig dependency and not a system one)
Why though? Just putting pkg-config --variable=prefix boost
root as the first search path should be enough. It will only fall back to a system dependency if there is an issue with the prefix (ABI mismatch, etc.).
@mensinda , this is needed to minimize potential errors and unexpected behaviour. A simple example:
boost.pc
. boost.pc
in search paths and uses system boost instead. Should be fixed (as in the meson system boost dependency should find a Conan boost installation) with #7185.
@TheQwertiest This problem arguably already exists anyway. A better solution would be to set the minimum boost version in the dependency()
call, so that the correct boost dependency is found.
@agurtovoy, @Randshot could you please check if the linked PR works for you? I have no experience with Conan (and not much with Windows), so checking this myself is a bit tricky.
A better solution would be to set the minimum boost version in the dependency() call, so that the correct boost dependency is found.
boost versioning is orthogonal with the problem I've described. They might have the same version (or might not). A user might not want to use just any boost (even if it meets the version requirements), he might need a specific boost, and this can be easily achieved by using a fixed method for searching dependencies. E.g. you can't get wrong boost using only wrap
or only pkg-config
, if they are configured properly.
To be clear, I am not against adding a pure pkg-config
method if it fixes a real problem. However, I don't think that just adding *.pc
support will fix the issue you described. A simple example: What happens if you have installed multiple boost versions that provide *.pc
files?
There is already a mechanism to specify a specific boost version without fallbacks and that is setting BOOST_ROOT
. Meson won't find a system boost, if it can't find what it is looking for inside the prefix. Assuming that there only ever exists one boost version with *.pc
files doesn't seem like a foolproof solution to me either (especially since BOOST_ROOT
already does what you want).
Also, you should be able to uninstall (rm -rf
should also work in extreme cases) any system boost versions in your CI. If you want to be sure to use the correct version and don't to or can't set BOOST_ROOT
this would be the best solution.
Regarding why env BOOST_ROOT
is bad (or at least not really good) I think we've already discussed in in the other issue =)
What happens if you have installed multiple boost versions that provide *.pc files?
AFAIK, meson searches for .pc
only in directories specified in PKG_CONFIG_PATH. So if one provides a curated list of these paths, then there won't be a problem.
But (again AFAIK) there is no way to forbid meson from searching system paths.
Also, you should be able to uninstall (rm -rf should also work in extreme cases) any system boost versions in your CI.
It's not always possible: some system packages (system as in installed via apt
and friends) might depend on it.
TBH, I don't really see why it's not possible (or undesirable) to add a method:pkgconfig
- boost meson dependency already uses it internally, this kwarg would just restrict the available search paths in the internal logic, without changing anything else.
TBH, I don't really see why it's not possible (or undesirable) to add a method:pkgconfig - boost meson dependency already uses it internally, this kwarg would just restrict the available search paths in the internal logic, without changing anything else.
The internal dependency will use it once #7185 is merged. However, method: 'pkg-config'
will just use the boost.pc
file without the internal logic of BoostDependency
(so basically just return a PkgConfigDependency('boost') instead of a BoostDependency
). This is a significant difference with all the disadvantages associated with the monolithic boost.pc
(most importantly no modules support).
Also, for a CI environment, wouldn't export BOOST_ROOT=$(pkg-config --variable=prefix boost)
be acceptable in your .travis.yml
or GitHub workflow? Not ideal but if you install boost via Conan and want to be really sure that only that version is used then this would be IMHO the best option. This way you can also sanity check if BOOST_ROOT
is correct before calling meson.
However, method: 'pkg-config' will just use the boost.pc file without the internal logic of BoostDependency (so basically just return a PkgConfigDependency('boost') instead of a BoostDependency)
Thanks, I didn't know that, I thought method
only influenced search methods.
Also, for a CI environment, wouldn't export BOOST_ROOT=$(pkg-config --variable=prefix boost) be acceptable in your .travis.yml or GitHub workflow?
It doesn't work exactly like that (because .pc
file locations are not really known outside of conan, it's known only to conan recipe), but it will still require to special-case boost dependency. So, yeah, not ideal =)
Currently I have to use the following code in conan recipes to handle this problem (cleaned up for brevity):
if has_dep('boost'):
# Gets boost root path and sets it to BOOST_ROOT
cm = with_boost_dep_env(get_dep_info('boost'))
else:
cm = nullcontext()
with cm:
meson.configure(...)
I want to emphasize that this is the only dependency that needs such special handling.
Well, you only need to set BOOST_ROOT
because you want to ensure that your CI doesn't accidentally break. If you don't need that requirement then it just works (TM) with #7185.
So if this additional insurance (required for Conan only as far as I can tell) is the only thing that is missing, I would say that it isn't worth adding support for the plain boost.pc
file. Especially since it is possible to support this (arguable niche) problem in Conan, even if it is a bit clunky.
Meson 0.58.1, still stumbling around this problem. dependency('boost')
works fine, but dependency('boost', modules: ...)
cannot find any libraries.
My setup is slightly weird, since I install Meson itself via Conan. I run activate.ps1
, which setups various environment variables. Among others, it patches PATH
to add path to meson, and creates BOOST_ROOT
with the valid path to Boost. From here things get weird.
If now I simply try to build my project, at some point BoostDependency.__init__
is called. It reacts to BOOST_ROOT
envvar and eventually Meson actually finds all the needed directories and libraries. But then BoostDependency.filter_libraries
happens, where it happily filters away any found boost libraries - since in Conan they are named simply boost_filesystem.lib
without version and runtime postfixes.
I unset BOOST_ROOT
, get myself a pkg-config
executable and put it directly into the project folder, from where I run meson compile -C build
. But Meson says Did not find pkg-config by name 'pkg-config'
- apparently, because it does chdir()
somewhere and the current directory changes from S:/myproj
to S:/myproj/build
, while pkg-config is located as S:/myproj/pkg-config.exe
. Unexpected... but kinda reasonable from Python perspective.
So I put pkg-config
somewhere in PATH
, and also setup PKG_CONFIG_PATH
(as was suggested in the thread above) to the directory where Conan creates .pc
. Yet Meson says in logs that env[PKG_CONFIG_PATH]:
- meaning that the envvar that I myself set is somehow empty. Apparently, at some point PkgConfigDependency.setup_env
gets called and does
new_pkg_config_path = ':'.join([p for p in extra_paths])
env['PKG_CONFIG_PATH'] = new_pkg_config_path
If I comment this out, then Meson successfully calls pkg-config to get all the required directories... and then still performs the whole lookup, and calls filter_libraries
instead of simply using what was provided in .pc
. So basically, those are used only to get the include/library directories, but the actual linker and compiler flags are ignored :(
PATH
does matter, yes, and finding it "directly in the project folder" won't work on OSes where programs in the current folder are never detected to begin with (unless they are ALSO on PATH
).
PKG_CONFIG_PATH
should be picked up from the environment when running meson setup, but gets serialized to -Dpkg_config_path and won't be detected from the environment on a reconfigure, IIRC. Also if you use a cross file it is never read...
...
I don't know about your boost issue specifically.
since in Conan they are named simply
boost_filesystem.lib
without version and runtime postfixes.
Is this a general conan problem with the boost dependency, irrespective of pkg-config?
I'm using meson together with conan/pkg-config (on Windows). One of the project dependencies is Boost, and conan dutifully generates the corresponding
boost.pc
file, but meson ends up trying to use the built inBoostDependency
class before even checking whether theboost.pc
file exists (https://github.com/mesonbuild/meson/blob/master/mesonbuild/dependencies/base.py#L2554):Other pkg-config dependencies are picked up as expected. Here's my
meson.build
:Thoughts?