Open jwillikers opened 6 months ago
Have managed to reproduce some issues related to this - to see how we can best accommodate:
.pc
files to stand-in for system dependencies.pc
files generated by ConanThere are a few challenges, namely:
libfoo.so
is in the sysroot and is not meant to be provided by conan, the only flag should be -lfoo
, without any-I
and no -L
- I'm not currently observing this. This can be a problem because if the sysroot contains things that are also provided by Conan, we would expect conan to take precedence, but pkg-config may end up altering this order.pc
files in the build machine for executables - we can have different behaviour for the host and build profiles I have this working more or less but meson seems to be doing some additional path prefixing that makes this not work at all - investigating
other comments:
Apparently, PKG_CONFIG_LIBDIR is for setting the primary search paths while PKG_CONFIG_PATH is for setting the secondary search paths. In the case of libselinux, it appears that setting PKG_CONFIG_LIBDIR fixes issues with system directories being picked up for the build when cross-compiling.
However, the PKG_CONFIG_LIBDIR variable has a huge advantage over PKG_CONFIG_PATH in that it has precedence.
My understanding is that PKG_CONFIG_PATH
is searched first, and then it falls back to PKG_CONFIG_LIBDIR
, which, if undefined, uses the values pkg-config
was compiled with.
See https://bugs.freedesktop.org/show_bug.cgi?id=88992
This resulted in pcre2 being picked up from my sysroot and compiling without error, although it still should have been using the pcre2 package from Conan. This is indeed the biggest challenge at the moment
pkg-config locates the system's pcre2 pkg-config file and adds -I/usr/include to the compiler flags, breaking cross-compilation.
depending on how it is configured pkg-config
may locate .pc files in the host - but I'm susprised it would emit -I/usr/include
- pkg-config erases default system paths for good reasons - these are consisdered "system directories", in the sense that the compiler and linker will already look there and additional flags are not necessary: https://gitlab.freedesktop.org/pkg-config/pkg-config/-/blob/master/pkg.c?ref_type=heads#L817-838
Probably worth noting that pkgconf
and pkg-config
aren't 1:1 in how they handle everything. Also, I think issue #16468 is related, particularly with sysroot
handling in Meson. I'm linking to it for additional context.
What is your suggestion?
Working on cross-compilation problems with
libselinux
in conan-io/conan-center-index#22952, I discovered that there's another pkg-config environment variable for configured the search pathsPKG_CONFIG_LIBDIR
. Apparently,PKG_CONFIG_LIBDIR
is for setting the primary search paths whilePKG_CONFIG_PATH
is for setting the secondary search paths. In the case oflibselinux
, it appears that settingPKG_CONFIG_LIBDIR
fixes issues with system directories being picked up for the build when cross-compiling.libselinux
uses an ad-hoc Makefiles-based build system. It locates the necessary compiler flags for thepcre2
dependency by runningpkg-config --cflags libpcre2-8
. When cross-compiling and using the defaultPKG_CONFIG_PATH
variable to point to Conan's generators directory,pkg-config
locates the system'spcre2
pkg-config file and adds-I/usr/include
to the compiler flags, breaking cross-compilation. The next thing I tried was to setPKG_CONFIG_SYSROOT_DIR
to the path to my sysroot directory. This resulted inpcre2
being picked up from my sysroot and compiling without error, although it still should have been using thepcre2
package from Conan. Finally, I setPKG_CONFIG_LIBDIR
appropriately and viola, pkg-config found Conan'spcre2
package as expected.It occurred to me that Conan's handling of these environment variables could be greatly improved, especially since it only works with
PKG_CONFIG_PATH
. Here are the improvements I think that could be made. Note that handling of these variables might be better handled in the build system. Meson already makes several accommodations for these variables, such as:I think that it would make sense for Conan to follow suite for the pkg-config sysroot directory by doing the following:
sys_root
according to thesysroot
conf variable in the Meson cross file when it is defined.PKG_CONFIG_SYSROOT_DIR
automatically to the value of thesysroot
conf option.The handling of the
PKG_CONFIG_LIBDIR
variable is more complicated in that user's might want to incorporate system pkg-config files, especially when usingplatform_requires
. However, thePKG_CONFIG_LIBDIR
variable has a huge advantage overPKG_CONFIG_PATH
in that it has precedence. I think it makes more sense to use thePKG_CONFIG_LIBDIR
environment variable for AutotoolsToolchain andpkg_config_libdir
option for Meson to point to Conan's generated pkg-config files. This is important to ensure that the provided Conan packages are always preferred over external pkg-config files.This still leaves some gaps though, such as the fact that
PKG_CONFIG_LIBDIR
overwrites the the system pkg-config directories. This will break consumers relying on external pkg-config files. However, I think this is advantageous since it would require explicitly opting-in to using pkg-config files not provided by Conan which should improve reproducibility when creating Conan packages. To accommodate consumers and recipes wishing to use system packages, it would make sense to allow overriding the default behavior. For consumers, this would most likely be done via a conf value that allows adding paths toPKG_CONFIG_PATH
. Again, it's important that Conan packages are always preferred and usingplatform_requires
will ensure that Conan doesn't generate any pkg-config files for a given recipe, thus allowing the external pkg-config files to be picked up. System pkg-config search paths could also be added toPKG_CONFIG_PATH
so that consumers are not required to add this path explicitly.I'm pretty sure there are recipes that use system packages implicitly. If system include directories were to not be added to
PKG_CONFIG_PATH
, then recipes would probably want a convenience method to opt in to having the system pkg-config search paths added toPKG_CONFIG_PATH
.The
PKG_CONFIG_SYSROOT_DIR
is prepended to every path defined inPKG_CONFIG_PATH
, so handlingPKG_CONFIG_SYSROOT_DIR
with Conan will automatically correct system include directories, like/usr/include
, inPKG_CONFIG_PATH
for pkg-config.I think that better handling of these variables can significantly improve reproducibility while reducing the barrier of entry for those cross-compiling packages.
Have you read the CONTRIBUTING guide?