conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.3k stars 986 forks source link

[question] cross building packages that uses PkgConfig.fill_cpp_info #13962

Open LadyzRoman opened 1 year ago

LadyzRoman commented 1 year ago

What is your question?

I've been trying to cross compile some packages from Windows to raspberry pi with conan 2.0.4 host profile:

[settings]
os=Linux
arch=armv7
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=8
[conf]
tools.build:compiler_executables={"c": "arm-linux-gnueabihf-gcc-8", "cpp": "arm-linux-gnueabihf-g++"}
tools.build:sysroot=C:\\SysGCC\\raspberry\\arm-linux-gnueabihf\\sysroot
tools.gnu:make_program=C:/SysGCC/raspberry/bin/make
[buildenv]
PKG_CONFIG_PATH=C:\SysGCC\raspberry\arm-linux-gnueabihf\sysroot\lib\arm-linux-gnueabihf\pkgconfig;C:\SysGCC\raspberry\arm-linux-gnueabihf\sysroot\usr\lib\arm-linux-gnueabihf\pkgconfig;C:\SysGCC\raspberry\arm-linux-gnueabihf\sysroot\usr\lib\pkgconfig;C:\SysGCC\raspberry\arm-linux-gnueabihf\sysroot\usr\share\pkgconfig

Build profile

[settings]
os=Windows
arch=x86_64
build_type=Release
compiler=msvc
compiler.runtime=dynamic
compiler.cppstd=20
compiler.version=193
[conf]
tools.cmake.cmaketoolchain:generator=Ninja

And I having an issue with any system package, for example opengl/system, that uses PkgConfig.fill_cpp_info in package_info method.

ERROR: opengl/system: Error in package_info() method, line 61
        pkg_config.fill_cpp_info(self.cpp_info, is_system=self.settings.os != "FreeBSD")
        ConanException: Command 'pkg-config --print-provides gl --print-errors' failed with errorcode '1'
b"Package gl was not found in the pkg-config search path.\r\nPerhaps you should add the directory containing `gl.pc'\r\nto the PKG_CONFIG_PATH environment variable\r\nNo package 'gl' found\r\n"

I can set PKG_CONFIG_PATH before calling conan install and everything works quite well then, but it seems not right for me. Is there a way to set this environment variable by conan?

Have you read the CONTRIBUTING guide?

memsharded commented 1 year ago

Hi @LadyzRoman

Thanks for your question. Not sure if it could be the issue, but PKG_CONFIG_PATH in the profile as you defined will overwrite other possible values. Also it is not defined as a path, maybe you want to append or prepend it?

[buildenv]
PKG_CONFIG_PATH=+(path)C:\SysGCC....
PKG_CONFIG_PATH=+(path)C:\Other....

You can add one per line if it helps (and it might avoid having to think about the path separator ";" or ":"

Check https://docs.conan.io/2/reference/config_files/profiles.html#buildenv

LadyzRoman commented 1 year ago

Hi @memsharded Yes, I want PKG_CONFIG_PATH in my profile to override system values, because I need only host context values during cross-compilation. I have done some investigations and it seems for me that PkgConfig helper doesn't use buildenv at all here: https://github.com/conan-io/conan/blob/release/2.0/conan/tools/gnu/pkgconfig.py#L23-L30 I don't know whether it is intended or not, but I think it should use appropriate environment.

memsharded commented 1 year ago

I have been investigating this, and yes, the buildenv is not being used at all:

 def _parse_output(self, option):
        executable = self._conanfile.conf.get("tools.gnu:pkg_config", default="pkg-config")
        command = cmd_args_to_string([executable, '--' + option, self._library, '--print-errors'])
        env = Environment()
        if self._pkg_config_path:
            env.prepend_path("PKG_CONFIG_PATH", self._pkg_config_path)
        with env.vars(self._conanfile).apply():
            return check_output_runner(command).strip()

There are good reasons for this, the PkgConfig.fill_cpp_info happens in the package_info(). This is not a "build" step. The package_info() happens for every package installed, even if it is not built from source and downloaded from the server. Also for "system" recipes that do not have "build" step at all, the current case. So the [buildenv] cannot and it is not applied at all, and I'd say it makes sense that is not applied.

So at the moment it seems a current limitation of the PkgConfig helper:

We would need to investigate some other approach if we want to have this supported built-in.

LadyzRoman commented 1 year ago

OK, thank you for explanation, for now I will stick to the current approach.

memsharded commented 1 year ago

FIrst steps have been done:

Both will be available in 2.0.8

That means that it will be possible to run PkgConfig in the package() method, serialize the result, and load it in the package_info() method, so it will use all the information from Conan profiles and dependencies toolchains.

However, this is still a quite experimental flow, and it might require some time until it is allowed in ConanCenter. I'd encourage to test it following the tests in https://github.com/conan-io/conan/pull/13985, and give feedback. Many thanks!

LadyzRoman commented 1 year ago

Hi I've tested this approach and it work well for my case. But there is questionable moment. Now "system" packages have to always be built from sources as we can rely only on local serialized data. Maybe a new value for attribute build_policy="always" will do the job.

memsharded commented 1 year ago

But there is questionable moment. Now "system" packages have to always be built from sources as we can rely only on local serialized data.

I'd say this is actually a very expected behavior. If it is a system package, it is not packaging pre-compiled binaries, and the only way to be fully functional is if it "builds" in the machine it is going to be used. build_policy="missing" with upload_policy="skip" should achieve this behavior