[package] xxxxx/system: Cannot be installed in Windows because of package_info()

[memsharded]

Package and Environment Details (include every applicable attribute)

• Package Name/Version: xxxx/system
• Operating System+version: Windows
• Conan version: conan 1..39
• Python version: Python 3.7.4

Steps to reproduce (Include if Applicable)

(conan36) λ conan install opengl/system@ -s os=Linux
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=16
os=Linux
os_build=Windows
[options]
[build_requires]
[env]

Installing package: opengl/system
Requirements
    opengl/system from 'conancenter' - Cache
Packages
    opengl/system:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Download

Cross-build from 'Windows:x86_64' to 'Linux:x86_64'
Installing (downloading, building) binaries...
opengl/system: Retrieving package 5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 from remote 'conancenter'
Downloading conanmanifest.txt completed [0.06k]
Downloading conaninfo.txt completed [0.16k]
Downloading conan_package.tgz completed [0.06k]
Decompressing conan_package.tgz completed [0.00k]
opengl/system: Package installed 5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
opengl/system: Downloaded package revision 0
ERROR: xorg/system: Error in package_info() method, line 92
        self._fill_cppinfo_from_pkgconfig(name)
while calling '_fill_cppinfo_from_pkgconfig', line 24
        if not pkg_config.provides:
        ConanException: pkg-config command ['pkg-config', '--print-provides', 'x11', '--print-errors'] failed with error: Command 'pkg-config --print-provides x11 --print-errors' returned non-zero exit status 1.
'pkg-config' is not recognized as an internal or external command,
operable program or batch file.

This means that I cannot do a promotion from a machine running Windows (or any machine that doesn't have pkg-config) for a Linux dependency in ConanCenter (there are many recipes using some opengl/system or xorg/system) to my own Artifactory repo by doing a conan install xxxx -pr=myprofile followed by conan upload

These recipes should probably relax this requirement, it would be ok to return empty package_info() for those packages, not a big issue, as they will not be used.

[ericLemanissier]

AFAIU, the problem with the /system packages as they are now, is that they don't raise ConanInvalidConfiguration in case of cross-compilation, which they don't support. Do we agree that this use case pre-supposes that all the recipes in the dependency graph are able to be cross-compiled ? Wouldn't it better to call conan download in this case?

[SSE4]

they won't be able to cross-compile, that's right. the only workflow I see is that you provide your own package like opengl/something or x11/something where you specify OpenGL/X11 libraries installed for the given architecture/system (cross-compiler, perhaps, in sysroot). then you use an override feature to bypass the conan-center provided system packages (thus, never invoke, apt or yum). of course, it's just a theory, I didn't test that workflow, someone brave enough, who actually needs it should follow the path, and reports problems if any. worth starting from https://www.linuxfromscratch.org/blfs/view/svn/x/xorg7.html. for the promotion... I guess recipe(s) should just raise for cross-compiling to have a meaningful error message.

[SSE4]

also, some generic reasons not to use Windows promotions for Linux libraries - the following information might be silently lost, making builds invalid and corrupted:

• permissions (including executable permissions)
• symlinks (which are often used for shared objects)
• file names that are invalid in Windows for whatever reason (restricted characters, a reserved name like AUX, too long path, case sensitivity, etc)
• timestamps (base and resolution is usually different from Windows)

[SpaceIm]

AFAIU, the problem with the /system packages as they are now, is that they don't raise ConanInvalidConfiguration in case of cross-compilation, which they don't support.

They should support cross-build when they don't need to call a system package manager. For example opengl/system supports cross-build from Linux to Windows, but doesn't support cross-build to Linux and FreeBSD.

[SSE4]

I've dug into the cross-building use-case a little bit out of curiosity. I've tried Windows x64 -> Linux ARMv8 (RPI3) case. first off, I've downloaded a toolchain: https://gnutoolchains.com/raspberry/. installed it into C:\SysGCC (default location). then, I needed some conan profile (call it rpi):

[settings]
os=Linux
arch=armv8
compiler=gcc
compiler.version=8
compiler.libcxx=libstdc++11
build_type=Release
[options]
[build_requires]
[env]
CC=C:\SysGCC\raspberry\bin\arm-linux-gnueabihf-gcc.exe
CXX=C:\SysGCC\raspberry\bin\arm-linux-gnueabihf-g++.exe
CONAN_MAKE_PROGRAM=C:\SysGCC\raspberry\bin\make.exe
CONAN_CMAKE_GENERATOR=Unix Makefiles

with two profile approach I've used cmd like:

conan install . -pr:b default -pr:h rpi --build missing

nothing especial. first observation, our */system packages are useless for that use-case, and produce non-friendly error messages:

glu/system: Already installed!                                                                                                                      ERROR: glu/system: Error in package_info() method, line 64                                                                                                  self._fill_cppinfo_from_pkgconfig("glu")                                                                                                    while calling '_fill_cppinfo_from_pkgconfig', line 39                                                                                                       if not pkg_config.provides:                                                                                                                         ConanException: pkg-config command ['pkg-config', '--print-provides', 'glu', '--print-errors'] failed with error: Command 'pkg-config --print-provides glu --print-errors' returned non-zero exit status 1.                                                                                     'pkg-config' is not recognized as an internal or external command,                                                                                  operable program or batch file.             

(that could be fixed easily by rejecting cross-compiling with a nice ConanInvalidConfiguration message)

why useless? well, RPI toolchain already contains sysroot with many stuff cross-compiled (prebuilt). e.g. there are:

• X11 headers (C:\SysGCC\raspberry\arm-linux-gnueabihf\sysroot\usr\include\X11)
• X11 libraries (C:\SysGCC\raspberry\arm-linux-gnueabihf\sysroot\usr\lib\arm-linux-gnueabihf with libX11.so and others)
• OpenGL headers (C:\SysGCC\raspberry\arm-linux-gnueabihf\sysroot\usr\include\GL, plus GLES/GLES2/GLES3)
• OpenGL libraries (C:\SysGCC\raspberry\arm-linux-gnueabihf\sysroot\usr\lib\arm-linux-gnueabihf\libGL.so and others)

so, we don't need to install X11/GL packages with apt, they are already shipped. I think that's the case with many embedded systems, include toolchains for routers, TVs, etc. (surprisingly, there are lots of stuff, e.g. there are prebuilt Qt and boost!) (otoh, some important stuff is missing, there is no GLU/glu.h and X11/extensions/XInput.h, probably I should compile them myself, or they aren't just supported).

so I just created some dummy packages (conan new + manual editing) and used override feature in my profile:

[requires]
# override system packages
xorg/rpi@
opengl/rpi@
glu/rpi@

this works from the conan side:

WARN: freeglut/3.2.1: requirement opengl/system overridden by your conanfile to opengl/rpi
WARN: freeglut/3.2.1: requirement glu/system overridden by your conanfile to glu/rpi
WARN: freeglut/3.2.1: requirement xorg/system overridden by your conanfile to xorg/rpi

...

    glu/rpi from local cache - Cache
    opengl/rpi from local cache - Cache
    xorg/rpi from local cache - Cache

so system packages are bypassed and dummy packages are used instead. dummy packages do nothing right now, as includes and libraries are inside default search directories within sysroot.

obviously, on macOS and Windows it's the way to go for now. maybe there should be an easier way to generate dummy package (wrapping existing headers/libraries in sysroot).

I also believe it doesn't make such sense to use */system packages even on Linux -> Linux cross-compiling. why? well, on modern distributions like Ubuntu you may add additional architectures (e.g. ARM) and install that packages. but that will be packages for desktop ARM Ubuntu. even Raspberry Pi I think uses different distro's (like Pidora), routers will likely use OpenWRT and other hardware will use something like modified Gentoo or even own distro. these embedded systems might be even not glibc-based, but ulibc, newlib or musl based, which renders prebuilt packages from your desktop system effectively useless, even if they match architecture. so yeah, for real cross-building, you better wrap dummy override for system packages yourself.

TLDR: add some code to the system recipes:

if tools.cross_building(self):
    raise ConanInvalidConfiguration("cross-building is not supported, please use your own package with override!")

[jgsogo]

This is not about cross-building, the issue is much simpler than that.

package_info() is evaluated whenever Conan needs to compute the dependency graph. If the evaluation requires running pkg-config and the tool is not available in your system, the command will just fail. This will happen for commands like conan info ..., for example.

The only way to go, would be to use just Python in the package_info() method as it is the only tool we know is available at that time. We need to isolate usage of pkg-config in the build() or package() method, which are the only ones that run in the build machine, where we can install and request several tools to be available. We can generate an additional file (json, yaml) during build() with the information of the package and use that file to populate self.cpp_info in package_info() method. wdyt?

Once we can manage existing xxx/system recipes, we can think about cross-building scenarios, which is something I would really like to approach... and these previous comments are really valuable.

[ericLemanissier]

Do I understand correctly that you are suggesting to cache all the information returned by pkg-config during build time, in the package folder ? This means that the information will be then cached on the remote (CCI) and distributed to the consumers, whatever their distribution ? There is a very high risk of breakage, because the package is not a proxy to the system libraries any more.

[jgsogo]

Do I understand correctly that you are suggesting to cache all the information returned by pkg-config during build time, in the package folder ? This means that the information will be then cached on the remote (CCI) and distributed to the consumers, whatever their distribution ? There is a very high risk of breakage, because the package is not a proxy to the system libraries any more.

You are right, Eric. It should match the layout of the machine where the builds or execution happens.

[SSE4]

just in case, I want to clarify:

1. conan info opengl/system@ -s os=Linux works on Windows/macOS, there is no issue
2. conan install opengl/system@ -s os=Linux fails on Windows/macOS

[perseoGI]

Hi there, I was reviewing old issues and I have come across this one. This is a conan v1 issue which was fixed with conan v2 build-host model of cross-compile.

I'm closing this thread, if this problem remains in conan v2 do not hesitate to open a new issue! Happy coding 🐸