conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.29k stars 982 forks source link

[question] Download the ARM development package in x86_64 fails. #17285

Open sunhao-DBH opened 2 weeks ago

sunhao-DBH commented 2 weeks ago

What is your question?

Problem Description I'm cross-compiling ffmpeg for armv7 on x86_64 Ubuntu. However, Conan is going to download the development package libva-dev from arm. May I ask where the settings are incorrect? Hope for guidance.

Environment details Conan version 2.9.1 This is my conanfile.txt:

[requires]
ffmpeg/4.2.1

[generators]
SConsDeps

Steps to reproduce

$ conan install . --output-folder=build --build=missing -pr armv7

Logs

======== Input profiles ========
Profile host:
[settings]
arch=armv7
build_type=Release
compiler=gcc
compiler.cppstd=98
compiler.libcxx=libstdc++
compiler.version=4.9
os=Linux
[buildenv]
CC=arm-linux-gnueabihf-gcc
CXX=arm-linux-gnueabihf-g++
LD=arm-linux-gnueabihf-ld

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=gnu14
compiler.libcxx=libstdc++11
compiler.version=7
os=Linux

======== Computing dependency graph ========
Graph root
    conanfile.txt: /mnt/hgfs/Desktop/Conan/AwtkApplication-ffmpeg/conanfile.txt
Requirements
    autoconf/2.71#f9307992909d7fb3df459340f1932809 - Cache
    automake/1.16.5#058bda3e21c36c9aa8425daf3c1faf50 - Cache
    brotli/1.1.0#d56d7bb9ca722942aba17369cb5c0519 - Cache
    bzip2/1.0.8#d00dac990f08d991998d624be81a9526 - Cache
    ffmpeg/4.2.1#8bd30bd1ea06b582da850bcd44ac5eaf - Cache
    flac/1.4.2#d4051f8b37ee93725dea40dfd6a10168 - Cache
    freetype/2.13.2#5d2563803c8558d4ef47271a82c73d20 - Cache
    libalsa/1.2.10#e64d5e1ced869a2f676145bab4f4a181 - Cache
    libaom-av1/3.6.1#a2b22c70d6fce43887881431808ab8a6 - Cache
    libcap/2.69#7ef2d60864c2e58e89db957be936dc49 - Cache
    libfdk_aac/2.0.3#0115f6598be7303e042684e3a846b12d - Cache
    libiconv/1.17#73fefc1b696e069df90fd1d18aa63edd - Cache
    libmp3lame/3.100#44b12d19316eb2b223d98d3e75dae438 - Cache
    libpng/1.6.44#9e1aa08fb46946c7c91e4ae03bd49811 - Cache
    libsndfile/1.2.2#b47faa587d50886068a4d2f385383ab9 - Cache
    libtool/2.4.7#08316dad5c72c541ed21e039e4cf217b - Cache
    libvpx/1.13.1#de05642042bff632e366b0a13c72285a - Cache
    libwebp/1.3.2#52f69c4a31c5cf033fdd9230d77a8e38 - Cache
    libx264/cci.20220602#a9d536e10203f4c5a27a437f103661b4 - Cache
    libx265/3.4#719e50b2b2c3fd1b9133fea12da42c62 - Cache
    m4/1.4.19#b38ced39a01e31fef5435bc634461fd2 - Cache
    mpg123/1.31.2#ce831c936b2284e2066ab3dc58a2628e - Cache
    ogg/1.3.5#062626875f5c8c59f069f76f148098ef - Cache
    openh264/2.3.1#8990004bf4953883c68d24118790424a - Cache
    openjpeg/2.5.0#2886244f2492121aa7c6c493a814db06 - Cache
    openssl/3.3.2#9f9f130d58e7c13e76bb8a559f0a6a8b - Cache
    opus/1.4#3c98a306d127dce1e74d58a0e2c850b5 - Cache
    pulseaudio/14.2#30afeff38a7770cb9cda17a66da49a19 - Cache
    vaapi/system#9fa40059fb979af4f9b301b84fb2dfa1 - Cache
    vdpau/system#bdd7d010c4e3d8762a8f6f08a3cf6760 - Cache
    vorbis/1.3.7#37e58f52e59a6232199b34ef402714a6 - Cache
    xorg/system#98f82cb669e4ebc6b4d9d8a4f3f1faf4 - Cache
    xz_utils/5.4.5#b885d1d79c9d30cff3803f7f551dbe66 - Cache
    zlib/1.3.1#f52e03ae3d251dec704634230cd806a2 - Cache
Build requirements
    autoconf/2.71#f9307992909d7fb3df459340f1932809 - Cache
    automake/1.16.5#058bda3e21c36c9aa8425daf3c1faf50 - Cache
    gettext/0.21#a2bd3513f212013764c9040f4c70ed69 - Cache
    gnu-config/cci.20210814#dc430d754f465e8c74463019672fb97b - Cache
    libiconv/1.17#73fefc1b696e069df90fd1d18aa63edd - Cache
    libtool/2.4.7#08316dad5c72c541ed21e039e4cf217b - Cache
    m4/1.4.19#b38ced39a01e31fef5435bc634461fd2 - Cache
    meson/1.2.2#21b73818ba96d9eea465b310b5bbc993 - Cache
    meson/1.3.2#26ce8a76a36cc275cdfee1d757bc6561 - Cache
    meson/1.4.1#43b753adf9a089cadfa17c8034f5c897 - Cache
    ninja/1.12.1#fd583651bf0c6a901943495d49878803 - Cache
    pkgconf/2.0.3#f996677e96e61e6552d85e83756c328b - Cache
    pkgconf/2.1.0#27f44583701117b571307cf5b5fe5605 - Cache
    pkgconf/2.2.0#6462942a22803086372db44689ba825f - Cache
    yasm/1.3.0#fb800a15413dca19bfaef9e4b5d50694 - Cache
Resolved version ranges
    libpng/[>=1.6 <2]: libpng/1.6.44
    ninja/[>=1.10.2 <2]: ninja/1.12.1
    openssl/[>=1.1 <4]: openssl/3.3.2
    zlib/[>=1.2.11 <2]: zlib/1.3.1

======== Computing necessary packages ========
libfdk_aac/2.0.3: Main binary package '1202021ff6721b5b3241bb0c1446ce018c465f21' missing
libfdk_aac/2.0.3: Checking 5 compatible configurations
libfdk_aac/2.0.3: Compatible configurations not found in cache, checking servers
libfdk_aac/2.0.3: 'a90e3d43014c251aec776929c3dca250307248e4': compiler.cppstd=gnu98
libfdk_aac/2.0.3: 'e8eeaf49a704154e127f9bc268514957dedda2e0': compiler.cppstd=11
libfdk_aac/2.0.3: '3057da643bfcc3b42fb166bd0b268b7b3087430b': compiler.cppstd=gnu11
libfdk_aac/2.0.3: 'f11ee86fd912b8b4eac7887b306be7319b16dd88': compiler.cppstd=14
libfdk_aac/2.0.3: '5cf156348bf66992fb29725e7b77997b5a975209': compiler.cppstd=gnu14
libx265/3.4: Main binary package '27f9a8e1f6add0ac421bbab1cba21392d8dae8e9' missing
libx265/3.4: Checking 5 compatible configurations
libx265/3.4: Compatible configurations not found in cache, checking servers
libx265/3.4: '1f6db8eb9e7d07468ae2adac8f57d57a346b85b0': compiler.cppstd=gnu98
libx265/3.4: '736be10e392831888d0ba547f1e6091226a8e488': compiler.cppstd=11
libx265/3.4: '7fedfc636ee0d07db33e4c9b18812ff4ddf92cbf': compiler.cppstd=gnu11
libx265/3.4: 'f9b884c39df1526cf6d07f794b18e7ec808ab24a': compiler.cppstd=14
libx265/3.4: '636af3c50734a289df8ee9f341c97ccafe25643b': compiler.cppstd=gnu14
flac/1.4.2: Main binary package '6d559790f7605bbc0385970f11af3c2dfbac5d35' missing
flac/1.4.2: Checking 5 compatible configurations
flac/1.4.2: Compatible configurations not found in cache, checking servers
flac/1.4.2: '1b5c1ea6e4d3a7051d91e6e82fdc43db2d2d4d46': compiler.cppstd=gnu98
flac/1.4.2: 'f8c6f55568350b3947354107e82d21d6aa730416': compiler.cppstd=11
flac/1.4.2: '8970fc2b88c6dd8f5a64f0ae198b3d644651a643': compiler.cppstd=gnu11
flac/1.4.2: 'b43bb1807c63f3ae19cb0ad3ba8cafbd1cb5fccc': compiler.cppstd=14
flac/1.4.2: 'dc09b7b8c1df0f60fc755e38873016b74a808a3e': compiler.cppstd=gnu14
libvpx/1.13.1: Main binary package '1202021ff6721b5b3241bb0c1446ce018c465f21' missing
libvpx/1.13.1: Checking 5 compatible configurations
libvpx/1.13.1: Compatible configurations not found in cache, checking servers
libvpx/1.13.1: 'a90e3d43014c251aec776929c3dca250307248e4': compiler.cppstd=gnu98
libvpx/1.13.1: 'e8eeaf49a704154e127f9bc268514957dedda2e0': compiler.cppstd=11
libvpx/1.13.1: '3057da643bfcc3b42fb166bd0b268b7b3087430b': compiler.cppstd=gnu11
libvpx/1.13.1: 'f11ee86fd912b8b4eac7887b306be7319b16dd88': compiler.cppstd=14
libvpx/1.13.1: '5cf156348bf66992fb29725e7b77997b5a975209': compiler.cppstd=gnu14
openh264/2.3.1: Main binary package '1202021ff6721b5b3241bb0c1446ce018c465f21' missing
openh264/2.3.1: Checking 5 compatible configurations
openh264/2.3.1: Compatible configurations not found in cache, checking servers
openh264/2.3.1: 'a90e3d43014c251aec776929c3dca250307248e4': compiler.cppstd=gnu98
openh264/2.3.1: 'e8eeaf49a704154e127f9bc268514957dedda2e0': compiler.cppstd=11
openh264/2.3.1: '3057da643bfcc3b42fb166bd0b268b7b3087430b': compiler.cppstd=gnu11
openh264/2.3.1: 'f11ee86fd912b8b4eac7887b306be7319b16dd88': compiler.cppstd=14
openh264/2.3.1: '5cf156348bf66992fb29725e7b77997b5a975209': compiler.cppstd=gnu14
Requirements
    autoconf/2.71#f9307992909d7fb3df459340f1932809:da39a3ee5e6b4b0d3255bfef95601890afd80709#5b77f70c17ad1741f5845d4e468a347e - Cache
    automake/1.16.5#058bda3e21c36c9aa8425daf3c1faf50:9a4eb3c8701508aa9458b1a73d0633783ecc2270#9719e51a6a62041af6a63e00eef35434 - Cache
    brotli/1.1.0#d56d7bb9ca722942aba17369cb5c0519:aa01c1ac62b0458581f663ae4714719756842887 - Build
    bzip2/1.0.8#d00dac990f08d991998d624be81a9526:865a5d462c834b1d3f8731abe1eed86d3f5d351c - Build
    ffmpeg/4.2.1#8bd30bd1ea06b582da850bcd44ac5eaf:c889df406a1d285fb970f22c4484885de5923215 - Build
    flac/1.4.2#d4051f8b37ee93725dea40dfd6a10168:6d559790f7605bbc0385970f11af3c2dfbac5d35 - Build
    freetype/2.13.2#5d2563803c8558d4ef47271a82c73d20:0ebbaf3719eb91405a60fb22927f5b5128219ec2 - Build
    libalsa/1.2.10#e64d5e1ced869a2f676145bab4f4a181:6e79fda323cb31b1178443fea75c4ae55ff0b07a - Build
    libaom-av1/3.6.1#a2b22c70d6fce43887881431808ab8a6:59ca810128384c10cc00bfa99b1e05f204ac5bfd - Build
    libcap/2.69#7ef2d60864c2e58e89db957be936dc49:c65b76e542cbaa028e850371eb5466d695c9dc9d - Build
    libfdk_aac/2.0.3#0115f6598be7303e042684e3a846b12d:1202021ff6721b5b3241bb0c1446ce018c465f21 - Build
    libiconv/1.17#73fefc1b696e069df90fd1d18aa63edd:59ca810128384c10cc00bfa99b1e05f204ac5bfd - Build
    libmp3lame/3.100#44b12d19316eb2b223d98d3e75dae438:59ca810128384c10cc00bfa99b1e05f204ac5bfd - Build
    libpng/1.6.44#9e1aa08fb46946c7c91e4ae03bd49811:d1a3328658da10e1cb6694d2ced1efa937ba305f - Build
    libsndfile/1.2.2#b47faa587d50886068a4d2f385383ab9:de5176c0aa3441d7c3302c69478a4fba6b3f7c6d - Build
    libtool/2.4.7#08316dad5c72c541ed21e039e4cf217b:59ca810128384c10cc00bfa99b1e05f204ac5bfd - Build
    libvpx/1.13.1#de05642042bff632e366b0a13c72285a:1202021ff6721b5b3241bb0c1446ce018c465f21 - Build
    libwebp/1.3.2#52f69c4a31c5cf033fdd9230d77a8e38:7dfeded24221a01a6e0d02de933cd49f07dce624 - Build
    libx264/cci.20220602#a9d536e10203f4c5a27a437f103661b4:d1135d0a0a44d9fedcfecc13b5f7cf110aa3c615 - Build
    libx265/3.4#719e50b2b2c3fd1b9133fea12da42c62:27f9a8e1f6add0ac421bbab1cba21392d8dae8e9 - Build
    m4/1.4.19#b38ced39a01e31fef5435bc634461fd2:ded9ebf53856b9e6effe24cd64f649c25030457b - Build
    mpg123/1.31.2#ce831c936b2284e2066ab3dc58a2628e:30c117d3c2f9617bceb93db9fcda78c967a6d9e1 - Build
    ogg/1.3.5#062626875f5c8c59f069f76f148098ef:59ca810128384c10cc00bfa99b1e05f204ac5bfd - Build
    openh264/2.3.1#8990004bf4953883c68d24118790424a:1202021ff6721b5b3241bb0c1446ce018c465f21 - Build
    openjpeg/2.5.0#2886244f2492121aa7c6c493a814db06:59ca810128384c10cc00bfa99b1e05f204ac5bfd - Build
    openssl/3.3.2#9f9f130d58e7c13e76bb8a559f0a6a8b:e273bfcda713523e04be650ea10f228942aec16b#1737368ce7fc67e905c0d47390761ff5 - Cache
    opus/1.4#3c98a306d127dce1e74d58a0e2c850b5:5e7abbe64f0cc44e11a4dd66bd1a1ab7f95501af - Build
    pulseaudio/14.2#30afeff38a7770cb9cda17a66da49a19:e7a0ab72c8c5590a179825f5901a97ab8a19cdb3 - Build
    vaapi/system#9fa40059fb979af4f9b301b84fb2dfa1:da39a3ee5e6b4b0d3255bfef95601890afd80709#0ba8627bd47edc3a501e8f0eb9a79e5e - Download (conancenter)
    vdpau/system#bdd7d010c4e3d8762a8f6f08a3cf6760:da39a3ee5e6b4b0d3255bfef95601890afd80709#0ba8627bd47edc3a501e8f0eb9a79e5e - Download (conancenter)
    vorbis/1.3.7#37e58f52e59a6232199b34ef402714a6:1922ef923dc700081ec252e3c93fc39834317d17 - Build
    xorg/system#98f82cb669e4ebc6b4d9d8a4f3f1faf4:da39a3ee5e6b4b0d3255bfef95601890afd80709#0ba8627bd47edc3a501e8f0eb9a79e5e - Download (conancenter)
    xz_utils/5.4.5#b885d1d79c9d30cff3803f7f551dbe66:59ca810128384c10cc00bfa99b1e05f204ac5bfd - Build
    zlib/1.3.1#f52e03ae3d251dec704634230cd806a2:59ca810128384c10cc00bfa99b1e05f204ac5bfd#89c34254f508d490661206d72dee42d9 - Cache
Build requirements
    autoconf/2.71#f9307992909d7fb3df459340f1932809:da39a3ee5e6b4b0d3255bfef95601890afd80709#5b77f70c17ad1741f5845d4e468a347e - Cache
    automake/1.16.5#058bda3e21c36c9aa8425daf3c1faf50:9a4eb3c8701508aa9458b1a73d0633783ecc2270#9719e51a6a62041af6a63e00eef35434 - Cache
    gettext/0.21#a2bd3513f212013764c9040f4c70ed69:b4120e8c1e2c977a29df862ea05f40353962bfa3 - Build
    gnu-config/cci.20210814#dc430d754f465e8c74463019672fb97b:da39a3ee5e6b4b0d3255bfef95601890afd80709#22618e30bd9e326eb95e824dc90cc860 - Cache
    libiconv/1.17#73fefc1b696e069df90fd1d18aa63edd:47620b9691817df16e56c395a6204b80badfc353 - Build
    libtool/2.4.7#08316dad5c72c541ed21e039e4cf217b:47620b9691817df16e56c395a6204b80badfc353#ac1e80ec04c44c46fe31a2a623a0dac3 - Cache
    m4/1.4.19#b38ced39a01e31fef5435bc634461fd2:3593751651824fb813502c69c971267624ced41a#cd8019a4f9b70880d548f7cc26569604 - Cache
    meson/1.3.2#26ce8a76a36cc275cdfee1d757bc6561:da39a3ee5e6b4b0d3255bfef95601890afd80709#3ba677cf44c95996f4f326c668f92f00 - Download (conancenter)
    meson/1.4.1#43b753adf9a089cadfa17c8034f5c897:da39a3ee5e6b4b0d3255bfef95601890afd80709#4a43a051838fc016591c956c0389e89e - Download (conancenter)
    ninja/1.12.1#fd583651bf0c6a901943495d49878803:3593751651824fb813502c69c971267624ced41a#a46e32b2b79add597b73d8b42c415ed1 - Download (conancenter)
    pkgconf/2.0.3#f996677e96e61e6552d85e83756c328b:c0b621fd4b3199fe05075171573398833dba85f4#625ebc76863307827004d778d898184a - Download (conancenter)
    pkgconf/2.1.0#27f44583701117b571307cf5b5fe5605:c0b621fd4b3199fe05075171573398833dba85f4#74a82dec52448dda20253a19d65965c7 - Cache
    pkgconf/2.2.0#6462942a22803086372db44689ba825f:c0b621fd4b3199fe05075171573398833dba85f4#35e5163b1cf42becef616e6b7873202e - Download (conancenter)
    yasm/1.3.0#fb800a15413dca19bfaef9e4b5d50694:3593751651824fb813502c69c971267624ced41a#de209941eeb4029e79a7a33bfefe9a98 - Download (conancenter)
Skipped binaries
    meson/1.2.2
dpkg-query: No package matching libva-dev:arm was found.
ERROR: vaapi/system: Error in system_requirements() method, line 38
    apt.install(["libva-dev"], update=True, check=True)
    ConanException: System requirements: 'libva-dev' are missing but can't install because tools.system.package_manager:mode is 'check'.Please update packages manually or set 'tools.system.package_manager:mode' to 'install' in the [conf] section of the profile, or in the command line using '-c tools.system.package_manager:mode=install'

Have you read the CONTRIBUTING guide?

memsharded commented 2 weeks ago

Hi @sunhao-DBH

Thanks for your question.

It would be important to know first what is your specific platform, like Linux distro details, and see if the liba-dev:arm package exists there or not. It

You might also try to avoid that installation/check with tools.system.package_manager:mode=report, which will allow to pass that point, but it is very likely that when trying to compile it will fail because it will miss the liba-dev package for the specific architecture.

It is possible that this cross-build scenario requires some more advanced setup. Are you using some sysroot for this cross-compilation? If that is the case, maybe what is necessary is to skip the dependency to vaapi/system and fallback to the "system" one, that would be found in your sysroot. This can be done with the [platform_requires] feature for the vaapi/system.

Please let me know if this helps.

sunhao-DBH commented 2 weeks ago

Thank you for your reply.

I am using Ubuntu 22.04 on the x86_64 architecture.

I think Conan will install the development packages for the x86_64 architecture for me, instead of installing the development packages for the ARM architecture from the sources of Ubuntu.

May I ask how I should set it up?

memsharded commented 2 weeks ago

I think Conan will install the development packages for the x86_64 architecture for me, instead of installing the development packages for the ARM architecture from the sources of Ubuntu.

Sorry, I don't know what you mean. If you are trying to cross-compile to arm, Conan shouldn't be installing the x86_64 packages, because it needs to build for arm architecture, this is why it is trying to install the liba-dev:arm package, not the native one. To cross compile the arm package is needed, or it might be provided somewhere else, but installing the x86_64 doesn't seem a solution.

Have you tried my suggestion above of tools.system.package_manager:mode=report?

sunhao-DBH commented 2 weeks ago

Thank you very much for your patient explanation.

I have tried tools.system.package_manager:mode=report, and indeed the following error will occur:

ERROR: vaapi/system: Error in package_info() method, line 55
        pkg_config.fill_cpp_info(self.cpp_info.components[name])
        ConanException: PkgConfig failed. Command: pkg-config --print-provides libva --print-errors
    stdout:

    stderr:
    Package libva was not found in the pkg-config search path.
    Perhaps you should add the directory containing `libva.pc'
    to the PKG_CONFIG_PATH environment variable
    Package 'libva', required by 'virtual:world', not found

However, when I use the tools.system.package_manager:mode=install command, Conan will execute the apt install libva-dev:arm command on the amd64 architecture. Obviously, even if I use the dpkg --add-architecture arm command to enable support for the arm architecture, it is highly unlikely that this command will succeed. It seems that only amd64 and i386 packages can be installed on the amd64 architecture.

jcar87 commented 2 weeks ago

Hi @sunhao-DBH - Debian-based distros are multi-architecture, you can install packages from other architectures - they are placed in different system subfolders.

Although I suspsect the issue here is that 32-bit armv7 should map to the armhf dpkg architecture

sunhao-DBH commented 2 weeks ago

Thank you for your reply. I also think that libva-dev:armhf should be downloaded. However, what I don't understand is why Conan automatically downloads libva-dev:arm. Is there something that I haven't configured properly?

jcar87 commented 2 weeks ago

That may be a bug - will look into it!

In the meantime to unblock you, you can try:

dpkg --add-architecture armhf
apt-get install libva-dev:armhf

And then proceed with the conan install invocation.

There may be other packages to install - you can get a list from Conan You can also run a

 conan graph- info  .  --build=missing -pr armv7 -c tools.system.package_manager:mode=report

This should report all packages that need to be installed and you can install them manually while we look into the correct dpkg architecture for armv7!

sunhao-DBH commented 1 week ago

OK, thank you. I like Conan. Looking forward to Conan getting better and better.

memsharded commented 1 week ago

I see the current mapping is:

self._arch_names = {"x86_64": "amd64",
                            "x86": "i386",
                            "ppc32": "powerpc",
                            "ppc64le": "ppc64el",
                            "armv7": "arm",
                            "armv7hf": "armhf",
                            "armv8": "arm64",
                            "s390x": "s390x"} if arch_names is None else arch_names

It seems the armv7 mapping to arm is not correct because arm architecture is not recognized in debian apt repos?

I have done a quick search for a explicit list of architectures, but it is not evident to me.