conan-io / cmake-conan

CMake wrapper for conan C and C++ package manager
MIT License
823 stars 250 forks source link

[develop2] CMAKE_SYSTEM_VERSION variable is not parsed when cross-compiling for Android #563

Closed nmgwddj closed 11 months ago

nmgwddj commented 1 year ago

https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-for-android-with-the-ndk

This article describes that when the target platform of cross-compilation is Android, CMake can determine the API level of Android by specifying CMAKE_SYSTEM_VERSION. My historical script is also implemented in this way, but in cmake-conan dev2, it does not detect this variable, but only ANDROID_PLATFORM is used. Is there any special purpose for this, and will support for this variable be added?

CMAKE_ANDROID_STL_TYPE has the same problem, I have to use ANDROID_STL instead.

memsharded commented 1 year ago

Hi @nmgwddj

Thanks for your feedback. I think this sounds a valid request (feature or fix), and we want to parse CMAKE_SYSTEM_VERSION as well besides the ANDROID_PLATFORM. (same for STL)

I am summoning @ssrobins and @jcar87 to see their opinions, if they agree, this sounds relatively easy to implement.

ssrobins commented 1 year ago

Make sense to support all of the ways to generate an Android build in CMake. I recall this method being broken awhile back with certain NDKs, which is when I started using the toolchain method, but I guess it got fixed.

I observed that CMAKE_ANDROID_STL_TYPE already gets set when ANDROID_STL is set so conan_provider.cmake could just get the CMAKE_ANDROID_STL_TYPE.

CMAKE_SYSTEM_VERSION is a bit more complicated, I see it defaulting to 1 when not explicitly set by the user, which would be an invalid API setting in modern NDKs.

But as long as this Android generation works:

cmake -D CMAKE_SYSTEM_NAME=Android -D CMAKE_ANDROID_NDK=/path/to/ndk/ -D CMAKE_ANDROID_ARCH_ABI=arm64-v8a -D CMAKE_SYSTEM_VERSION=28 -D CMAKE_ANDROID_STL_TYPE=c++_static -D CMAKE_PROJECT_TOP_LEVEL_INCLUDES="../conan_provider.cmake" -B build_android

...and this still works as well:

cmake -D CMAKE_TOOLCHAIN_FILE=/path/to/ndk/build/cmake/android.toolchain.cmake -D ANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-28 -D ANDROID_STL=c++_static -D CMAKE_PROJECT_TOP_LEVEL_INCLUDES="../conan_provider.cmake" -B build_android

Everyone should be happy :)

I think additional test coverage will be important here. Happy to help review and test!

nmgwddj commented 1 year ago

Make sense to support all of the ways to generate an Android build in CMake. I recall this method being broken awhile back with certain NDKs, which is when I started using the toolchain method, but I guess it got fixed.

I observed that CMAKE_ANDROID_STL_TYPE already gets set when ANDROID_STL is set so conan_provider.cmake could just get the CMAKE_ANDROID_STL_TYPE.

CMAKE_SYSTEM_VERSION is a bit more complicated, I see it defaulting to 1 when not explicitly set by the user, which would be an invalid API setting in modern NDKs.

But as long as this Android generation works:

cmake -D CMAKE_SYSTEM_NAME=Android -D CMAKE_ANDROID_NDK=/path/to/ndk/ -D CMAKE_ANDROID_ARCH_ABI=arm64-v8a -D CMAKE_SYSTEM_VERSION=28 -D CMAKE_ANDROID_STL_TYPE=c++_static -D CMAKE_PROJECT_TOP_LEVEL_INCLUDES="../conan_provider.cmake" -B build_android

...and this still works as well:

cmake -D CMAKE_TOOLCHAIN_FILE=/path/to/ndk/build/cmake/android.toolchain.cmake -D ANDROID_ABI=arm64-v8a -DANDROID_PLATFORM=android-28 -D ANDROID_STL=c++_static -D CMAKE_PROJECT_TOP_LEVEL_INCLUDES="../conan_provider.cmake" -B build_android

Everyone should be happy :)

I think additional test coverage will be important here. Happy to help review and test!

Hi @ssrobins, thank you for paying attention to this issue. I tested it based on your example, but I still encounter some problems. Could you please help me find out where the error is?

Test project: https://github.com/nmgwddj/conan-cmake-v2-example

I get some errors using the following commands:

cmake -Bbuild -DCMAKE_BUILD_TYPE=Debug \
    -DCMAKE_SYSTEM_NAME=Android \
    -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \
    -DCMAKE_SYSTEM_VERSION=21 \
    -DCMAKE_ANDROID_STL_TYPE=c++_static \
    -DCMAKE_ANDROID_NDK=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529 \
    -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=$(pwd)/cmake/conan_provider.cmake
Output: ``` conan-cmake-v2-example git:(main) cmake -Bbuild -DCMAKE_BUILD_TYPE=Debug \ -DCMAKE_SYSTEM_NAME=Android \ -DCMAKE_ANDROID_NDK=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529 \ -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \ -DCMAKE_SYSTEM_VERSION=21 \ -DCMAKE_ANDROID_STL_TYPE=c++_static \ -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=$(pwd)/cmake/conan_provider.cmake -- Android: Targeting API '21' with architecture 'arm64', ABI 'arm64-v8a', and processor 'aarch64' -- Android: Selected unified Clang toolchain -- The C compiler identification is Clang 9.0.9 -- The CXX compiler identification is Clang 9.0.9 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- CMake-Conan: first find_package() found. Installing dependencies with Conan -- CMake-Conan: Checking if a default profile exists /Users/jj.deng/.conan2/profiles/default -- CMake-Conan: cmake_system_name=Android CMake Error at cmake/conan_provider.cmake:22 (string): string sub-command REGEX, mode MATCH needs at least 5 arguments total to command. Call Stack (most recent call first): cmake/conan_provider.cmake:192 (detect_os) cmake/conan_provider.cmake:359 (detect_host_profile) CMakeLists.txt:7 (find_package) -- CMake-Conan: android_platform= -- CMake-Conan: cmake_system_processor=armv8 -- CMake-Conan: CMake compiler=Clang -- CMake-Conan: CMake compiler version=9.0.9 -- CMake-Conan: [settings] compiler=clang -- CMake-Conan: [settings] compiler.version=9 -- CMake-Conan: android_stl= -- CMake-Conan: Creating profile /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan_host_profile -- CMake-Conan: Profile: include(default) [settings] arch=armv8 os=Android compiler=clang compiler.version=9 compiler.cppstd=14 build_type=Debug [conf] tools.cmake.cmaketoolchain:generator=Unix Makefiles tools.android:ndk_path=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529 -- CMake-Conan: Installing single configuration Debug -- CMake-Conan: conan install /Users/jj.deng/Documents/open-source/conan-cmake-v2-example -of=/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan -pr;/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan_host_profile;--build=missing;-g;CMakeDeps ======== Input profiles ======== Profile host: [settings] arch=armv8 build_type=Debug compiler=clang compiler.cppstd=14 compiler.libcxx=libc++ compiler.version=9 os=Android [conf] tools.cmake.cmaketoolchain:generator=Unix Makefiles tools.android:ndk_path=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529 Profile build: [settings] arch=armv8 build_type=Release compiler=clang compiler.cppstd=gnu17 compiler.libcxx=libc++ compiler.version=16 os=Macos ======== Computing dependency graph ======== Graph root conanfile.txt: /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/conanfile.txt Requirements fmt/9.1.0#f79808141cb396759a829be09dd5592c - Cache Build requirements cmake/3.27.5#aa6b0dfc03844c3ce4bb57e2dfc33058 - Cache ======== Computing necessary packages ======== Requirements fmt/9.1.0#f79808141cb396759a829be09dd5592c:930f6a8f4244e3e04fefa9d588fe7d718b21da63 - Invalid Build requirements cmake/3.27.5#aa6b0dfc03844c3ce4bb57e2dfc33058:9e5323c65b94ae38c3c733fe12637776db0119a5#03a2ac6ed5a6afcffad19fd333252455 - Cache ======== Installing packages ======== ERROR: There are invalid packages: fmt/9.1.0: Invalid: 'settings.os.api_level' value not defined CMake Error at cmake/conan_provider.cmake:295 (message): Conan install failed='6' Call Stack (most recent call first): cmake/conan_provider.cmake:363 (conan_install) CMakeLists.txt:7 (find_package) CMake Error at CMakeLists.txt:7 (find_package): By not providing "Findfmt.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "fmt", but CMake did not find one. Could not find a package configuration file provided by "fmt" with any of the following names: fmtConfig.cmake fmt-config.cmake Add the installation prefix of "fmt" to CMAKE_PREFIX_PATH or set "fmt_DIR" to a directory containing one of the above files. If "fmt" provides a separate development package or SDK, be sure it has been installed. -- Configuring incomplete, errors occurred! ```

After replacing the parameters supported by Gradle, it passes normally:

cmake -Bbuild -DCMAKE_BUILD_TYPE=Debug \
    -DCMAKE_SYSTEM_NAME=Android \
    -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \
    -DANDROID_PLATFORM=21 \
    -DANDROID_STL=c++_static \
    -DCMAKE_ANDROID_NDK=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529 \
    -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=$(pwd)/cmake/conan_provider.cmake
Output: ``` conan-cmake-v2-example git:(main) ✗ cmake -Bbuild -DCMAKE_BUILD_TYPE=Debug \ -DCMAKE_SYSTEM_NAME=Android \ -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \ -DANDROID_PLATFORM=21 \ -DANDROID_STL=c++_static \ -DCMAKE_ANDROID_NDK=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529 \ -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=$(pwd)/cmake/conan_provider.cmake -- CMake-Conan: first find_package() found. Installing dependencies with Conan -- CMake-Conan: Checking if a default profile exists /Users/jj.deng/.conan2/profiles/default -- CMake-Conan: cmake_system_name=Android -- CMake-Conan: android_platform=21 -- CMake-Conan: cmake_system_processor=armv8 -- CMake-Conan: CMake compiler=Clang -- CMake-Conan: CMake compiler version=9.0.9 -- CMake-Conan: [settings] compiler=clang -- CMake-Conan: [settings] compiler.version=9 -- CMake-Conan: android_stl=c++_static -- CMake-Conan: Creating profile /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan_host_profile -- CMake-Conan: Profile: include(default) [settings] arch=armv8 os=Android os.api_level=21 compiler=clang compiler.version=9 compiler.cppstd=14 compiler.libcxx=c++_static build_type=Debug [conf] tools.cmake.cmaketoolchain:generator=Unix Makefiles tools.android:ndk_path=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529 -- CMake-Conan: Installing single configuration Debug -- CMake-Conan: conan install /Users/jj.deng/Documents/open-source/conan-cmake-v2-example -of=/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan -pr;/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan_host_profile;--build=missing;-g;CMakeDeps ======== Input profiles ======== Profile host: [settings] arch=armv8 build_type=Debug compiler=clang compiler.cppstd=14 compiler.libcxx=c++_static compiler.version=9 os=Android os.api_level=21 [conf] tools.cmake.cmaketoolchain:generator=Unix Makefiles tools.android:ndk_path=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529 Profile build: [settings] arch=armv8 build_type=Release compiler=clang compiler.cppstd=gnu17 compiler.libcxx=libc++ compiler.version=16 os=Macos ======== Computing dependency graph ======== Graph root conanfile.txt: /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/conanfile.txt Requirements fmt/9.1.0#f79808141cb396759a829be09dd5592c - Cache Build requirements cmake/3.27.5#aa6b0dfc03844c3ce4bb57e2dfc33058 - Cache ======== Computing necessary packages ======== Requirements fmt/9.1.0#f79808141cb396759a829be09dd5592c:4d6f05da4738bd9f34f5a7b210fa73e602bc3370#27de9bf664c0d78c08ac9a1acf0ece3e - Cache Build requirements cmake/3.27.5#aa6b0dfc03844c3ce4bb57e2dfc33058:9e5323c65b94ae38c3c733fe12637776db0119a5#03a2ac6ed5a6afcffad19fd333252455 - Cache ======== Installing packages ======== cmake/3.27.5: Already installed! (1 of 2) cmake/3.27.5: Appending PATH environment variable: /Users/jj.deng/.conan2/p/cmakef8adadd3ea5f6/p/CMake.app/Contents/bin fmt/9.1.0: Already installed! (2 of 2) WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X: WARN: deprecated: 'env_info' used in: cmake/3.27.5 WARN: deprecated: 'cpp_info.names' used in: fmt/9.1.0 ======== Finalizing install (deploy, generators) ======== conanfile.txt: Writing generators to /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan/build/Debug/generators conanfile.txt: Generator 'CMakeDeps' calling 'generate()' conanfile.txt: Generating aggregated env files conanfile.txt: Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh'] Install finished successfully -- CMake-Conan: CONAN_GENERATORS_FOLDER=/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan/build/Debug/generators -- CMake-Conan: CONANFILE=/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/conanfile.txt -- Conan: Component target declared 'fmt::fmt' -- Configuring done (0.6s) -- Generating done (0.0s) -- Build files have been written to: /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build ```
ssrobins commented 11 months ago

Hi @nmgwddj I posted a fix in https://github.com/conan-io/cmake-conan/pull/580 Does this fix your setup as well?

nmgwddj commented 11 months ago

Sorry, I didn't reply in time because I was busy. I have verified the new cmake script and it recognizes these parameters normally, below is the record after executing the command.

conan-cmake-v2-example git:(main) ✗ cmake -Bbuild -DCMAKE_BUILD_TYPE=Debug \
    -DCMAKE_SYSTEM_NAME=Android \
    -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \
    -DCMAKE_SYSTEM_VERSION=21 \
    -DCMAKE_ANDROID_STL_TYPE=c++_static \
    -DCMAKE_ANDROID_NDK=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529 \
    -DCMAKE_PROJECT_TOP_LEVEL_INCLUDES=$(pwd)/cmake/conan_provider.cmake
-- CMake-Conan: first find_package() found. Installing dependencies with Conan
-- CMake-Conan: Checking if a default profile exists
/Users/jj.deng/.conan2/profiles/default
-- CMake-Conan: cmake_system_name=Android
-- CMake-Conan: android api level=21
-- CMake-Conan: cmake_system_processor=armv8
-- CMake-Conan: CMake compiler=Clang
-- CMake-Conan: CMake compiler version=9.0.9
-- CMake-Conan: [settings] compiler=clang
-- CMake-Conan: [settings] compiler.version=9
-- CMake-Conan: android_stl=c++_static
-- CMake-Conan: Creating profile /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan_host_profile
-- CMake-Conan: Profile: 
[settings]
arch=armv8
os=Android
os.api_level=21
compiler=clang
compiler.version=9
compiler.cppstd=14
compiler.libcxx=c++_static
build_type=Debug
[conf]
tools.cmake.cmaketoolchain:generator=Unix Makefiles
tools.build:compiler_executables={"c":"/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang","cpp":"/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++"}
tools.android:ndk_path=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529

-- CMake-Conan: Installing single configuration Debug
-- CMake-Conan: conan install /Users/jj.deng/Documents/open-source/conan-cmake-v2-example -of=/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan --profile:host=default;--profile:host=/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan_host_profile;--profile:build=default;--build=missing;-g;CMakeDeps

======== Input profiles ========
Profile host:
[settings]
arch=armv8
build_type=Debug
compiler=clang
compiler.cppstd=14
compiler.libcxx=c++_static
compiler.version=9
os=Android
os.api_level=21
[conf]
tools.cmake.cmaketoolchain:generator=Unix Makefiles
tools.build:compiler_executables={'c': '/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang', 'cpp': '/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++'}
tools.android:ndk_path=/Users/jj.deng/Library/Android/sdk/ndk/21.4.7075529

Profile build:
[settings]
arch=armv8
build_type=Release
compiler=clang
compiler.cppstd=gnu17
compiler.libcxx=libc++
compiler.version=16
os=Macos

======== Computing dependency graph ========
Graph root
    conanfile.txt: /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/conanfile.txt
Requirements
    fmt/9.1.0#f79808141cb396759a829be09dd5592c - Cache
Build requirements
    cmake/3.27.5#aa6b0dfc03844c3ce4bb57e2dfc33058 - Cache

======== Computing necessary packages ========
Requirements
    fmt/9.1.0#f79808141cb396759a829be09dd5592c:4d6f05da4738bd9f34f5a7b210fa73e602bc3370#27de9bf664c0d78c08ac9a1acf0ece3e - Cache
Build requirements
    cmake/3.27.5#aa6b0dfc03844c3ce4bb57e2dfc33058:9e5323c65b94ae38c3c733fe12637776db0119a5#03a2ac6ed5a6afcffad19fd333252455 - Cache

======== Installing packages ========
cmake/3.27.5: Already installed! (1 of 2)
cmake/3.27.5: Appending PATH environment variable: /Users/jj.deng/.conan2/p/cmakef8adadd3ea5f6/p/CMake.app/Contents/bin
fmt/9.1.0: Already installed! (2 of 2)
WARN: deprecated: Usage of deprecated Conan 1.X features that will be removed in Conan 2.X:
WARN: deprecated:     'env_info' used in: cmake/3.27.5
WARN: deprecated:     'cpp_info.names' used in: fmt/9.1.0

======== Finalizing install (deploy, generators) ========
conanfile.txt: Writing generators to /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan/build/Debug/generators
conanfile.txt: Generator 'CMakeDeps' calling 'generate()'
conanfile.txt: Generating aggregated env files
conanfile.txt: Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh']
Install finished successfully
-- CMake-Conan: CONAN_GENERATORS_FOLDER=/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build/conan/build/Debug/generators
-- CMake-Conan: CONANFILE=/Users/jj.deng/Documents/open-source/conan-cmake-v2-example/conanfile.txt
-- Conan: Component target declared 'fmt::fmt'
-- Configuring done (0.5s)
-- Generating done (0.0s)
-- Build files have been written to: /Users/jj.deng/Documents/open-source/conan-cmake-v2-example/build
jcar87 commented 11 months ago

Fixed in https://github.com/conan-io/cmake-conan/pull/580

Thanks @nmgwddj and @ssrobins!!