Closed CBenoit closed 2 months ago
thenextman
commented
3 months ago The CMakeLists asks for a minimum cmake version of 3.0.
Modern Android NDKs require at least 3.6 to enable the proper policies: https://github.com/android/ndk/issues/2032.
justsmth
commented
3 months ago Hello!
Looking at the error message (Android: Unknown processor CMAKE_SYSTEM_PROCESSOR='i686'.), it appears to originate here:
if(CMAKE_SYSTEM_PROCESSOR AND NOT DEFINED "NDK_PROC_${CMAKE_SYSTEM_PROCESSOR}_ABI")
message(FATAL_ERROR "Android: Unknown processor CMAKE_SYSTEM_PROCESSOR='${CMAKE_SYSTEM_PROCESSOR}'.")
endif()There does seem to be a matching NDK_PROC_i686_ABI defined here in CMake 3.30.2
set(NDK_PROC_i686_ABI "x86")However, the above definition is only available when NDK_KNOWN_DEVICE_ABIS is not defined by the NDK.(?)
I don't have a solution (yet), but here's a suggestions to try:
ANDROID_NDK_HOME is being set in your environment, but CMake only recognizes the following environment variables from the environment: ANDROID_NDK_ROOT, ANDROID_NDK, or ANDROID_STANDALONE_TOOLCHAIN. Would you try setting one of these in your environment (or verify that one is already set).Something like the following should do it:
echo "[env]" >> $CargoConfigFile
echo "ANDROID_NDK_ROOT=`"${Env:ANDROID_NDK_HOME}`"" >> $CargoConfigFile
echo "ANDROID_STANDALONE_TOOLCHAIN=`"${Env:ANDROID_NDK_HOME}/toolchains/llvm/prebuilt/linux-x86_64`"" >> $CargoConfigFileAlso, I often see more success using Ninja as the CMake Generator. If you have it installed, you might try setting it as the CMAKE_GENERATOR in your build environment (or in the [env] section of the cargo config):
echo "CMAKE_GENERATOR=`"Ninja`"" >> $CargoConfigFileLet us know whether you have any success with this, or if there are changes you recommend for our build scripts. Thanks!
CBenoit
commented
3 months ago * It appears that `ANDROID_NDK_HOME` is being set in your environment, but CMake only recognizes [the following environment variables](https://github.com/Kitware/CMake/blob/d88682dff6bf053e5bbdc10accf5d6825303e656/Modules/Platform/Android-Determine.cmake#L139-L182) from the environment: `ANDROID_NDK_ROOT`, `ANDROID_NDK`, or `ANDROID_STANDALONE_TOOLCHAIN`. Would you try setting one of these in your environment (or verify that one is already set).Something like the following should do it:
echo "[env]" >> $CargoConfigFile echo "ANDROID_NDK_ROOT=`"${Env:ANDROID_NDK_HOME}`"" >> $CargoConfigFile echo "ANDROID_STANDALONE_TOOLCHAIN=`"${Env:ANDROID_NDK_HOME}/toolchains/llvm/prebuilt/linux-x86_64`"" >> $CargoConfigFile
Indeed, we used ANDROID_NDK_HOME in our script, but we also have ANDROID_NDK_ROOT set with the same value.
It’s set at the runner image level:
https://github.com/actions/runner-images/blob/d72a4523b15eb2e303c1ea5100a23c34d9475f4f/images/ubuntu/Ubuntu2004-Readme.md?plain=1#L281-L283
Also, I often see more success using Ninja as the CMake Generator. If you have it installed, you might try setting it as the CMAKE_GENERATOR in your build environment (or in the
[env]section of the cargo config):echo "CMAKE_GENERATOR=`"Ninja`"" >> $CargoConfigFile
Just to be on the safe side, I tried to apply all your suggestions, even if ANDROID_NDK_ROOT is already properly defined:
@@ -93,7 +93,9 @@ jobs:
shell: pwsh
run: |
$CargoConfigFile = "~/.cargo/config.toml"
- $AndroidToolchain="${Env:ANDROID_NDK_HOME}/toolchains/llvm/prebuilt/linux-x86_64"
+
+ $AndroidToolchain="${Env:ANDROID_NDK_ROOT}/toolchains/llvm/prebuilt/linux-x86_64"
+ Write-Host "\$AndroidToolchain=$AndroidToolchain"
Get-ChildItem -Path $AndroidToolchain "libunwind.a" -Recurse | ForEach-Object {
$libunwind = $_.FullName
@@ -130,6 +132,13 @@ jobs:
echo "CXX_aarch64-linux-android=$AndroidToolchain/bin/aarch64-linux-android21-clang++" >> $Env:GITHUB_ENV
echo "AR_aarch64-linux-android=$AndroidToolchain/bin/llvm-ar" >> $Env:GITHUB_ENV
+ echo "[env]" >> $CargoConfigFile
+ echo "ANDROID_NDK_ROOT=`"${Env:ANDROID_NDK_ROOT}`"" >> $CargoConfigFile
+ echo "ANDROID_STANDALONE_TOOLCHAIN=`"${Env:ANDROID_NDK_HOME}/toolchains/llvm/prebuilt/linux-x86_64`"" >> $CargoConfigFile
+ echo "CMAKE_GENERATOR=`"Ninja`"" >> $CargoConfigFile
+
+ Get-Content -Path "$CargoConfigFile"
+
- name: Configure macOS deployement target
if: ${{ matrix.os == 'osx' }}
shell: pwshThe final cargo.toml ends up looking like this:
[target.i686-linux-android]
linker="/usr/local/lib/android/sdk/ndk/27.0.12077973/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android21-clang"
[target.x86_64-linux-android]
linker="/usr/local/lib/android/sdk/ndk/27.0.12077973/toolchains/llvm/prebuilt/linux-x86_64/bin/x86_64-linux-android21-clang"
[target.armv7-linux-androideabi]
linker="/usr/local/lib/android/sdk/ndk/27.0.12077973/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi21-clang"
[target.aarch64-linux-android]
linker="/usr/local/lib/android/sdk/ndk/27.0.12077973/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android21-clang"
[env]
ANDROID_NDK_ROOT="/usr/local/lib/android/sdk/ndk/27.0.12077973"
ANDROID_STANDALONE_TOOLCHAIN="/usr/local/lib/android/sdk/ndk/27.0.12077973/toolchains/llvm/prebuilt/linux-x86_64"
CMAKE_GENERATOR="Ninja"I still get the exact same error.
Let us know whether you have any success with this, or if there are changes you recommend for our build scripts. Thanks!
At this point, all I can recommend is to bump the minimum supported CMake version to 3.6 in the CMakeLists files. CMake 3.6 was released in 2016, and we are already getting deprecation warnings when running with a recent CMake version for anything below 3.5.
CMake Deprecation Warning at CMakeLists.txt:4 (cmake_minimum_required):
Compatibility with CMake < 3.5 will be removed from a future version of
CMake.
Update the VERSION argument <min> value or use a ...<max> suffix to tell
CMake that the project does not need compatibility with older versions.I believe both the CMakeLists files from both aws-lc-rs and aws-lc should be updated accordingly.
I’ll try to fork your repositories and give it a try myself a bit later.
CBenoit
commented
3 months ago Updating the the minimum CMake version did not fix the problem. The only difference is that the deprecation warning is gone.
The following warnings were emitted during compilation:
warning: CMAKE environment variable set: cmake
warning: Generating bindings - external bindgen. Platform: i686-linux-android
error: failed to run custom build command for `aws-lc-sys v0.21.0 (https://github.com/CBenoit/aws-lc-rs?rev=0267230#02672305)`
Caused by:
process didn't exit successfully: `/home/runner/work/IronRDP/IronRDP/target/release/build/aws-lc-sys-e4c4b09ca74409c5/build-script-main` (exit status: 101)
--- stdout
cargo:rerun-if-env-changed=AWS_LC_SYS_NO_PREFIX
cargo:rerun-if-env-changed=AWS_LC_SYS_INTERNAL_BINDGEN
cargo:rerun-if-env-changed=AWS_LC_SYS_EXTERNAL_BINDGEN
cargo:rerun-if-env-changed=AWS_LC_SYS_NO_ASM
cargo:rerun-if-env-changed=AWS_LC_SYS_CMAKE_BUILDER
cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
cargo:rerun-if-env-changed=CMAKE
cargo:rerun-if-env-changed=CMAKE
cargo:warning=CMAKE environment variable set: cmake
cargo:warning=Generating bindings - external bindgen. Platform: i686-linux-android
cargo:rerun-if-env-changed=AWS_LC_SYS_INCLUDES
cargo:rustc-cfg=use_bindgen_generated
cargo:rerun-if-env-changed=AWS_LC_SYS_STATIC
cargo:rerun-if-env-changed=CMAKE_TOOLCHAIN_FILE
cargo:rerun-if-env-changed=CMAKE_TOOLCHAIN_FILE_i686_linux_android
cargo:rerun-if-env-changed=CC
cargo:rerun-if-env-changed=CXX
CMAKE_TOOLCHAIN_FILE_i686-linux-android = None
CMAKE_TOOLCHAIN_FILE_i686_linux_android = None
TARGET_CMAKE_TOOLCHAIN_FILE = None
CMAKE_TOOLCHAIN_FILE = None
CMAKE_GENERATOR_i686-linux-android = None
CMAKE_GENERATOR_i686_linux_android = None
TARGET_CMAKE_GENERATOR = None
CMAKE_GENERATOR = Some("Ninja")
CMAKE_PREFIX_PATH_i686-linux-android = None
CMAKE_PREFIX_PATH_i686_linux_android = None
TARGET_CMAKE_PREFIX_PATH = None
CMAKE_PREFIX_PATH = None
CMAKE_i686-linux-android = None
CMAKE_i686_linux_android = None
TARGET_CMAKE = None
CMAKE = Some("cmake")
running: cd "/home/runner/work/IronRDP/IronRDP/target/i686-linux-android/release/build/aws-lc-sys-839edbc0e7b15a0d/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/home/runner/.cargo/git/checkouts/aws-lc-rs-2f2036c626affde5/0267230/aws-lc-sys" "-G" "Ninja" "-DBUILD_SHARED_LIBS=0" "-DCMAKE_BUILD_TYPE=release" "-DBORINGSSL_PREFIX=aws_lc_0_21_0_" "-DBORINGSSL_PREFIX_HEADERS=/home/runner/.cargo/git/checkouts/aws-lc-rs-2f2036c626affde5/0267230/aws-lc-sys/generated-include" "-DBUILD_TESTING=OFF" "-DBUILD_LIBSSL=OFF" "-DDISABLE_PERL=ON" "-DDISABLE_GO=ON" "-DCMAKE_SYSTEM_NAME=Android" "-DCMAKE_SYSTEM_PROCESSOR=i686" "-DCMAKE_INSTALL_PREFIX=/home/runner/work/IronRDP/IronRDP/target/i686-linux-android/release/build/aws-lc-sys-839edbc0e7b15a0d/out" "-DCMAKE_C_FLAGS= -ffile-prefix-map=/home/runner/.cargo/git/checkouts/aws-lc-rs-2f2036c626affde5/0267230/aws-lc-sys= -DANDROID -ffunction-sections -fdata-sections -fPIC -m32" "-DCMAKE_C_COMPILER=/usr/local/lib/android/sdk/ndk/27.0.12077973/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android21-clang" "-DCMAKE_CXX_FLAGS= -DANDROID -ffunction-sections -fdata-sections -fPIC -m32" "-DCMAKE_CXX_COMPILER=/usr/local/lib/android/sdk/ndk/27.0.12077973/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android21-clang++" "-DCMAKE_ASM_FLAGS= -DANDROID -ffunction-sections -fdata-sections -fPIC -m32" "-DCMAKE_ASM_COMPILER=/usr/local/lib/android/sdk/ndk/27.0.12077973/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android21-clang" "--no-warn-unused-cli"
Not searching for unused variables given on the command line.
-- Configuring incomplete, errors occurred!
--- stderr
CMake Error at /usr/local/share/cmake-3.30/Modules/Platform/Android-Determine.cmake:379 (message):
Android: Unknown processor CMAKE_SYSTEM_PROCESSOR='i686'.
Call Stack (most recent call first):
/usr/local/share/cmake-3.30/Modules/CMakeDetermineSystem.cmake:184 (include)
CMakeLists.txt:6 (project)Changing the minimum version to 3.30.2 wasn’t effective either. I’ll try setting the variables such as NDK_PROC_i686_ABI manually.
CBenoit
commented
3 months ago @justsmth I’m able to build with this patch: https://github.com/CBenoit/aws-lc-rs/commit/f9868b531b649a1bf11656122d08436b846c8673
Would you be open to me sending you a PR to address the problem like this?
justsmth
commented
3 months ago @CBenoit - I'm glad you got the build working! 🎉
It looks like your patch is reproducing the values as specified here in CMake's own configuration files. Apparently, theNDK_KNOWN_DEVICE_ABIS build flag is being set somewhere, while leaving the NDK_PROC_*_ABI (and other flags?) without proper values.
The comment here indicates that NDK >= 23 should set all of the values (including NDK_KNOWN_DEVICE_ABIS and NDK_PROC_*_ABI). I wonder if something might be missing from the version of the NDK used by your build?
CBenoit
commented
3 months ago That’s my understanding too. Because of that, the fix above does not feel right. Our build is using the default NDK provided in GitHub Workflow images which is >= 27 as of today at least. Thanks to your last link, I found out that the NDK is providing a file called abis.cmake. I found the content of this file as defined in NDK 27: https://android.googlesource.com/toolchain/prebuilts/ndk/r27/+/refs/heads/main/build/cmake/abis.cmake
By the way, this is how it looked in, say, NDK 23: https://android.googlesource.com/toolchain/prebuilts/ndk/r23/+/refs/heads/main/build/cmake/abis.cmake
Basically, newer NDK are not providing us with the expected variables.
For some reason, they changed the variables like this:
set(NDK_PROC_i686_ABI "x86") => set(NDK_ABI_x86_PROC "i686")I’m guessing the proper fix would be to revert whatever they did on NDK side (unlikely), or to update CMake to handle that…
What do you think? It would help to have my patch as a crutch, but it would be wise to at least put that behind an optional feature flag or environment variable since it’s not exactly the cleanest approach. (I personally suggest an environment variable.) Or, if you know of some way to set this kind of variables without calling cmake manually I would be happy with that too. Your build.rs script is handling that part, so I can’t change how it’s called unless I modify the sources directly like this.
justsmth
commented
3 months ago This is interesting.
NDK_PROC_i686_ABI is not being set by the NDK r27, but it was being set in r26.
The logic in CMake here fails when CMAKE_SYSTEM_PROCESSOR is set but NDK_PROC_i686_ABI is not set:
if(CMAKE_SYSTEM_PROCESSOR)
if(NOT DEFINED "NDK_PROC_${CMAKE_SYSTEM_PROCESSOR}_ABI")
message(FATAL_ERROR "Android: Unknown processor CMAKE_SYSTEM_PROCESSOR='${CMAKE_SYSTEM_PROCESSOR}'.")
endif()
set(CMAKE_ANDROID_ARCH_ABI "${NDK_PROC_${CMAKE_SYSTEM_PROCESSOR}_ABI}")I wonder if what we should do here is just not set the CMAKE_SYSTEM_PROCESSOR when the NDK version is r27 or later?
CBenoit
commented
3 months ago Yes! This would likely work by fall-backing on CMake built-in variables! However, @thenextman sent me this link in private a bit earlier: https://github.com/android/ndk/issues/2049 It looks like it was broke by mistake, and then fixed in a more recent version… Just waiting for a newer version of the NDK to be used in GitHub runner images may be the best fix after all :laughing: That being said, I would be happy to have an option via env variables to get a working build sooner than later. The env variable could be removed in the future.
justsmth
commented
3 months ago I was looking at the GitHub runner configurations. One option might be to set ANDROID_NDK (and ANDROID_NDK_HOME, ANDROID_NDK_ROOT, ANDROID_NDK_LATEST_HOME) to /usr/local/lib/android/sdk/ndk/26.3.11579264 so that the build would use NDK r26.
This might fix the build until the GitHub runner images are updated? (This also assumes that you're not currently needing r27 for your build.)
CBenoit
commented
2 months ago Oh, you’re right. I think we can do that temporarily! I’ll close the issue since I don’t think there is any specific action that need to be taken on aws-lc-rs side. Thank you for following up!
Problem:
We are running the build in a GitHub workflow with something looking like this:
But it fails when CMake is run in the build.rs.
Relevant details
AWS-LC for Rust versions or commit:
aws-lc-sys v0.20.1System information: ubuntu-20.04 GitHub runner
Build log: