Unable to compile AWS C++ SDK for Android with NDK 13b

[DeepakArora76]

Using the below command, I am unable to compile for Android.

Command: cmake -G "NMake Makefiles" -DTARGET_ARCH=ANDROID -DBUILD_ONLY="cognito-identity,lambda" -DCMAKE_BUILD_TYPE=Release -DANDROID_STL=gnustl_static -DANDROID_NATIVE_API_LEVEL=21 -DANDROID_ABI=armeabi-v7a -DNDK_DIR=C:/Android-ndk-r13b -DANDROID_NDK=C:/Android-ndk-r13b -DANDROID_TOOLCHAIN_NAME=arm-linux-androideabi-4.9 D:/AWS/aws-sdk-cpp

[DeepakArora76]

Using aforementioned command I am getting the following output on console.

-- Building AWS libraries as shared objects -- Android ABI: armeabi-v7a -- Android std lib: gnustl_shared -- Android API level: 21 -- Standalone toolchain disabled; this is not a well-supported option -- Building project version: 1.0.37 -- The CXX compiler identification is GNU 4.9.0 -- Check for working CXX compiler: C:/Android-ndk-r13b/toolchains/arm-linux-androideabi-4.9/prebuilt/windows-x86_64/bin/arm-linux-androideabi-g++.exe -- Check for working CXX compiler: C:/Android-ndk-r13b/toolchains/arm-linux-androideabi-4.9/prebuilt/windows-x86_64/bin/arm-linux-androideabi-g++.exe -- broken CMake Error at C:/Program Files/CMake/share/cmake-3.7/Modules/CMakeTestCXXCompiler.cmake:44 (message): The C++ compiler "C:/Android-ndk-r13b/toolchains/arm-linux-androideabi-4.9/prebuilt/windows-x86_64/bin/arm-linux-androideabi-g++.exe" is not able to compile a simple test program.

It fails with the following output:

Change Dir: D:/AWS/aws-sdk-cpp/CMakeFiles/CMakeTmp

Run Build Command:"nmake" "/NOLOGO" "cmTC_a9d5a\fast"

    "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\nmake.exe" -f

CMakeFiles\cmTC_a9d5a.dir\build.make /nologo -L CMakeFiles\cmTC_a9d5a.dir\build

Building CXX object CMakeFiles/cmTC_a9d5a.dir/testCXXCompiler.cxx.o

    C:\Android-ndk-r13b\toolchains\arm-linux-androideabi-4.9\prebuilt\windows-x86_64\bin\arm-linux-androideabi-g++.exe

-DANDROID -isystem C:\Android-ndk-r13b\platforms\android-21\arch-arm\usr\include -isystem C:\Android-ndk-r13b\sources\cxx-stl\gnu-libstdc++\4.9\include -isystem C:\Android-ndk-r13b\sources\cxx-stl\gnu-libstdc++\4.9\libs\armeabi-v7a\include -isystem C:\Android-ndk-r13b\sources\cxx-stl\gnu-libstdc++\4.9\include\backward -Wno-psabi --sysroot=C:/Android-ndk-r13b/platforms/android-21/arch-arm -funwind-tables -finline-limit=64 -fsigned-char -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fdata-sections -ffunction-sections -Wa,--noexecstack -fPIE -o CMakeFiles\cmTC_a9d5a.dir\testCXXCompiler.cxx.o -c D:\AWS\aws-sdk-cpp\CMakeFiles\CMakeTmp\testCXXCompiler.cxx

Linking CXX executable cmTC_a9d5a

    CMAKE_C_COMPILER -Wno-psabi

--sysroot=C:/Android-ndk-r13b/platforms/android-21/arch-arm -funwind-tables -finline-limit=64 -fsigned-char -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fdata-sections -ffunction-sections -Wa,--noexecstack -Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,-allow-shlib-undefined -Wl,--gc-sections -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-z,nocopyreloc -fPIE -pie "CMakeFiles\cmTC_a9d5a.dir\testCXXCompiler.cxx.o" -o cmTC_a9d5a -L\libs\armeabi-v7a "C:/Android-ndk-r13b/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/libgnustl_shared.so" "C:/Android-ndk-r13b/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/libsupc++.a" -lm

'CMAKE_C_COMPILER' is not recognized as an internal or external command,

operable program or batch file.

NMAKE : fatal error U1077: 'CMAKE_C_COMPILER' : return code '0x1'

Stop.

NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\nmake.exe"' : return code '0x2'

Stop.

CMake will not be able to correctly generate this project. Call Stack (most recent call first): CMakeLists.txt:90 (project)

-- Configuring incomplete, errors occurred! See also "D:/AWS/aws-sdk-cpp/CMakeFiles/CMakeOutput.log". See also "D:/AWS/aws-sdk-cpp/CMakeFiles/CMakeError.log".

[DeepakArora76]

I tried with the following types for ANDROID_STL and none of them are working. none
system
system_re
gabi++_static gabi++_shared stlport_static stlport_shared gnustl_static gnustl_shared

[bretambrose]

Of the stl implementations you list, only gnustl will ever have a chance of successfully compiling the SDK. The rest are incomplete and/or out-of-date with C++11 (to the best of my knowledge).

That being said, unless you don't have a choice in the matter, we strongly recommend not using any of them and instead using libc++ (which is selected by default if you do not specify ANDROID_STL). Gnustl has a severe performance problem due to a necessary workaround against a misguided string optimization setting in the standard library.

We (and Google for that matter) also recommend not using gcc as the compiler and instead using clang (again this is also the default if you leave off ANDROID_TOOLCHAIN_NAME) on Android.

I will look into the current state of the gnustl build against 13b, but given Google's deprecation of gcc in favor of clang, I'm really reluctant to spend much time trying to support gcc as well. If you are able to resolve the issue, we are, of course, happy to take pull requests.

[bretambrose]

I should also add, one of the things Jonathan has on his mind is a customer contract page where we detail our (the C++ team here) intended contract with users of the SDK (backwards compatibility, supported platforms, etc...) Android is certainly a part of that and while we haven't hammered out the details yet, the support baseline will likely match what I've detailed above (NDK10 and up, clang, libc++ fully, gnustl partially). We would love to support gnustl fully but we just don't see a solution short of completely replacing std::string and every related type (stringstream, stringbuf, etc...).

[DeepakArora76]

One important point which I missed to highlight was that I have a Desktop running Windows 10 Pro 64 bit

I tried your advice and I am getting "Host system 'windows-x86' is not supported by the source NDK! Try --system= with one of: windows"

Where I could specify that the build host machine is Windows 64 bit?

Appreciate your inputs on this.

Console output: cmake -G "NMake Makefiles" -DTARGET_ARCH=ANDROID -DBUILD_ONLY="cognito-identity,lambda" -DCMAKE_BUILD_TYPE=Release -DANDROID_STL=gnustl_static -DANDROID_ABI=armeabi-v7a -DNDK_DIR=C:/Android/android-ndk-r10d -DANDROID_NDK=C:/Android/android-ndk-r10d -DANDROID_TOOLCHAIN_NAME=arm-linux-androideabi-4.9 D:/AWS/aws-sdk-cpp

-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.10.1.windows.1") -- Building AWS libraries as shared objects -- Android ABI: armeabi-v7a -- Android std lib: gnustl_shared -- Android API level: android-19 -- Detected NDK version r10d -- Could not find an appropriate standalone toolchain. Generating one into D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-arm-linux-androideabi-4.9-android-19-gnustl_shared-10004 Host system 'windows-x86' is not supported by the source NDK! Try --system= with one of: windows chmod: cannot access 'D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-arm-linux-androideabi-4.9-android-19-gnustl_shared-10004': No such file or directory -- Building project version: 1.0.37 -- The CXX compiler identification is GNU 4.9.0 -- Check for working CXX compiler: C:/Android/Android-ndk-r10d/toolchains/arm-linux-androideabi-4.9/prebuilt/windows/bin/arm-linux-androideabi-g++.exe -- Check for working CXX compiler: C:/Android/Android-ndk-r10d/toolchains/arm-linux-androideabi-4.9/prebuilt/windows/bin/arm-linux-androideabi-g++.exe -- broken CMake Error at C:/Program Files/CMake/share/cmake-3.7/Modules/CMakeTestCXXCompiler.cmake:44 (message): The C++ compiler "C:/Android/Android-ndk-r10d/toolchains/arm-linux-androideabi-4.9/prebuilt/windows/bin/arm-linux-androideabi-g++.exe" is not able to compile a simple test program.

It fails with the following output:

Change Dir: D:/AWS/aws-sdk-cpp-build/CMakeFiles/CMakeTmp Run Build Command:"nmake" "/NOLOGO" "cmTC_293f6\fast" "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\nmake.exe" -f CMakeFiles\cmTC_293f6.dir\build.make /nologo -L CMakeFiles\cmTC_293f6.dir\build

Building CXX object CMakeFiles/cmTC_293f6.dir/testCXXCompiler.cxx.o

C:\Android\Android-ndk-r10d\toolchains\arm-linux-androideabi-4.9\prebuilt\windows\bin\arm-linux-androideabi-g++.exe -DANDROID -isystem C:\Android\Android-ndk-r10d\platforms\android-19\arch-arm\usr\include -isystem C:\Android\Android-ndk-r10d\sources\cxx-stl\gnu-libstdc++\4.9\include -isystem C:\Android\Android-ndk-r10d\sources\cxx-stl\gnu-libstdc++\4.9\libs\armeabi-v7a\include -isystem C:\Android\Android-ndk-r10d\sources\cxx-stl\gnu-libstdc++\4.9\include\backward -fexceptions -frtti -Wno-psabi --sysroot=C:/Android/Android-ndk-r10d/platforms/android-19/arch-arm -funwind-tables -finline-limit=64 -fsigned-char -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fdata-sections -ffunction-sections -Wa,--noexecstack -fPIE -o CMakeFiles\cmTC_293f6.dir\testCXXCompiler.cxx.o -c D:\AWS\aws-sdk-cpp-build\CMakeFiles\CMakeTmp\testCXXCompiler.cxx

Linking CXX executable cmTC_293f6

    CMAKE_C_COMPILER -fexceptions -frtti -Wno-psabi

--sysroot=C:/Android/Android-ndk-r10d/platforms/android-19/arch-arm -funwind-tables -finline-limit=64 -fsigned-char -no-canonical-prefixes -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -fdata-sections -ffunction-sections -Wa,--noexecstack -Wl,--fix-cortex-a8 -Wl,--no-undefined -Wl,--gc-sections -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,-z,nocopyreloc -fPIE -pie "CMakeFiles\cmTC_293f6.dir\testCXXCompiler.cxx.o" -o cmTC_293f6 -L\libs\armeabi-v7a "C:/Android/Android-ndk-r10d/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/libgnustl_shared.so" "C:/Android/Android-ndk-r10d/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/libsupc++.a" -lm

'CMAKE_C_COMPILER' is not recognized as an internal or external command,

operable program or batch file.

NMAKE : fatal error U1077: 'CMAKE_C_COMPILER' : return code '0x1'

Stop.

NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\BIN\nmake.exe"' : return code '0x2'

Stop.

CMake will not be able to correctly generate this project. Call Stack (most recent call first): CMakeLists.txt:90 (project)

-- Configuring incomplete, errors occurred! See also "D:/AWS/aws-sdk-cpp-build/CMakeFiles/CMakeOutput.log". See also "D:/AWS/aws-sdk-cpp-build/CMakeFiles/CMakeError.log".

[bretambrose]

The toolchain setup script tends to get tweaked a lot so something has probably changed between 12 and 13 on Windows. I will look into it; I only have Win7 at work so I may not be able to reproduce this until the weekend at home.

[DeepakArora76]

I will be looking forward to your inputs.

BTW, I have cleaned a bit the build command which now looks like:

cmake -G "NMake Makefiles" -DTARGET_ARCH=ANDROID -DBUILD_ONLY="cognito-identity,lambda" -DCMAKE_BUILD_TYPE=Release -DNDK_DIR=C:/Android/android-ndk-r13b -DANDROID_NDK=C:/Android/android-ndk-r13b -DANDROID_TOOLCHAIN_NAME=arm-linux-androideabi-4.9 D:/AWS/aws-sdk-cpp

-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.10.1.windows.1") -- Building AWS libraries as shared objects -- Android ABI: none specified, defaulting to armeabi-v7a -- Libc++ requires setting API level to at least 21 -- Android std lib: libc++_shared -- Android API level: android-21 -- Detected NDK version r13b -- Could not find an appropriate standalone toolchain. Generating one into D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-arm-linux-androideabi-4.9-android-21-libc++_shared-13002 chmod: cannot access 'D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-arm-linux-androideabi-4.9-android-21-libc++_shared-13002': No such file or directory -- Building project version: 1.0.37 CMake Error at cmake/platform/android.toolchain.cmake:1054 (message): Unknown runtime: libc++_shared Call Stack (most recent call first): C:/Program Files/CMake/share/cmake-3.7/Modules/CMakeDetermineSystem.cmake:88 (include) CMakeLists.txt:90 (project)

CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage -- Configuring incomplete, errors occurred!

[bretambrose]

Try dropping the "ANDROID_NDK=" option as well as either dropping "ANDROID_TOOLCHAIN_NAME" or changing it to "standalone" or "standalone-clang"

[bretambrose]

If you get an error while building zlib about 'patch', either add it to your path or change line 30 of build_external.cmake to:

PATCH_COMMAND ""

The patch dependency will be removed in a release next week, so if you want to wait and not make SDK changes, you can do that too.

[DeepakArora76]

Still same or similar problems.

Attempt-1: Without ANDROID_NDK and ANDROID_TOOLCHAIN_NAME

cmake -G "NMake Makefiles" -DTARGET_ARCH=ANDROID -DBUILD_ONLY="cognito-identity;lambda" -DCMAKE_BUILD_TYPE=Release -DNDK_DIR=C:/Android/Android-ndk-r13b D:/AWS/aws-sdk-cpp

-- Found Git: C:/Program Files/Git/cmd/git.exe (found version "2.10.1.windows.1") -- Building AWS libraries as shared objects -- Android ABI: none specified, defaulting to armeabi-v7a -- Android toolchain unspecified, defaulting to standalone-clang -- Libc++ requires setting API level to at least 21 -- Android std lib: libc++_shared -- Android API level: android-21 -- Detected NDK version r13b -- Could not find an appropriate standalone toolchain. Generating one into D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-standalone-clang-android-21-libc++_shared-13002 chmod: cannot access 'D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-standalone-clang-android-21-libc++_shared-13002': No such file or directory -- Building project version: 1.0.37 CMake Error at cmake/platform/android.toolchain.cmake:308 (file): file STRINGS file "D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-standalone-clang-android-21-libc++_shared-13002/sysroot/usr/include/android/api-level.h" cannot be read. Call Stack (most recent call first): cmake/platform/android.toolchain.cmake:507 (__DETECT_NATIVE_API_LEVEL) C:/Program Files/CMake/share/cmake-3.7/Modules/CMakeDetermineSystem.cmake:88 (include) CMakeLists.txt:90 (project)

CMake Error at cmake/platform/android.toolchain.cmake:512 (message): Could not determine machine name of your toolchain. Probably your Android standalone toolchain is broken. Call Stack (most recent call first): C:/Program Files/CMake/share/cmake-3.7/Modules/CMakeDetermineSystem.cmake:88 (include) CMakeLists.txt:90 (project)

CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage -- Configuring incomplete, errors occurred!

[DeepakArora76]

Attempt-2: With ANDROID_TOOLCHAIN_NAME="standalone-clang"

cmake -G "NMake Makefiles" -DTARGET_ARCH=ANDROID -DBUILD_ONLY="cognito-identity;lambda" -DANDROID_TOOLCHAIN_NAME="standalone-clang" -DCMAKE_BUILD_TYPE=Release -DNDK_DIR=C:/Android/Android-ndk-r13b D:/AWS/aws-sdk-cpp

-- Building AWS libraries as shared objects -- Android ABI: none specified, defaulting to armeabi-v7a -- Android std lib: libc++_shared -- Android API level: android-21 -- Standalone toolchain disabled; this is not a well-supported option -- Building project version: 1.0.37 CMake Error at cmake/platform/android.toolchain.cmake:447 (message): Could not find neither Android NDK nor Android standalone toolchain.

  You should either set an environment variable:
    export ANDROID_NDK=~/my-android-ndk
  or
    export ANDROID_STANDALONE_TOOLCHAIN=~/my-android-toolchain
  or put the toolchain or NDK in the default path:
    sudo ln -s ~/my-android-ndk C:/Program Files/android-ndk
    sudo ln -s ~/my-android-toolchain /opt/android-toolchain

Call Stack (most recent call first): C:/Program Files/CMake/share/cmake-3.7/Modules/CMakeDetermineSystem.cmake:88 (include) CMakeLists.txt:90 (project)

CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage -- Configuring incomplete, errors occurred!

[DeepakArora76]

I reran the same command as in the Attempt-2. This time the output was different. Attempt-3: With ANDROID_TOOLCHAIN_NAME="standalone-clang"

cmake -G "NMake Makefiles" -DTARGET_ARCH=ANDROID -DBUILD_ONLY="cognito-identity;lambda" -DANDROID_TOOLCHAIN_NAME="standalone-clang" -DCMAKE_BUILD_TYPE=Release -DNDK_DIR=C:/Android/Android-ndk-r13b D:/AWS/aws-sdk-cpp

-- Building AWS libraries as shared objects -- Android ABI: none specified, defaulting to armeabi-v7a -- Android std lib: libc++_shared -- Android API level: android-21 -- Detected NDK version r13b -- Could not find an appropriate standalone toolchain. Generating one into D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-standalone-clang-android-21-libc++_shared-13002 chmod: cannot access 'D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-standalone-clang-android-21-libc++_shared-13002': No such file or directory -- Building project version: 1.0.37 CMake Error at cmake/platform/android.toolchain.cmake:308 (file): file STRINGS file "D:/AWS/aws-sdk-cpp/toolchains/android/armeabi-v7a-standalone-clang-android-21-libc++_shared-13002/sysroot/usr/include/android/api-level.h" cannot be read. Call Stack (most recent call first): cmake/platform/android.toolchain.cmake:507 (__DETECT_NATIVE_API_LEVEL) C:/Program Files/CMake/share/cmake-3.7/Modules/CMakeDetermineSystem.cmake:88 (include) CMakeLists.txt:90 (project)

CMake Error at cmake/platform/android.toolchain.cmake:512 (message): Could not determine machine name of your toolchain. Probably your Android standalone toolchain is broken. Call Stack (most recent call first): C:/Program Files/CMake/share/cmake-3.7/Modules/CMakeDetermineSystem.cmake:88 (include) CMakeLists.txt:90 (project)

CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage -- Configuring incomplete, errors occurred!

[DeepakArora76]

I even tried setting ANDROID_TOOLCHAIN_NAME="standalone" and the output was same as with attempts 2 and 3.

I started to giveup on this. I believe, the only option I have is to use Android SDK and use JNI to Interface with C++.

I am using Windows 10 64bit Pro. Visual Studio 2015. NDK13rb. I have not installed LLVM (CLang) stuff.

[DeepakArora76]

Just wondering, are you not getting these errors on the aforementioned build environment (Windows 10 64bit Pro. Visual Studio 2015. NDK13rb.)?

[bretambrose]

Given the changes I noted in the command line, I am not getting any errors. I am using a Developer Command Prompt for Visual Studio 2015 as the shell (although it shouldn't really matter). Does the toolchain directory in the error output exist or is the problem that chmod isn't on the path? If chmod isn't on the path you can probably just delete that line in android.cmake:

execute_process(COMMAND chmod a+r ${STANDALONE_TOOLCHAIN_DIR})

and we will make it more forgiving (it was originally added for someone who was building in a shell that wasn't correctly setting up read permissions on the generated toolchain directory).

[bretambrose]

Hmm, looking closer, it's just not generating the toolchain and also not generating any errors during the process.

If that's the case, then that's something you're going to need to dig into by yourself. The relevant cmake line (android.cmake again) is this:

            execute_process(
                COMMAND python ${NDK_DIR}/build/tools/make_standalone_toolchain.py --arch=${__ARCH} --api=${ANDROID_NATIVE_API_LEVEL_NUM} --stl=${STANDALONE_TOOLCHAIN_STL} --install-dir=${STANDALONE_TOOLCHAIN_DIR} WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
            )

That invokes the NDK's python script for making a standalone toolchain. In order to see what's actually being run, you can add some output before that line like this:

message(STATUS "Running: python ${NDK_DIR}/build/tools/make_standalone_toolchain.py --arch=${__ARCH} --api=${ANDROID_NATIVE_API_LEVEL_NUM} --stl=${STANDALONE_TOOLCHAIN_STL} --install-dir=${STANDALONE_TOOLCHAIN_DIR} WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}")

Now you'll be able to see the exact command being executed which should give you a starting point figuring out why that script isn't making you a toolchain. My first guess would be some sort of permission problem such that the folder wasn't even getting created.

[DeepakArora76]

Bret, thank you for your very valuable inputs. The tip which you gave really proved an ice-breaker. The problem orginated due to the fact Python was not installed.

Would be really nice if this could be mentioned under Prerequisite.

All looks good and rocking for now. Thanks again for your support.

[DeepakArora76]

It would be nice if the build script have some kind of checklist, or mechanism to ensure that the necessary toolchain is in place. This could save time and the trouble that any Developers go through.

[bretambrose]

I'm glad we found what was wrong. I will add a python install check for the Android build so that no one else has these troubles.