CheckCompileLinkOption fails when Xcode compiler is invoked from command line

[aciereszko]

When testing for the compiler capabilities with:

CheckCompileLinkOption("-msse2" CRYPTOPP_IA32_SSE2 "${TEST_PROG_DIR}/test_x86_sse2.cxx")

CheckCompileLinkOption invokes the CMAKE_CXX_COMPILER directly and compiles the test file. This file will fail to compile with an error message that it can't locate stdlib.h, when the compiler is clang from the xcode toolchain. Try to run:

/Applications/Xcode9.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -msse2 test_x86_sse2.cxx

This is the error:

In file included from test_x86_sse2.cxx:1:
In file included from /Applications/Xcode9.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include/emmintrin.h:27:
In file included from /Applications/Xcode9.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include/xmmintrin.h:39:
In file included from /Applications/Xcode9.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/9.0.0/include/mm_malloc.h:27:
/Applications/Xcode9.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/stdlib.h:94:15: fatal error: 'stdlib.h' file not found
#include_next <stdlib.h>
              ^~~~~~~~~~
1 error generated.

However as soon as you add the -isysroot pointing to the correct sysroot, compiles without a problem.

This seems to be a problem with the compilers in this path in the toolchain, the clang in /usr/bin/ works fine.

So, there should be a way to modify CmakeLists to take the sysroot into account, but I'm not sure what would be the elegant way to do that...

[noloader]

Thanks @aciereszko.

Please show how you are invoking CMake. I think you need to set some options like below. That's just the way CMake works.

export CXX=/Applications/Xcode9.2.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
export CXXFLAGS="... -isysroot=..."
cmake <source dir> -DCMAKE_CXX_COMPILER="$CXX" -DCMAKE_CXX_FLAGS="$CXXFLAGS"

We cut-in CheckCompileLinkOption recently and it uses CXXFLAGS. You should be able to use --isysroot if needed:

function(CheckCompileLinkOption opt var prog)

    message(STATUS "Performing Test ${var}")
    execute_process(
      COMMAND sh -c "${CMAKE_CXX_COMPILER} ${CMAKE_CXX_FLAGS} ${opt} ${prog} 2>&1"
      RESULT_VARIABLE COMMAND_RESULT
      OUTPUT_VARIABLE COMMAND_OUTPUT
      OUTPUT_STRIP_TRAILING_WHITESPACE)
    ...

endfunction(CheckCompileLinkOption)

---

There is also a related problem that we can't get CMake to stop using the C compiler. I believe it is due to a handful of CMake bugs. I believe project is the recommended way to state "C++ project", but I've never been able to get project(cryptopp CXX) to work. It fails to work as expected on Solaris and PowerPC.

I've also tried to set the C compiler to the C++ compiler in CMakeList.txt : 8, but CMake fails the build claiming the C compiler is a C++ compiler:

# Should we be setting things like this? We are not a C project
# so nothing should be done with the C compiler. But there is
# no reliable way to tell CMake we are C++.
# Cannot set this... Breaks Linux PowerPC with Clang:
# SET(CMAKE_C_COMPILER ${CMAKE_CXX_COMPILER})
# # error "The CMAKE_C_COMPILER is set to a C++ compiler"

I mention it because the random C compiler may screw up your cross-compile.

[aciereszko]

Hi noloader,

Thanks for the quick response. I'm using CLion on macos, this is the cmake command that is called when you reload the project:

/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake -DCMAKE_BUILD_TYPE=Debug -G "CodeBlocks - Unix Makefiles" <project_folder_path>

Forgive my CMake ignorance... So I do not have to pass sysroot to cmake for CLion and generated makefiles to know which sysroot to tack on when compiling. From what I understand the generator is able to figure out based on the xcode toolchain that is active (xcode-select).

I understand that you're bypassing all this and calling a shell command during the cmake execution to determine which processor instruction sets are available, so at this point there is no knowledge of sysroot yet?

Since this is a cross platform project, built for macos, ios, windows, linux, I'm relying on cmake and the generators to figure out sysroot, and would rather not have to pass it it, as this would make the maintenance more of a problem, just for this one dependency...

Thanks for your help!

Andrzej

[noloader]

From what I understand the generator is able to figure out based on the xcode toolchain that is active (xcode-select)... Since this is a cross platform project, built for macos, ios, windows, linux, I'm relying on cmake and the generators to figure out sysroot, and would rather not have to pass it it, as this would make the maintenance more of a problem, just for this one dependency...

The part about having CMake figure out sysroot and add it to compile and link flags sounds fine with me. I thought that's the way it happened. Maybe it is not happening like we expect.

Do you know why CMake is not setting flags like --sysroot or CMAKE_CXX_FLAGS correctly? Or maybe I should ask, how do we tell CMake to set options like --sysroot as expected?

Let's ping @abdes. He knows more about CMake than I do.

---

Related, we had to cut-in our own CheckCompileOption and CheckCompileLinkOption because CMake was mis-reporting option availability when we used CHECK_CXX_COMPILER_FLAG. The compiler would reject an option with a message like "illegal option" and the tool would report success. Then, we would compile with the option and the build would fail. My apologies if it is causing problems.

[aciereszko]

To be honest I am not sure when CMake sets the CMAKE_CXX_FLAGS, but from what I understand you are saying that at the time you are executing the shell command to compile the test cxx files you would expect CMAKE_CXX_FLAGS to contain the -isysroot flag? I will have to do some digging around to see what is happening.

The interesting part is that if I use the /usr/bin/clang that test cxx file compiles without the need to add the -isysroot flag but when I use the clang from the xcode toolchain location it does not (I dumped the command from your cmakelists file with a message statement and ran it in terminal).

Anyway no need to apologize, I suspect I am doing something wrong, you guys are doing great work!

[mouse07410]

My suggestion is within CMakeLists.txt check for OS, and if it's MacOS set SYSROOT to output of xcrun --sdk-path. I'll post details when I get to my computer, but it's pretty simple (unlike many things with CMake).

[noloader]

@mouse07410 ,

What is the common case among developers when using Xcode? Do developers bounce around with different compilers from the SDK? Or do they set the default and stick to one compiler from the SDK?

I ask because I don't ever recall setting a SDK path. I usually just use a compiler from the toolchain of interest or the default one. In fact, we bounce around SDKs in setenv-ios.sh (though we set a --sysroot).

In fact it looks like we usually use the default toolchain at line 166 and then build a sysroot like /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS8.2.sdk around line 180.

[aciereszko]

There is an easy for way for you guys to reproduce this problem. On macos, you can download CLion, and load your cmake project (follow your directions to get the cryptopp repo and drop your two cmake files).

Open the project and CLion will run cmake and you will see the Tests fail. You will need to have XCode installed, and xcode-select install is run, to install the command line tools. Verify that cmake is using the compiler from the xcode toolchain.

Or you can just run the command the CLion is using without installing it, I get the same error (the output above is actually from that):

cmake -DCMAKE_BUILD_TYPE=Debug -G "CodeBlocks - Unix Makefiles" <cryptopp_path>

I'm not sure why CLion uses the CodeBlocks generator, but the problem is the same using the "Unix Makefiles" generator as well.

Not sure if this is helpful at all, I appreciate your guys' time and effort here.

[noloader]

Not sure if this is helpful at all...

Thanks. I think the problem is with CheckCompileOption and CheckCompileLinkOption:

function(CheckCompileLinkOption opt var prog)

    message(STATUS "Performing Test ${var}")
    execute_process(
      COMMAND sh -c "${CMAKE_CXX_COMPILER} ${CMAKE_CXX_FLAGS} ${opt} ${prog} 2>&1"
      RESULT_VARIABLE COMMAND_RESULT
      OUTPUT_VARIABLE COMMAND_OUTPUT
      OUTPUT_STRIP_TRAILING_WHITESPACE)
    ...

endfunction(CheckCompileLinkOption)

It is easy enough to confirm we are getting an incomplete set of flags by adding message(STATUS "CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS}") to the CMake function.

We use the CMake specific variables CMAKE_CXX_COMPILER and CMAKE_CXX_FLAGS. Maybe there is another CMake specific CXXFLAGS we should be using.

In any case CMake has to give us the C++ flags and options for the project. It has to provide them to us for feature testing.

(The bad thing here is, it does not matter if CMake fixes CHECK_CXX_COMPILER_FLAG. We still have to work around years of downlevel versions of the tool, like version 2.8.12 on Ubuntu 14.03 or 2.8.6 on Solaris 11.3. Users expect things to "just work" so the technological debt has been pushed onto us).

---

And to be clear, the feature testing is compiler feature testing. We need to know if a compiler supports, say, SHA instructions. So we need to test for -msha on x86, -mcpu=power8 -maltivec on PowerPC and -march=armv8-a+crypto on Aarch64.

We don't care about the actual cpu features at build time. We use that at runtime. If a cpu feature is available at runtime then we use the code that was compiled earlier. Otherwise we use other code, like straight C++ code. (Intel calls this dynamic dispatching).

Here's a typical rule from the makefile. SHA_FLAG will have the additional flags or it will be empty. All the build systems work the same way. They just do it in different ways.

sha_simd.o : sha_simd.cpp
    $(CXX) $(strip $(CXXFLAGS) $(SHA_FLAG) -c) $<

[aciereszko]

Just verified that the same error happens with the XCode cmake generator.

[mouse07410]

@noloader in my experience, usually one compiler is used fur most of the development, and other compilers/toolchains - only if/when necessary fur some reason (typically when checking portability, or trying to work around a showstopper bug in the "main" toolchain).

[aciereszko]

In my case three are useful in daily work, xcode macos toolchain for development (test app, unit tests etc) this is 80% of dev, xcode ios toolchain for actually testing the app out on device, I would say 10% of dev and the windows vs toolchain which usually gets used in spurts (when working on some particular issue in windows, or when testing) so this I would say less often than the other too on daily work, but when it does it may be for a few days at a time.

[aciereszko]

Hi, just wondering, were you able to reproduce the problem, building just vanilla cryptopp with your CMakeLists file, on a mac using xcode toolchain with the Unix Makefiles generator?

[noloader]

This is now officially CMake's problem: Issue 18813, CXX and CXXFLAGS used during feature tests are not accurate in CmakeList.txt.

When the CMake folks come back with a work around I will add it to our CmakeList.txt.