search

conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.26k stars 981 forks source link

[bug] Conan ignores CONAN_CMAKE_SYSTEM_VERSION #10756

Closed Ju1He1 closed 2 years ago

Ju1He1 commented 2 years ago

Hey guys, unfortunately on our buildserver there are lots of old Windows SDKs installed. By default CMake chooses an old SDK which is not compatible with MSCV 17 any more. I managed to get rid of this issue inside the project by setting "CMAKE_SYSTEM_VERSION" to "10.0" The build without any thirdparties works.

Now If I add the thirdparties Couchbase-Lite suffers the same problem. Conan uses the old Windows SDK when trying to build couchbase lite

My conan profile looks like this

[settings]
os=Windows
os_build=Windows
arch=x86_64
arch_build=x86_64
compiler=Visual Studio
compiler.version=17
build_type=Release
[options]
[build_requires]
[env]
CONAN_CMAKE_SYSTEM_VERSION=10.0

The recipe used to build Couchbase Lite looks like this

from pathlib import Path

from conan.tools.layout import cmake_layout
from conan.tools.cmake import CMakeToolchain, CMake
from conans import ConanFile, tools

class CouchbaseLiteCoreConan(ConanFile):
    name = "LiteCore"
    version = "2.0"
    license = "Apache 2.0"
    author = "couchbase"
    url = "https://gitlab.cds.local/third_party/couchbase-lite-core"
    description = "couchbase lite core"
    topics = ("c++", "noSQL")
    settings = "os", "compiler", "build_type", "arch"
    options = {"shared": [True, False], "fPIC": [True, False]}
    default_options = {"shared": True, "fPIC": True}

    def source(self):
        # clone main repo
        git = tools.Git("src")
        git.clone("git@gitlab.cds.local:third_party/couchbase-lite-core.git", "test")
        self.run("cd ./src/vendor && git submodule update --init --recursive")

    def config_options(self):
        if self.settings.os == "Windows":
            del self.options.fPIC

    def layout(self):
        cmake_layout(self)

    def generate(self):
        tc = CMakeToolchain(self)
        tc.generate()

    def build(self):
        cmake = CMake(self)
        cmake.configure(build_script_folder="src")
        cmake.build()

    def package(self):
        cmake = CMake(self)
        cmake.install()

    def package_info(self):
        self.cpp_info.libs = ["LiteCore"]

I'm calling conan create . -pr:b=foo to build the package

The (truncated) output looks like this


Exporting package recipe
LiteCore/2.0: A new conanfile.py version was exported
LiteCore/2.0: Folder: C:\Users\Administrator\.conan\data\LiteCore\2.0\_\_\export
LiteCore/2.0: Exported revision: ca616f2d33852284c8c2e07239395519
Configuration (profile_host):
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=17
os=Windows
os_build=Windows
[options]
[build_requires]
[env]

Configuration (profile_build):
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=17
os=Windows
os_build=Windows
[options]
[build_requires]
[env]
CONAN_CMAKE_SYSTEM_VERSION=10.0
LiteCore/2.0: Forced build from source
LiteCore/2.0 (test package): Installing package
Requirements
    LiteCore/2.0 from local cache - Cache
Packages
    LiteCore/2.0:e9a552ebe8f994398de9ceee972f0ad207df0658 - Build

Installing (downloading, building) binaries...
LiteCore/2.0: Configuring sources in C:\Users\Administrator\.conan\data\LiteCore\2.0\_\_\source
Submodule 'vendor/BLIP-Cpp' (git@gitlab.cds.local:third_party/couchbase-lite-core-blip-cpp.git) registered for path 'BLIP-Cpp'
Submodule 'vendor/SQLiteCpp' (git@gitlab.cds.local:third_party/sqlitecpp.git) registered for path 'SQLiteCpp'
Submodule 'vendor/civetweb' (https://github.com/couchbasedeps/civetweb.git) registered for path 'civetweb'
Submodule 'vendor/fleece' (git@gitlab.cds.local:third_party/couchbase-lite-core-fleece.git) registered for path 'fleece'
Submodule 'vendor/linenoise-ng' (https://github.com/arangodb/linenoise-ng) registered for path 'linenoise-ng'
Submodule 'vendor/mbedtls' (git@gitlab.cds.local:third_party/couchbase-lite-core-mbedtls.git) registered for path 'mbedtls'
Submodule 'vendor/sqlite3-unicodesn' (https://github.com/snej/sqlite3-unicodesn.git) registered for path 'sqlite3-unicodesn'
Submodule 'wiki' (https://github.com/couchbase/couchbase-lite-core.wiki.git) registered for path '../wiki'
Cloning into 'C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/source/src/vendor/BLIP-Cpp'...
Cloning into 'C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/source/src/vendor/SQLiteCpp'...
Cloning into 'C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/source/src/vendor/civetweb'...
Cloning into 'C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/source/src/vendor/fleece'...
Cloning into 'C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/source/src/vendor/linenoise-ng'...
Cloning into 'C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/source/src/vendor/mbedtls'...
Cloning into 'C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/source/src/vendor/sqlite3-unicodesn'...
Cloning into 'C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/source/src/wiki'...
Submodule path 'BLIP-Cpp': checked out 'fc00d178b504176aefb493a56af696a36c9390d8'
.....

LiteCore/2.0: Copying sources to build folder
LiteCore/2.0: Building your package in C:\Users\Administrator\.conan\data\LiteCore\2.0\_\_\build\e9a552ebe8f994398de9ceee972f0ad207df0658
LiteCore/2.0: Generator txt created conanbuildinfo.txt
LiteCore/2.0: Calling generate()
LiteCore/2.0: Aggregating env generators
LiteCore/2.0: Calling build()
LiteCore/2.0: CMake command: cmake -G "Visual Studio 17 2022" -DCMAKE_TOOLCHAIN_FILE="C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/build/e9a552ebe8f994398de9ceee972f0ad207df0658/build/conan/conan_toolchain.cmake" -DCMAKE_INSTALL_PREFIX="C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/package/e9a552ebe8f994398de9ceee972f0ad207df0658" "C:\Users\Administrator\.conan\data\LiteCore\2.0\_\_\build\e9a552ebe8f994398de9ceee972f0ad207df0658\.\src"
Using Conan toolchain: C:/Users/Administrator/.conan/data/LiteCore/2.0/_/_/build/e9a552ebe8f994398de9ceee972f0ad207df0658/build/conan/conan_toolchain.cmake.
-- Conan toolchain: Setting BUILD_SHARED_LIBS = ON
-- The C compiler identification is MSVC 19.31.31104.0
-- The CXX compiler identification is MSVC 19.31.31104.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Tools/MSVC/14.31.31103/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
cmakegeneratorisVisual Studio 17 2022
 compiler version 19.31.31104.0
....
C:\Program Files (x86)\Windows Kits\10\Include\10.0.14393.0\um\propidlbase.h(319,24): error C2059: syntax error: '/' (compiling source file C:\Users\Administrator\.conan\data\LiteCore\2.0\_\_\build\e9a552ebe8f994398de9ceee972f0ad207df0658\src\vendor\mbedtls\library\timing.c) [C:\Users\Administrator\.conan\data\LiteCore\2.0\_\_\build\e9a552ebe8f994398de9ceee972f0ad207df0658\build\vendor\mbedtls\library\mbedcrypto.vcxproj]

Now if you check the cmake -G command there is not CMAKE_SYSTEM_VERSION passed here. Should the CMAKE_SYSTEM_VERSION not being passed to the cmake generator? Or Am I missing something?

Environment Details (include every applicable attribute)

Thx for your help :)

franramirez688 commented 2 years ago

Hi @Ju1He1

I guess you should add this line in your recipe (have a look at this docs section):

class CouchbaseLiteCoreConan(ConanFile):
    # ...
    def generate(self):
        tc = CMakeToolchain(self)
        tc.variables["CMAKE_SYSTEM_VERSION"] = "10.0"
        tc.generate()

and remove the CONAN_CMAKE_SYSTEM_VERSION=10.0 line from [env] section. I think it could do the trick.

memsharded commented 2 years ago

@franramirez688 There is already some logic in the GenericSystemBlock that will set the CMAKE_SYSTEM_VERSION for some cases. Maybe it would better to change it in its context "cmake_system_version": system_version instead, to avoid possible colisions.

But not necessary, there is new conf for this:

system_name = self._conanfile.conf.get("tools.cmake.cmaketoolchain:system_name")
system_version = self._conanfile.conf.get("tools.cmake.cmaketoolchain:system_version")
system_processor = self._conanfile.conf.get("tools.cmake.cmaketoolchain:system_processor")

Use this new conf in the profile instead of hardcoding in the recipe.

Some other recommended changes:

from conan import ConanFile
from conan.tools.cmake import CMakeToolchain, CMake, cmake_layout
from conans import tools  # This one has also other new tools like ``conan.tools.scm import Git``
Ju1He1 commented 2 years ago

Thank you both for your fast answer :)

@franramirez688 your solution worked. Couchbase build properly :) However Windows is just one config out of many. I'd like to keep my recipe as generic as possible. That's why we are using build profiles ;)

@memsharded
I did add 

[options]
system_version=10.0

to my build profile and replaced the imports with your suggestions :) However this does not seam to have any effect. Our build server are still running conan 1.45. Do i have to upgrade conan?

Do I have to explicitly set system_version = self._conanfile.conf.get("tools.cmake.cmaketoolchain:system_version")somewhere in my recipe?

memsharded commented 2 years ago

Not, it is not an option, is a "conf". In the profile:

[conf]
tools.cmake.cmaketoolchain:system_version=10.0
Ju1He1 commented 2 years ago

ups yeah you are right. I fixed that. But still it does not seam to take effect

$ conan create . -pr:b=foo
Exporting package recipe
LiteCore/2.0: A new conanfile.py version was exported
LiteCore/2.0: Folder: C:\Users\Administrator\.conan\data\LiteCore\2.0\_\_\export
LiteCore/2.0: Exported revision: efb74f9e1cdd536c4136dc628621ca13
Configuration (profile_host):
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=17
os=Windows
os_build=Windows
[options]
[build_requires]
[env]

Configuration (profile_build):
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=17
os=Windows
os_build=Windows
[options]
[build_requires]
[env]
[conf]
tools.cmake.cmaketoolchain:system_version=10.0

do I have to modify my recipe as well? Sorry I'm a little bit stuck here :O

memsharded commented 2 years ago

You are specifying it in the build profile only, but it should be done in the host profile, not in the build one.

Ju1He1 commented 2 years ago

thank you James. That did work :)