[bug] Cannot build on MSYS2 Clang64 ("Visual Studio Runtime version not defined")

stevenwdv commented 11 months ago

Environment details

Operating System+version: Windows 11
Compiler+version: mingw-w64-clang-x86_64-clang 17.0.2
Conan version: 2.0.14
Python version: 3.11.6

Steps to reproduce

Install MSYS2 with mingw-w64-clang-x86_64-clang and mingw-w64-clang-x86_64-python-conan
Create a Conan profile like here (customizing msys64 paths)
Add a Conanfile to a project, requiring e.g. zlib/1.2.11

Run Conan install:

conan install ./ --output-folder=./build/ --build=missing

Get unexpected error:

ERROR: zlib/1.2.11: Error in generate() method, line 63
    tc.generate()
    ConanException: Visual Studio Runtime version (v140-v143) not defined

Similar with Conan installed via pip on Windows:

conanfile.txt: ERROR: Traceback (most recent call last):
  File "C:\Users\Steven\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\conans\client\generators\__init__.py", line 94, in write_generators
    generator.generate()
  File "C:\Users\Steven\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\conan\tools\cmake\toolchain\toolchain.py", line 197, in generate
    VCVars(self._conanfile).generate()
  File "C:\Users\Steven\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\conan\tools\microsoft\visual.py", line 125, in generate
    raise ConanException("Visual Studio Runtime version (v140-v143) not defined")
conans.errors.ConanException: Visual Studio Runtime version (v140-v143) not defined

ERROR: Error in generator 'CMakeToolchain': Visual Studio Runtime version (v140-v143) not defined

Why is it trying to use Visual Studio? I specified clang as compiler.

Logs

$ conan install ./ --output-folder=./cmake-build-debug/ --build=missing

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Debug
compiler=clang
compiler.cppstd=20
compiler.libcxx=libc++
compiler.runtime=dynamic
compiler.runtime_type=Debug
compiler.version=17
os=Windows
[buildenv]
PATH+=(path)C:/msys64/clang64/bin
[runenv]
PATH+=(path)C:/msys64/clang64/bin

Profile build:
[settings]
arch=x86_64
build_type=Debug
compiler=clang
compiler.cppstd=20
compiler.libcxx=libc++
compiler.runtime=dynamic
compiler.runtime_type=Debug
compiler.version=17
os=Windows
[buildenv]
PATH+=(path)C:/msys64/clang64/bin
[runenv]
PATH+=(path)C:/msys64/clang64/bin

======== Computing dependency graph ========
Graph root
    conanfile.py (pep-core/0.0.0): C:/dev/pep/core/conanfile.py
Requirements
    zlib/1.2.11#dc0e384f0551386cd76dc29cc964c95e - Cache
Build requirements
    cmake/3.22.6#32cced101c6df0fab43e8d00bd2483eb - Cache

======== Computing necessary packages ========
Requirements
    zlib/1.2.11#dc0e384f0551386cd76dc29cc964c95e:5cea60b7b9935e77d401ba90fcc99e975259faf5 - Build
Build requirements
    cmake/3.22.6#32cced101c6df0fab43e8d00bd2483eb:522dcea5982a3f8a5b624c16477e47195da2f84f#a9d024f459972755e5a815b775408fff - Cache

======== Installing packages ========
cmake/3.22.6: Already installed! (1 of 2)
cmake/3.22.6: Appending PATH environment variable: C:/Users/Steven/.conan2/p/cmake1927748c2604b/p/bin

-------- Installing package zlib/1.2.11 (2 of 2) --------
zlib/1.2.11: Building from source
zlib/1.2.11: Package zlib/1.2.11:5cea60b7b9935e77d401ba90fcc99e975259faf5
zlib/1.2.11: Copying sources to build folder
zlib/1.2.11: Building your package in C:/Users/Steven/.conan2/p/b/zlib41869a87a668a/b
zlib/1.2.11: Calling generate()
zlib/1.2.11: Generators folder: C:/Users/Steven/.conan2/p/b/zlib41869a87a668a/b/build/Debug/generators
zlib/1.2.11: CMakeToolchain generated: conan_toolchain.cmake
ERROR: zlib/1.2.11: Error in generate() method, line 63
        tc.generate()
        ConanException: Visual Studio Runtime version (v140-v143) not defined

memsharded commented 11 months ago

Hi @stevenwdv

Thanks for your question. I am not sure this is a bug yet, I'd need some clarifications.

You are defining in your profile compiler.runtime=dynamic. That means that it expects to use the LLVM/Clang that has Visual Studio as backend, and using the VS runtime. When doing it, then Conan will also expect to get defined a runtime_version, to select which VS runtime will be selected

If you want to use msys2 clang, with the libc++.dll runtime, then:

Do not define compiler.runtime nor compiler.runtime_type at all
Define compiler.libcxx

Please try that and let us know.

memsharded commented 10 months ago

Hi @stevenwdv

Any update here? Did you try the above? Thanks

stevenwdv commented 10 months ago

Thanks, that works a lot better! I now also explicitly specified tools.build:compiler_executables={'c': 'clang', 'cpp': 'clang++'} to be sure it uses clang, since it autodetected gcc on another install, but Boost does not seem to like this, should I file an issue for this?

MSYS2 Clang64$ conan install ./ --build=missing

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Debug
compiler=clang
compiler.cppstd=20
compiler.libcxx=libc++
compiler.version=17
os=Windows
[conf]
tools.build:compiler_executables={'c': 'clang', 'cpp': 'clang++'}
tools.cmake.cmaketoolchain:generator=Ninja
[buildenv]
PATH+=(path)C:/msys64/clang64/bin
[runenv]
PATH+=(path)C:/msys64/clang64/bin

Profile build:
[settings]
arch=x86_64
build_type=Debug
compiler=clang
compiler.cppstd=20
compiler.libcxx=libc++
compiler.version=17
os=Windows
[conf]
tools.build:compiler_executables={'c': 'clang', 'cpp': 'clang++'}
tools.cmake.cmaketoolchain:generator=Ninja
[buildenv]
PATH+=(path)C:/msys64/clang64/bin
[runenv]
PATH+=(path)C:/msys64/clang64/bin

======== Computing dependency graph ========
Graph root
    conanfile.txt: C:/Users/Steven/Downloads/tmp/conanfile.txt
Requirements
    boost/1.83.0#7825569cd0622461dec7bc87dfdf47ae - Cache
    bzip2/1.0.8#411fc05e80d47a89045edc1ee6f23c1d - Cache
    zlib/1.2.11#dc0e384f0551386cd76dc29cc964c95e - Cache
Build requirements
    b2/4.10.1#8dc3df1cc73ad65d86cbdfd31fdb011f - Cache
Resolved version ranges
    zlib/[>=1.2.11 <2]: zlib/1.2.11

======== Computing necessary packages ========
boost/1.83.0: Checking 9 compatible configurations
boost/1.83.0: Compatible configurations not found in cache, checking servers
boost/1.83.0: '55fcd6e57a0796b615bf8a9391e08ce069a72d3f': compiler.cppstd=11
boost/1.83.0: '6168d70981843148b3047c2dc2038c6a5d61e78f': compiler.cppstd=gnu11
boost/1.83.0: 'e226ae5fc30a27c1f386e2d3a20028b1216eb594': compiler.cppstd=14
boost/1.83.0: 'be1d59c644a38bff7ccfb8d88a592c6263a7c326': compiler.cppstd=gnu14
boost/1.83.0: 'e287a6f9f4f8425d7a217361b15067327ada8d32': compiler.cppstd=17
boost/1.83.0: 'ee54f0a1e16ef712b4ef36b3bdb54061f6a14f39': compiler.cppstd=gnu17
boost/1.83.0: 'c853c5311065a725a3b5585b61ee5c011adb75a7': compiler.cppstd=gnu20
boost/1.83.0: '609dd59a38ddbbacc5e4938df4a27bae8607e413': compiler.cppstd=23
boost/1.83.0: '610ca76f6d392a171eb698fad7af427c6631d582': compiler.cppstd=gnu23
Requirements
    boost/1.83.0#7825569cd0622461dec7bc87dfdf47ae:ba221cec0ce65c8ad129ac8da1f21a8347576e27 - Build
    bzip2/1.0.8#411fc05e80d47a89045edc1ee6f23c1d:dcb9d912e86b265b719a0333789b267662a80737#62eaf3a33bea695456953f654fcda533 - Cache
    zlib/1.2.11#dc0e384f0551386cd76dc29cc964c95e:f378def36594de66a7a3a7e0176ff4a7658c9e85#5aeee1f445976c8ac2164ce1a74292da - Cache
Build requirements
    b2/4.10.1#8dc3df1cc73ad65d86cbdfd31fdb011f:522dcea5982a3f8a5b624c16477e47195da2f84f#ea9c889ff5a1e61ab2ca8bc15105f439 - Cache

======== Installing packages ========
b2/4.10.1: Already installed! (1 of 4)
bzip2/1.0.8: Already installed! (2 of 4)
zlib/1.2.11: Already installed! (3 of 4)

-------- Installing package boost/1.83.0 (4 of 4) --------
boost/1.83.0: Building from source
boost/1.83.0: Package boost/1.83.0:ba221cec0ce65c8ad129ac8da1f21a8347576e27
boost/1.83.0: Building your package in C:/Users/Steven/.conan2/p/b/boost51e814acd9f66/b
boost/1.83.0: Calling generate()
boost/1.83.0: Generators folder: C:/Users/Steven/.conan2/p/b/boost51e814acd9f66/b/build-debug/conan
boost/1.83.0: Generating aggregated env files
boost/1.83.0: Generated aggregated env files: ['conanbuild.bat', 'conanrun.bat']
boost/1.83.0: Calling build()
boost/1.83.0: WARN: replace_in_file didn't find pattern '/* thread_local */' in 'C:/Users/Steven/.conan2/p/boost1d9916eed0e5d/s/src/boost/stacktrace/detail/libbacktrace_impls.hpp' file.
boost/1.83.0: WARN: replace_in_file didn't find pattern '/* static __thread */' in 'C:/Users/Steven/.conan2/p/boost1d9916eed0e5d/s/src/boost/stacktrace/detail/libbacktrace_impls.hpp' file.
boost/1.83.0: WARN: replace_in_file didn't find pattern 'local generic-os = [ set.difference $(all-os) : aix darwin vxworks solaris osf hpux ] ;' in 'C:/Users/Steven/.conan2/p/boost1d9916eed0e5d/s/src/tools/build/src/tools/gcc.jam' file.
boost/1.83.0: WARN: replace_in_file didn't find pattern 'local no-threading = android beos haiku sgi darwin vxworks ;' in 'C:/Users/Steven/.conan2/p/boost1d9916eed0e5d/s/src/tools/build/src/tools/gcc.jam' file.
boost/1.83.0: WARN: replace_in_file didn't find pattern '    <conditional>@numa' in 'C:/Users/Steven/.conan2/p/boost1d9916eed0e5d/s/src/libs/fiber/build/Jamfile.v2' file.
boost/1.83.0: WARN: Patching user-config.jam
boost/1.83.0: WARN:
using zlib : 1.2.11 : <include>"C:/Users/Steven/.conan2/p/b/zlib5238a1d2a45db/p/include" <search>"C:/Users/Steven/.conan2/p/b/zlib5238a1d2a45db/p/lib" <name>zlib ;
using bzip2 : 1.0.8 : <include>"C:/Users/Steven/.conan2/p/b/bzip21e1b8c0e0f52e/p/include" <search>"C:/Users/Steven/.conan2/p/b/bzip21e1b8c0e0f52e/p/lib" <name>bz2 ;
using "clang-win" :  :  "clang++" :
 ;
boost/1.83.0: WARN: b2 -q numa=on target-os=windows architecture=x86 address-model=64 binary-format=pe abi=ms --layout=system --user-config=C:/Users/Steven/.conan2/p/boost1d9916eed0e5d/s/src/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=off --disable-iconv threading=multi visibility=hidden link=static variant=debug --with-atomic --with-chrono --with-container --with-context --with-contract --with-coroutine --with-date_time --with-exception --with-fiber --with-filesystem --with-graph --with-iostreams --with-json --with-locale --with-log --with-math --with-nowide --with-program_options --with-random --with-regex --with-serialization --with-stacktrace --with-system --with-test --with-thread --with-timer --with-type_erasure --with-url --with-wave toolset=clang-win cxxflags=-std=c++20 pch=on linkflags="-stdlib=libc++" cxxflags="-fPIC -stdlib=libc++" install --prefix=C:/Users/Steven/.conan2/p/b/boost51e814acd9f66/p -j24 --abbreviate-paths -d0 --debug-configuration --build-dir="C:/Users/Steven/.conan2/p/b/boost51e814acd9f66/b/build-debug"
boost/1.83.0: RUN: b2 -q numa=on target-os=windows architecture=x86 address-model=64 binary-format=pe abi=ms --layout=system --user-config=C:/Users/Steven/.conan2/p/boost1d9916eed0e5d/s/src/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=off --disable-iconv threading=multi visibility=hidden link=static variant=debug --with-atomic --with-chrono --with-container --with-context --with-contract --with-coroutine --with-date_time --with-exception --with-fiber --with-filesystem --with-graph --with-iostreams --with-json --with-locale --with-log --with-math --with-nowide --with-program_options --with-random --with-regex --with-serialization --with-stacktrace --with-system --with-test --with-thread --with-timer --with-type_erasure --with-url --with-wave toolset=clang-win cxxflags=-std=c++20 pch=on linkflags="-stdlib=libc++" cxxflags="-fPIC -stdlib=libc++" install --prefix=C:/Users/Steven/.conan2/p/b/boost51e814acd9f66/p -j24 --abbreviate-paths -d0 --debug-configuration --build-dir="C:/Users/Steven/.conan2/p/b/boost51e814acd9f66/b/build-debug"
notice: found boost-build.jam at C:/Users/Steven/.conan2/p/boost1d9916eed0e5d/s/src/boost-build.jam
notice: loading B2 from C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/kernel/bootstrap.jam
notice: Searching 'C:\Users\Steven' 'C:\msys64\home\Steven' 'C:\Users\Steven' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/kernel' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/util' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/build' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/tools' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/contrib' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/.' for site-config configuration file 'site-config.jam'.
notice: Configuration file 'site-config.jam' not found in 'C:\Users\Steven' 'C:\msys64\home\Steven' 'C:\Users\Steven' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/kernel' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/util' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/build' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/tools' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/contrib' 'C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/.'.
notice: Loading explicitly specified user configuration file:
    C:\Users\Steven\.conan2\p\boost1d9916eed0e5d\s\src\tools\build\user-config.jam
notice: Searching 'C:\Users\Steven\.conan2\p\boost1d9916eed0e5d\s\src\tools\build' for user-config configuration file 'user-config.jam'.
notice: Loading user-config configuration file 'user-config.jam' from 'C:/Users/Steven/.conan2/p/boost1d9916eed0e5d/s/src/tools/build'.
notice: [zlib] Using pre-installed library
notice: [zlib] Condition
notice: [bzip2] Using pre-installed library
notice: [bzip2] Condition
notice: [msvc-cfg] msvc-14.3 detected, command: 'C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Tools\MSVC\14.39.33321\bin\Hostx64\arm\cl.exe'
warning: toolset clang-win initialization:
warning: can not find user-provided command 'clang++'
warning: initialized from C:\Users\Steven\.conan2\p\boost1d9916eed0e5d\s\src\tools\build\user-config.jam:5
C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/tools\clang-win.jam:70: in clang-win.init from module clang-win
error: Cannot configure toolset clang-win: no 'clang-cl.exe' command found or given
C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/build\toolset.jam:44: in toolset.using from module toolset
C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/build\project.jam:1260: in using from module project-rules
C:\Users\Steven\.conan2\p\boost1d9916eed0e5d\s\src\tools\build\user-config.jam:5: in modules.load from module user-config
C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2\build-system.jam:255: in load-config from module build-system
C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2\build-system.jam:449: in load-configuration-files from module build-system
C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2\build-system.jam:607: in load from module build-system
C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/kernel\modules.jam:294: in import from module modules
C:/Users/Steven/.conan2/p/b283b39a732024d/p/bin/.b2/kernel/bootstrap.jam:135: in module scope from module

boost/1.83.0: ERROR:
Package 'ba221cec0ce65c8ad129ac8da1f21a8347576e27' build failed
boost/1.83.0: WARN: Build folder C:/Users/Steven/.conan2/p/b/boost51e814acd9f66/b/build-debug
*********************************************************
Recipe 'boost/1.83.0' cannot build its binary
It is possible that this recipe is not Conan 2.0 ready
If the recipe comes from ConanCenter, report it at https://github.com/conan-io/conan-center-index/issues
If it is your recipe, check if it is updated to 2.0
*********************************************************

ERROR: boost/1.83.0: Error in build() method, line 887
        self.run(full_command)
        ConanException: Error 1 while executing

memsharded commented 10 months ago

Yes, it is possible that the boost recipe in ConanCenter is not following the compiler_executables configuration yet. If the other packages work correctly, then it is likely it could be a recipe issue, not a Conan issue. As boost is using its own build system, it might require some changes directly in the recipe, so this probably should be reported to conan-center-index github repo, where all the ConanCenter recipes are. I am not sure how it should be done, because as it is not relying on an absolute path for clang, it might have other ways to do it.

The problem is that such repo is extremely busy (like 5000 PRs per year), so these things might take some time to get attention there, specially now while finishing the migration to Conan 2.0, there is still a long backlog of PRs to process, so you might need some patience.

stevenwdv commented 10 months ago

it is possible that the boost recipe in ConanCenter is not following the compiler_executables configuration yet

It does follow the config, it just cannot find clang++ without an absolute path; without compiler_executables it compiles fine.

memsharded commented 10 months ago

I see why:

@property
    def _cxx(self):
        compilers_by_conf = self.conf.get("tools.build:compiler_executables", default={}, check_type=dict)
        cxx = compilers_by_conf.get("cpp") or VirtualBuildEnv(self).vars().get("CXX")
        if cxx:
            return cxx
        if is_apple_os(self) and self.settings.compiler == "apple-clang":
            return XCRun(self).cxx
        compiler_version = str(self.settings.compiler.version)
        major = compiler_version.split(".", maxsplit=1)[0]
        if self.settings.compiler == "gcc":
            return shutil.which(f"g++-{compiler_version}") or shutil.which(f"g++-{major}") or shutil.which("g++") or ""
        if self.settings.compiler == "clang":
            return shutil.which(f"clang++-{compiler_version}") or shutil.which(f"clang++-{major}") or shutil.which("clang++") or ""
        return ""

The recipe will search for an absolute path of the compiler with shutil.which if not defined by compiler_executables, but if defined by compiler_executables it assumes it already comes with a full path. The recipe might benefit from checking it and doing also a shutil.which for not absolute paths, but I guess it is just easier to define the full path in your profile?

memsharded commented 8 months ago

Hi @stevenwdv

Did the above help? Any further question or maybe this ticket can be closed?

stevenwdv commented 6 months ago

Did the above help?

I guess, but would be nice if it could be fixed, I'll create a separate issue.

Also, there seem to be multiple other issues building under MinGW, which I'll try to report

conan-io / conan