search

conan-io / conan

Conan - The open-source C and C++ package manager
https://conan.io
MIT License
8.17k stars 974 forks source link

Improve, simplify VS Visual Studio detection #1683

Closed memsharded closed 6 years ago

memsharded commented 7 years ago
piponazo commented 7 years ago

I just tried to use 0.26.0rc1 (having only installed in my system the Visual Studio 2017 Builds Tools update 3) and I could not compile a simple project:

Expat/2.2.1-1@pix4d/testing: Calling build()

-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
CMake Error at CMakeLists.txt:4 (project):
  No CMAKE_C_COMPILER could be found.

CMake Error at CMakeLists.txt:4 (project):
  No CMAKE_CXX_COMPILER could be found.

-- Configuring incomplete, errors occurred!
See also "C:/Users/Luis/.conan/data/Expat/2.2.1-1/pix4d/testing/build/970e773c5651dc2560f86200a4ea56c23f568ff9/build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/Luis/.conan/data/Expat/2.2.1-1/pix4d/testing/build/970e773c5651dc2560f86200a4ea56c23f568ff9/build/CMakeFiles/CMakeError.log".
Expat/2.2.1-1@pix4d/testing: 

Expat/2.2.1-1@pix4d/testing: ERROR: Package '970e773c5651dc2560f86200a4ea56c23f568ff9' build failed

Expat/2.2.1-1@pix4d/testing: WARN: Build folder C:\Users\Luis\.conan\data\Expat\2.2.1-1\pix4d\testing\build\970e773c5651dc2560f86200a4ea56c23f568ff9

ERROR: Expat/2.2.1-1@pix4d/testing: Error in build() method, line 34

    cmake.configure(source_dir='../libexpat/expat', build_dir='build', defs=cmake_args)

    ConanException: Error 1 while executing cd build && cmake -G "Visual Studio 15 2017 Win64" -DCONAN_LINK_RUNTIME="/MD" -DCONAN_EXPORTED="1" -DCONAN_COMPILER="Visual Studio" -DCONAN_COMPILER_VERSION="15" -DBUILD_SHARED_LIBS="ON" -DCMAKE_INSTALL_PREFIX="C:\Users\Luis\.conan\data\Expat\2.2.1-1\pix4d\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9" -DCONAN_CXX_FLAGS="/MP12" -DCONAN_C_FLAGS="/MP12" -Wno-dev -DCMAKE_INSTALL_PREFIX="C:\Users\Luis\.conan\data\Expat\2.2.1-1\pix4d\testing\package\970e773c5651dc2560f86200a4ea56c23f568ff9" -DBUILD_tests="OFF" -DBUILD_shared="True" -DBUILD_examples="OFF" -DBUILD_doc="OFF" -DBUILD_tools="OFF" ../libexpat/expat
memsharded commented 7 years ago

Did you just install VS2017? Installation of VS2017 messes with previous installations of VS2015, and requires to install also the Win10 SDK in order to be correctly detected by cmake.

piponazo commented 7 years ago

Yes, just the vs build tools (Note that there is an individual installer for that). Being more specific I run the installer like this:

vs_buildtools__1057011630.1503647474 --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Workload.VCTools

You can find more information about those argument in this link.

I will investigate a bit more this issue.

SSE4 commented 7 years ago

@piponazo can you share pls these two files: See also "C:/Users/Luis/.conan/data/Expat/2.2.1-/pix4d/testing/build/970e773c5651dc2560f86200a4ea56c23f568ff9/build/CMakeFiles/CMakeOutput.log". See also "C:/Users/Luis/.conan/data/Expat/2.2.1-/pix4d/testing/build/970e773c5651dc2560f86200a4ea56c23f568ff9/build/CMakeFiles/CMakeError.log".

memsharded commented 7 years ago

The typical error post VS2017 install is rc.exe missing (not only in VS2017, but it also removes it from VS2015!!) Some answers in StackOverflow suggest manually copying rc.exe and rc.dll to other folders, but I think the way to go is to re-install the Win SDK.

piponazo commented 7 years ago

Sure, these are the CMakeError.log and CMakeOutput.log files without calling the batch file C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvars64.bat

#CMakeError.log
Compiling the C compiler identification source file "CMakeCCompilerId.c" failed.
Compiler:  
Build flags: 
Id flags: 

The output was:
The system cannot find the file specified

Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler:  
Build flags: 
Id flags: 

The output was:
The system cannot find the file specified

and 

#CMakeOutput.log
The system is: Windows - 10.0.15063 - AMD64

If I call C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvars64.bat before running the conan create command it builds everything perfectly:

#CMakeError.log
Determining if the include file dlfcn.h exists failed with the following output:
Change Dir: C:/Users/Luis/.conan/data/Expat/2.2.1/pix4d/testing/build/6cc50b139b9c3d27b3e9042d5f5372d327b3a9f7/build/CMakeFiles/CMakeTmp

Run Build Command:"MSBuild.exe" "cmTC_c6030.vcxproj" "/p:Configuration=Debug" "/p:VisualStudioVersion=15.0"
Microsoft (R) Build Engine version 15.3.409.57025 for .NET Framework

Copyright (C) Microsoft Corporation. All rights reserved.

Build started 30/08/2017 16:42:27.

Project "C:\Users\Luis\.conan\data\Expat\2.2.1\pix4d\testing\build\6cc50b139b9c3d27b3e9042d5f5372d327b3a9f7\build\CMakeFiles\CMakeTmp\cmTC_c6030.vcxproj" on node 1 (default targets).

PrepareForBuild:

  Creating directory "cmTC_c6030.dir\Debug\".

  Creating directory "C:\Users\Luis\.conan\data\Expat\2.2.1\pix4d\testing\build\6cc50b139b9c3d27b3e9042d5f5372d327b3a9f7\build\CMakeFiles\CMakeTmp\Debug\".

  Creating directory "cmTC_c6030.dir\Debug\cmTC_c6030.tlog\".

InitializeBuildStatus:

  Creating "cmTC_c6030.dir\Debug\cmTC_c6030.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.

ClCompile:

  C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.11.25503\bin\HostX86\x64\CL.exe /c /Zi /W3 /WX- /diagnostics:classic /MP8 /Od /Ob0 /D WIN32 /D _WINDOWS /D _DEBUG /D "CMAKE_INTDIR=\"Debug\"" /D _MBCS /Gm- /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"cmTC_c6030.dir\Debug\\" /Fd"cmTC_c6030.dir\Debug\vc141.pdb" /Gd /TC /errorReport:queue C:\Users\Luis\.conan\data\Expat\2.2.1\pix4d\testing\build\6cc50b139b9c3d27b3e9042d5f5372d327b3a9f7\build\CMakeFiles\CMakeTmp\CheckIncludeFile.c

  Microsoft (R) C/C++ Optimizing Compiler Version 19.11.25507.1 for x64

  Copyright (C) Microsoft Corporation.  All rights reserved.

  cl /c /Zi /W3 /WX- /diagnostics:classic /MP8 /Od /Ob0 /D WIN32 /D _WINDOWS /D _DEBUG /D "CMAKE_INTDIR=\"Debug\"" /D _MBCS /Gm- /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"cmTC_c6030.dir\Debug\\" /Fd"cmTC_c6030.dir\Debug\vc141.pdb" /Gd /TC /errorReport:queue C:\Users\Luis\.conan\data\Expat\2.2.1\pix4d\testing\build\6cc50b139b9c3d27b3e9042d5f5372d327b3a9f7\build\CMakeFiles\CMakeTmp\CheckIncludeFile.c

  CheckIncludeFile.c
...

and 

#CMakeOutput.log
The system is: Windows - 10.0.15063 - AMD64
Compiling the C compiler identification source file "CMakeCCompilerId.c" succeeded.
Compiler:  
Build flags: 
Id flags: 

The output was:
0
Microsoft (R) Build Engine version 15.3.409.57025 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

Build started 30/08/2017 16:42:15.
Project "C:\Users\Luis\.conan\data\Expat\2.2.1\pix4d\testing\build\6cc50b139b9c3d27b3e9042d5f5372d327b3a9f7\build\CMakeFiles\3.7.2\CompilerIdC\CompilerIdC.vcxproj" on node 1 (default targets).
PrepareForBuild:
  Creating directory "Debug\".
  Creating directory "Debug\CompilerIdC.tlog\".
InitializeBuildStatus:
  Creating "Debug\CompilerIdC.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
ClCompile:
  C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.11.25503\bin\HostX86\x64\CL.exe /c /nologo /W0 /WX- /diagnostics:classic /Od /D _MBCS /Gm- /EHsc /RTC1 /MDd /GS /fp:precise /Zc:wchar_t /Zc:forScope /Zc:inline /Fo"Debug\\" /Fd"Debug\vc141.pdb" /Gd /TC /errorReport:queue CMakeCCompilerId.c
  CMakeCCompilerId.c
Link:
  C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Tools\MSVC\14.11.25503\bin\HostX86\x64\link.exe /ERRORREPORT:QUEUE /OUT:".\CompilerIdC.exe" /INCREMENTAL:NO /NOLOGO kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib /MANIFEST /MANIFESTUAC:"level='asInvoker' uiAccess='false'" /manifest:embed /PDB:".\CompilerIdC.pdb" /SUBSYSTEM:CONSOLE /TLBID:1 /DYNAMICBASE /NXCOMPAT /IMPLIB:".\CompilerIdC.lib" /MACHINE:X64 Debug\CMakeCCompilerId.obj
  CompilerIdC.vcxproj -> C:\Users\Luis\.conan\data\Expat\2.2.1\pix4d\testing\build\6cc50b139b9c3d27b3e9042d5f5372d327b3a9f7\build\CMakeFiles\3.7.2\CompilerIdC\.\CompilerIdC.exe
...
piponazo commented 7 years ago

Note that the command vswhere does not seem to be in my $PATH. I did not look at your python code ... maybe you are assuming that the command is accessible through the $PATH or is always in the same path ?

memsharded commented 7 years ago

vswhere path is hardcoded in ProgramFiles, as by Microsoft instructions. Conan code just uses this hardcoded path to locate it. All of this is a terrible mistake IMHO

piponazo commented 7 years ago

can you point me to these instructions please? We are currently in the migration process from Visual Studio 2013 to 2017 and I am trying to learn as much as possible from this new situation.

SSE4 commented 7 years ago

@piponazo https://github.com/Microsoft/vswhere could you share conanfile.py of expat recipe as well? does it invoke VisualStudioBuildEnvironment?

piponazo commented 7 years ago

Here you can find the complete recipe: https://github.com/Pix4D/conan-expat

I am not invoking VisualStudioBuildEnvironment , I am just using the CMake generator. Is it needed to use it now ?

SSE4 commented 7 years ago

probably not, usually CMake finds Visual Studio itself without any problems and outside of VS environment. can you tell me as well which CMake version are you using? there was a similar issue prior to CMake 3.8 https://gitlab.kitware.com/cmake/cmake/issues/16732

piponazo commented 7 years ago

I am using actually 3.7.2 in this computer, but at work we use 3.8.1. I can try to update and repeat the process ;)

memsharded commented 7 years ago

No, CMake 3.7 was broken as hell for me: cmake -G "Visual Studio 14" wouldn't work at all, even outside conan, because of the missing rc.exe issue. So shouldn't be related to VisualStudioEnvironment at all. Can you try just to build a simple CMakeLists.txt project, outside of conan, just to make sure?

piponazo commented 7 years ago

As @memsharded was pointing out, it seems a problem with the Windows10 SDK.

I tried to repeat the steps with VS2017 Community (I uninstalled first the Build Tools). This is the command line that I used to install it:

vs_Community.exe --add Microsoft.VisualStudio.Workload.NativeDesktop --includeRecommended --lang en-US

Then I opened a new terminal, run conan create ... and it worked perfectly. The trick is in the --includeRecommended argument. Here we can check that Microsoft.VisualStudio.Component.Windows10SDK.15063.Desktop is a recommended package for Microsoft.VisualStudio.Workload.NativeDesktop. It is exactly the same case for the Build tools , but I was not using the --includeRecommended in that case.

I am uninstalling the community version and installing again the Build Tools to check if this solves my situation.

piponazo commented 7 years ago

Confirmed! Installing the Visual Studio Build Tools like this:

vs_buildtools.exe --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Workload.VCTools --includeRecommended

Solves the issue. Conan 0.26.0rc1 finds correctly the tools and compile my packages.

Thanks guys! I hope this can be useful for other people having the same problem.

lilith commented 7 years ago

I'm running into this, but I can't get it working on AppVeyor Tried running 

SET PATH=%PATH%;C:\MinGW\bin
conan install --scope build_tests=True -s compiler="Visual Studio" -o shared=True --build missing -s build_type=Release -s arch=%CONAN_ARCH% -s target_cpu=%TARGET_CPU% -u ../
PROJECT: WARN: config() has been deprecated. Use config_options and configure
libjpeg-turbo/1.5.2@nathanaeljones/testing: Not found in local cache, looking in remotes...
libjpeg-turbo/1.5.2@nathanaeljones/testing: Trying with 'conan-center'...
libjpeg-turbo/1.5.2@nathanaeljones/testing: Trying with 'conan-transit'...
libjpeg-turbo/1.5.2@nathanaeljones/testing: Trying with 'imageflow'...
Downloading conanmanifest.txt
Downloading conanfile.py
PROJECT: Installing C:\projects\imageflow\c_components
Requirements
    catch/1.3.0@TyRoXx/stable from conan-transit
    libjpeg-turbo/1.5.2@nathanaeljones/testing from imageflow
    libpng/1.6.21@lasote/stable from conan-transit
    littlecms/2.7@lasote/stable from conan-transit
    nasm/2.12.02@lasote/stable from conan-transit
    zlib/1.2.8@lasote/stable from conan-transit
Packages
    catch/1.3.0@TyRoXx/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9
    libjpeg-turbo/1.5.2@nathanaeljones/testing:9fa351ac1b30ceabd1b27310408605a456b5fd86
    libpng/1.6.21@lasote/stable:ee5dab950b6ea94ae953b93c55343e98c2ec39df
    littlecms/2.7@lasote/stable:63da998e3642b50bee33f4449826b2d623661505
    nasm/2.12.02@lasote/stable:ca33edce272a279b24f87dc0d4cf5bbdcffbc187
    zlib/1.2.8@lasote/stable:63da998e3642b50bee33f4449826b2d623661505
catch/1.3.0@TyRoXx/stable: Already installed!
littlecms/2.7@lasote/stable: Already installed!
nasm/2.12.02@lasote/stable: Already installed!
zlib/1.2.8@lasote/stable: Already installed!
libjpeg-turbo/1.5.2@nathanaeljones/testing: Building your package in C:\Users\appveyor\.conan\data\libjpeg-turbo\1.5.2\nathanaeljones\testing\build\9fa351ac1b30ceabd1b27310408605a456b5fd86
libjpeg-turbo/1.5.2@nathanaeljones/testing: Configuring sources in C:\Users\appveyor\.conan\data\libjpeg-turbo\1.5.2\nathanaeljones\testing\source
libjpeg-turbo/1.5.2@nathanaeljones/testing: Copying sources to build folder
libjpeg-turbo/1.5.2@nathanaeljones/testing: Generator cmake created conanbuildinfo.cmake
libjpeg-turbo/1.5.2@nathanaeljones/testing: Generator txt created conanbuildinfo.txt
libjpeg-turbo/1.5.2@nathanaeljones/testing: Calling build()
libjpeg-turbo/1.5.2@nathanaeljones/testing: WARN: 
***********************************************************************
    WARNING!!!
    ConfigureEnvironment class is deprecated and will be removed soon.
    With ConfigureEnvironment, env variables from profiles and/or
    command line are not applied.
    Replace it with the right one according your needs:
      - AutoToolsBuildEnvironment
      - VisualStudioBuildEnvironment
    Check docs.conan.io
***********************************************************************

-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
CMake Error at CMakeLists.txt:5 (project):
  No CMAKE_C_COMPILER could be found.
CMake Error at CMakeLists.txt:5 (project):
  No CMAKE_CXX_COMPILER could be found.
-- Configuring incomplete, errors occurred!
See also "C:/Users/appveyor/.conan/data/libjpeg-turbo/1.5.2/nathanaeljones/testing/build/9fa351ac1b30ceabd1b27310408605a456b5fd86/libjpeg-turbo-a257900dfd4b49c6742c7e135801e77a5fb3324c/_build/CMakeFiles/CMakeOutput.log".
See also "C:/Users/appveyor/.conan/data/libjpeg-turbo/1.5.2/nathanaeljones/testing/build/9fa351ac1b30ceabd1b27310408605a456b5fd86/libjpeg-turbo-a257900dfd4b49c6742c7e135801e77a5fb3324c/_build/CMakeFiles/CMakeError.log".
PROJECT: 
libjpeg-turbo/1.5.2@nathanaeljones/testing: ERROR: Package '9fa351ac1b30ceabd1b27310408605a456b5fd86' build failed
libjpeg-turbo/1.5.2@nathanaeljones/testing: WARN: Build folder C:\Users\appveyor\.conan\data\libjpeg-turbo\1.5.2\nathanaeljones\testing\build\9fa351ac1b30ceabd1b27310408605a456b5fd86
ERROR: libjpeg-turbo/1.5.2@nathanaeljones/testing: Error in build() method, line 90
    self.run('%s && %s && cmake .. %s %s' % (env.command_line, cd_build, cmake.command_line, " ".join(cmake_options)))
    ConanException: Error 1 while executing call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\../../VC/vcvarsall.bat" amd64 && call _conan_env.bat && cd libjpeg-turbo-a257900dfd4b49c6742c7e135801e77a5fb3324c/_build && cmake .. -G "Visual Studio 14 2015 Win64" -DCONAN_LINK_RUNTIME="/MD" -DCONAN_EXPORTED="1" -DCONAN_COMPILER="Visual Studio" -DCONAN_COMPILER_VERSION="14" -DCONAN_CXX_FLAGS="/MP2" -DCONAN_C_FLAGS="/MP2" -Wno-dev -DENABLE_SHARED=0 -DENABLE_STATIC=1 -DWITH_SIMD=1
piponazo commented 7 years ago

Hi @nathanaeljones . I noticed that in this line:

conan install --scope build_tests=True -s compiler="Visual Studio" -o shared=True --build missing -s build_type=Release -s arch=%CONAN_ARCH% -s target_cpu=%TARGET_CPU% -u ../

The compiler version is missing. Then you should also update your recipe and try to deprecate the usage of ConfigureEnvironment as the warning is indicating at the end.

lasote commented 7 years ago

APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017 didn't work either? I don't know why cmake is not able to locate visual studio, I think it is not a conan issue.

Here is an example working: https://github.com/lasote/conan-zlib/blob/release/1.2.8/appveyor.yml

SSE4 commented 7 years ago

@nathanaeljones can you share these two files: See also "C:/Users/appveyor/.conan/data/libjpeg-turbo/1.5.2/nathanaeljones/testing/build/9fa351ac1b30ceabd1b27310408605a456b5fd86/libjpeg-turbo-a257900dfd4b49c6742c7e135801e77a5fb3324c/_build/CMakeFiles/CMakeOutput.log". See also "C:/Users/appveyor/.conan/data/libjpeg-turbo/1.5.2/nathanaeljones/testing/build/9fa351ac1b30ceabd1b27310408605a456b5fd86/libjpeg-turbo-a257900dfd4b49c6742c7e135801e77a5fb3324c/_build/CMakeFiles/CMakeError.log".

solvingj commented 7 years ago

So I might just be hitting this now, can someone confirm my issue is related? I can create the package fine on my machine (windows with VS2017), but Appveyor logs all say : 

npcap-dll/0.93@bincrafters/stable: Calling build()
The system cannot find the path specified.
vcvars = set "VSCMD_START_DIR=%CD%" && call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools\../../VC/Auxiliary/Build/vcvarsall.bat" x86
npcap-dll/0.93@bincrafters/stable: 
npcap-dll/0.93@bincrafters/stable: ERROR: Package 'f0a26c9ee5e5e72853094b567f427087deda1c02' build failed
npcap-dll/0.93@bincrafters/stable: WARN: Build folder C:\Users\appveyor\.conan\data\npcap-dll\0.93\bincrafters\stable\build\f0a26c9ee5e5e72853094b567f427087deda1c02
ERROR: npcap-dll/0.93@bincrafters/stable: Error in build() method, line 38
    self.run(vcvars)
    ConanException: Error 1 while executing set "VSCMD_START_DIR=%CD%" && call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\Tools\../../VC/Auxiliary/Build/vcvarsall.bat" x86
memsharded commented 6 years ago

This should be solved with: #2280.