Support for building packages with Visual Studio 2017

[barcharcraz]

I figure we should open up a discussion on how vcpkg will deal with building packages with VS15. As far as I know both VS 15's C run-time and it's c++ run-time are ABI compatible with VS2015. I've got a branch in my fork that switches everything to VS15, but there are some hard-coded paths. The developer command prompt batch scripts changed a whole lot and I can't find any environment variables saying where VS is installed. I also could not find any registry keys containing the install directory, although I didn't look all that hard.

[uecasm]

If run from a Developer Command Prompt (or from within Visual Studio itself -- one or the other ought to be assumed) then the VSINSTALLDIR environment variable contains the path to VS (and VCINSTALLDIR contains the path to the VC subdir of VS, which might be more useful). There's a bunch of other such paths; DevEnvDir is possibly another interesting one.

[barcharcraz]

I've been running vcpkg from a regular powershell. cmake knows how to find VS once you invoke it with the right generator and currently vcpkg does not seem to rely on being run in a dev command prompt.

[ras0219-msft]

You've hit the main issues directly: in VS15, there aren't any direct registry keys or environment variables to locate the installation. Instead, we need to redirect through the installer to locate all the instances on the machine and select one that meets our needs.

We explicitly don't want to require users to use a dev command prompt; we want vcpkg to be perfectly suitable for putting onto your default path and available everywhere. No matter your prompt, vcpkg install zlib:x86-windows should build the same thing.

If we switch our default generator to Ninja instead of Visual Studio 14 and we can find a way to locate the developer command prompt script from your VS15 instance, then I believe we can get builds to use VS15 automatically (from any prompt!). Ninja is super small, so it would be fine to download it as one of the initial "always available" tools.

[barcharcraz]

ninja probably performs better than VS as well. In the past there have been some bugs with it though, since it's not as mature as VS and Make and it requires cmake to ensure all dependencies are correct or things can fail to build.

[uecasm]

We explicitly don't want to require users to use a dev command prompt; we want vcpkg to be perfectly suitable for putting onto your default path and available everywhere. No matter your prompt, vcpkg install zlib:x86-windows should build the same thing.

Then how do you do multi-builds? If I want VS2015 binaries and VS15 binaries, such that I can use either one on demand, or make a build with default platform and another build with XP compatibility, how does that work? How does vcpkg know what compiler and settings I want it to use if it doesn't run through a dev command environment that tells it?

(Ideally it should be really easy to make other customisations as well, such as different library versions and options for Boost required by different applications, without having to recompile it repeatedly.)

[barcharcraz]

well I think for the 2015->"15" move it may be best to just stamp the compiler in the BUILD_INFO file and lint to make sure incompatible builds are not linked together. 2015 and "15" produce compatible binaries, so it's really just a matter of working when only one is installed, and preferring "15" when both are installed.

I think one of the reasons to move to ninja is that vcpkg would then be the program directly controlling which compiler to use, not cmake, so it can start to manage multi-builds and all the different little configurations that one may want.

[ras0219-msft]

@uecasm We want triplets to control build settings instead of the environment in order to make builds more easily reproducible and to better support cross-targeting (you don't need 8 prompts to target 8 platforms). So, if we supported xp, we might have a triplet like x86-xp and you could vcpkg install zlib:x86-xp.

In the particular case of VS 2015 and VS 2017, it's as barcharcraz mentioned: binaries built with either toolset can be consumed from either toolset, so we consider it a strict upgrade for all libraries managed by vcpkg (performance, correctness, etc). In an ideal world, we will prefer VS2017 when it's available and fall back to VS 2015 when it's not.

[vlj]

I can't get VS 2017 to use vcpkg.cmake from the "open folder" feature. The CMakeSettings.json is :

{
    "configurations": [
        {
        "name": "Visual Studio 15 x86",
        "generator": "Visual Studio 14 2015",
        "buildRoot":  "${env.LOCALAPPDATA}\\CMakeBuild\\${workspaceHash}\\build\\${name}",
        "cmakeCommandArgs":  "-DCMAKE_TOOLCHAIN_FILE=\"C:\\Users\\vljno\\Documents\\GitHub\\vcpkg\\scripts\\buildsystems\\vcpkg.cmake\"",
        "buildCommandArgs": "-m -v:minimal"
        },
        {
        "name": "Visual Studio 15 x64",
        "generator": "Visual Studio 14 2015 Win64",
        "buildRoot":  "${env.LOCALAPPDATA}\\CMakeBuild\\${workspaceHash}\\build\\${name}",
        "cmakeCommandArgs":  "-DCMAKE_TOOLCHAIN_FILE=C:\\Users\\vljno\\Documents\\GitHub\\vcpkg\\scripts\\buildsystems\\vcpkg.cmake",
        "buildCommandArgs": "-m -v:minimal"
        }
    ]
}

But cmake fails ("Could NOT find GLEW (missing: GLEW_INCLUDE_DIR GLEW_LIBRARY)") despite working properly with classic cmake.

[ras0219-msft]

I was able to get similar strange behavior, and it was fixed with two changes:

1. Add a space at the beginning of cmakeCommandArgs (maybe unneeded)
2. I had to delete the build tree and regenerate it. As you can see in your file, they're stored at %LOCALAPPDATA%\CMakeBuild.

You can then use Right Click -> reparse on the CMakeLists.txt.

[vlj]

Thanks. It looks indeed like VS 2017 is concatenating the cmakeCommandArgs and build root string without an extra space, not sure if it's a bug.

[mateka]

I guess there is a "better" way to define variables for cmake:

{
    "configurations": [
      {
        "name": "Visual Studio 15 x86",
        "generator": "Visual Studio 15 2017",
        "buildRoot": "${env.LOCALAPPDATA}\\CMakeBuild\\${workspaceHash}\\build\\${name}",
        "cmakeCommandArgs": "",
        "buildCommandArgs": "-m -v:minimal",
        "variables": [
          {
            "name": "CMAKE_TOOLCHAIN_FILE",
            "value": "D:\\Programowanie\\vcpkg\\scripts\\buildsystems\\vcpkg.cmake"
          }
        ]
      },
      {
        "name": "Visual Studio 15 x64",
        "generator": "Visual Studio 15 2017 Win64",
        "buildRoot": "${env.LOCALAPPDATA}\\CMakeBuild\\${workspaceHash}\\build\\${name}",
        "cmakeCommandArgs": "",
        "buildCommandArgs": "-m -v:minimal",
        "variables": [
          {
            "name": "CMAKE_TOOLCHAIN_FILE",
            "value": "D:\\Programowanie\\vcpkg\\scripts\\buildsystems\\vcpkg.cmake"
          }
        ]
      }
    ]
}

[barcharcraz]

Perhaps we should rename this to visual studio 2017 support, now that the RC is out

[vlj]

Is there a way to use VS 2017 to build and use vcpkg ? I removed VS2015 and I got an error because for some reason VS 2017 installer doesn't set the VS140COMMNTOOLS environment variable which is used by bootstrap.ps1. In addition the vsvarall.bat file is not in VS\ directory anymore but in VS\Auxiliary\Build. The vcxproj is relying on the vs140 toolset and Windows 8.1 SDK so the sln needs to be updated. I managed to build vcpkg using VS2017 by modifying the path and updating the toolsets however this breaks VS2015 build, there's probably a better way to do.

Unfortunatly "vcpkg install" is still using the wrong path to vcvarall.bat which prevents vcpkg usage so far.

[saulthu]

@vlj The GoingNative 54 video, from Nov 16 2016, says that VS2015 is required by vcpkg, at least for now.

[ras0219-msft]

In the GN54 video, I simply expressed the current state of vcpkg.

I would be more than happy to improve the situation and allow building packages with only VS 2017 installed once we are able to work through these issues :). The primary problem is as vlj mentioned above -- we need to locate the vcvarsall.bat file, which is made more difficult because it's not as simple as a single environment variable dereference like VS140COMNTOOLS is. Once the vcvarsall.bat script is located, we could easily switch to using the Ninja generator by default which will pick up the new compiler just fine. This may cause some problems when building for UWP, but we can address those issues when they come up.

[barcharcraz]

Resharper seems to be able to detect all the different versions of VS 2017, I may go ahead and attach procmon to its installer and see how it does this.

[barcharcraz]

I FOUND THE CONFIG INFORMATION. When VS first launches it creates it's folder in %LOCALAPPDATA%/Microsoft/VisualStudio inside this folder is a file called privateregistry.bin which is a registry hive. loading this hive will give you a view much like the old layout of the VS 14 registry!

I got this info via procmon by the way.

We could pull config information from here, and use it to find VS and all the tools.

[ras0219-msft]

Very cool about the private registry, however that smells extremely prone to breaking in the future if that key becomes internally unneeded.

I've asked around offline and was pointed at https://github.com/mluparu/vs-setup-samples. Specifically, we can use the COM APIs demonstrated in the "NoDeps" project [1] which dont require any NuGet packages or external headers/DLLs. That should let us find the root directories for all VS instances, upon which we can then do a simple fs::exists(root / "Common7/Tools/VsDevCmd.bat").

[1] https://github.com/mluparu/vs-setup-samples/tree/master/Setup.Configuration.VC.NoDeps

[willyd]

First thanks for this nice project, I feel that it can get better adoption than Nuget in the C++ community.

+1 For using Ninja as the default generator. I have been using it to build all sorts of OSS C++ libraries and it works quite well when paired with CMake. The build times are often much lower than what you get with CMake generated VS solution. Even on a single core machine it can cut the build times in half. On multi-core machine it makes a much better use of all the available threads, sometimes making your computer unusable for any other tasks :).

[barcharcraz]

I've written up some preliminary support for building packages with VS2017 in my vs2017support branch. It's not ready for a pull request yet but the method of finding VS2017 seems to work (I'm using this branch to build packages for myself).

One prominant issue is how to support different versions of VS, both different versions of 2017 and 2017 and 2015 on the same machine. While we're at it we may as well add support for ICC as well (it uses a similar batch file system to set things up)

[TheNightRider12]

How do I get the Developer Command Prompt in my Start Menu? I had it in Visual Studio 2015 but now when I installed 2017, I no longer can see or use the Developer Command Prompt outside of Visual Studio. Any help?

[ras0219-msft]

@barcharcraz I don't know what the compatibility story is for icc (against the msvc abi). In VS2017, we have made a huge effort to maintain forward and backward compatibility with VS2015, so it is safe to mix those two versions in any combination. This is why we should be able to always use 2017 if it is available.

If icc isn't ABI compatible,it should probably get its own triplets (x86-windows-icc, perhaps).

[alexkaratarakis]

ab33a7fd49904b5a4ff25af6f87d32ddf15fcd6c (vcpkg v0.0.70) adds support for VS2017.