MSVC Version 14.1 not found with 14.2 installed

[bryanwweber]

Describe the bug I have VS2019 installed, with the VC++ 14.2, 14.1, and 14.0 build tools selected in the installer. When MSVC_VERSION="14.1" is specified, SCons cannot find the 14.1 build tools. When MSVC_VERSION="14.2" or 14.0 are specified, SCons works as expected.

Required information

• Link to SCons Users thread discussing your issue: https://discord.com/channels/571796279483564041/571796280146133047/713183631547302008
• Version of SCons: 3.1.2
• Version of Python: 3.7.9
• Which python distribution if applicable (python.org, cygwin, anaconda, macports, brew,etc): Anaconda
• How you installed SCons: Conda
• What Platform are you on? (Linux/Windows and which version): Windows 10
• How to reproduce your issue? Please include a small self contained reproducer. Likely a SConstruct should do for most issues.

env = Environment(MSVC_VERSION="14.1")
env.Program("hello", "hello.cpp")

• How you invoke scons (The command line you're using "scons --flags some_arguments"): scons

Further information:

• With MSVC_VERSION="14.2", SCons generates this command for vswhere.exe: vswhere.exe -products * -version "[16.0, 17.0)" -property installationPath and the output when run on the command line is: C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise
• With MSVC_VERSION="14.1", SCons generates this command: vswhere.exe -products * -version "[15.0, 16.0)" -property installationPath and there is no output when run on the command line
• The 14.16 (current version as of this writing) build tools are located in subfolders of C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.16.27023
• The output of vswhere --products * -format json is:

[
  {
    "instanceId": "7876f29c",
    "installDate": "2020-02-04T16:37:05Z",
    "installationName": "VisualStudio/16.6.0+30114.105",
    "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise",
    "installationVersion": "16.6.30114.105",
    "productId": "Microsoft.VisualStudio.Product.Enterprise",
    "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\Common7\\IDE\\devenv.exe",
    "state": 4294967295,
    "isComplete": true,
    "isLaunchable": true,
    "isPrerelease": false,
    "isRebootRequired": false,
    "displayName": "Visual Studio Enterprise 2019",
    "description": "Scalable, end-to-end solution for teams of any size",
    "channelId": "VisualStudio.16.Release",
    "channelUri": "https://aka.ms/vs/16/release/channel",
    "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
    "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=660893#16.6.0",
    "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660909",
    "updateDate": "2020-05-20T17:41:11.1443849Z",
    "catalog": {
      "buildBranch": "d16.6",
      "buildVersion": "16.6.30114.105",
      "id": "VisualStudio/16.6.0+30114.105",
      "localBuild": "build-lab",
      "manifestName": "VisualStudio",
      "manifestType": "installer",
      "productDisplayVersion": "16.6.0",
      "productLine": "Dev16",
      "productLineVersion": "2019",
      "productMilestone": "RTW",
      "productMilestoneIsPreRelease": "False",
      "productName": "Visual Studio",
      "productPatchVersion": "0",
      "productPreReleaseMilestoneSuffix": "7.0",
      "productSemanticVersion": "16.6.0+30114.105",
      "requiredEngineVersion": "2.6.2108.39667"
    },
    "properties": {
      "campaignId": "",
      "channelManifestId": "VisualStudio.16.Release/16.6.0+30114.105",
      "nickname": "",
      "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"
    }
  }
]

[mwichmann]

Just as a "note to ourselves" - this failure isn't a surprise, as things are currently constituted. vswhere only gives us the top level information - the Visual Studio version (15.x for 2017, 16.x for 2019) - as you might expect from the tool name. It doesn't give us the Visual C++ compilers available under either of those. In fact, we have a mapping table that goes the opposite direction:

_VCVER_TO_VSWHERE_VER = {
    '14.2': '[16.0, 17.0)',
    '14.1': '[15.0, 16.0)',
}

So when asking for 14.1, we use the table to ask vswhere for Visual Studio 2017 aka 15.x. We're clearly going to miss 14.1 if it only lives under the 2019 product.

[mwichmann]

Just going to dump some research data here. On a setup with Build Tools 19 installed, we start with the 14.26 compiler. Used the setup tool to add 14.25 and 14.16 compilers to this. The list of cl.exe files now found (this is not saying anything different from the original note):

Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.16.27023/bin/HostX64/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.16.27023/bin/HostX64/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.16.27023/bin/HostX86/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.16.27023/bin/HostX86/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/HostX64/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/HostX64/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/HostX86/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.25.28610/bin/HostX86/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.26.28801/bin/Hostx64/x86/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.26.28801/bin/Hostx86/x64/cl.exe
Microsoft Visual Studio/2019/BuildTools/VC/Tools/MSVC/14.26.28801/bin/Hostx86/x86/cl.exe

[mwichmann]

Okay, I have a simple enough bit of code which detects the msvc versions installed under a given product, but the structure of vc.py is such that that information isn't widely usable - the place that makes use of the information that there's a cl.exe found doesn't share that information, it just answers True/False. I'll leave it here in case anyone else feels like running with this...

def _findall_vctools_files(startpath):
    """Extract versions of msvc available for a given product

    Given a path to a 2017-or-later install (where multi-version was enabled),
    look for Microsoft.VCToolsVersion*.txt files and extract the version
    string from them (we could also parse the .props file but this seems easier)

    Args:
        startpath: starting path

    Returns:
        list: the versions found, sorted in descending numerical order.

    """
    import glob

    vcversions = []
    for vfile in glob.glob(r'%s/**/Microsoft.VCToolsVersion*txt' % startpath, recursive=True):
        with open(vfile, 'r') as f:
            vcversions.append(f.readlines()[0].strip())
    vcversions = sorted(set(vcversions), reverse=True)  # trim any dupes
    debug('_findall_vctools_files(): located %s' % str(vcversions))
    return vcversions

[bryanwweber]

Let me know if I can help provide any information or test anything! Thanks 😃

[mwichmann]

Might be a bit... trying to think about how to set this up.

[jcbrill]

Just another note to ourselves. Informational only.

This appears to be a known problem that falls through the cracks between the VC++ development team and vswhere.

There was an issue filed for vswhere concerning this very problem that appears to have been opened and closed on March 30, 2020. The same issue was reported to the VC++ team much earlier in October 2019.

Of interest is the distinction between a "product" and a "feature":

vswhere is for finding VS and other products. Features within those products are up to the feature teams to support. For VS2019 the VC team made a version-compatible way of finding it for future toolsets they should maintain. That doesn't apply to older versions.

Please work with the VC++ team on how to properly locate older versions. See our wiki as well since I wrote some examples based on their recommendations.

Historical reference:

• March 30, 2020: https://github.com/microsoft/vswhere/issues/217
• April 2, 2020: https://developercommunity.visualstudio.com/content/problem/975002/cannot-use-vswhere-to-setup-a-vs-2017-that-was-ins.html
• October13, 2019: https://developercommunity.visualstudio.com/content/problem/775038/visual-studio-2017-build-tools-not-installed.html

[mwichmann]

After stepping through the logic in vc.py, it looks theoretically possible that if you have a batch file which sets up the environment correctly for the compiler you want to use, you can pass the path to it in env['MSVC_USE_SCRIPT'] and have SCons use that, bypassing a bunch of the detection logic (this is in the manpage). It might be necessary to craft that batch file yourself, i.e. it's not certain that a 14.1 under VS2019 actually provides the right one. Won't know until it's tried, it was apparently intended that way but it's also possible that a bunch of subsequent wiring and duct taping might cause it to no longer work.

[bdbaddog]

From the vswhere issue listed above. Sugguestion that the following would work

"C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Auxiliary\Build\vcvarsall.bat" x64 -vcvars_ver=14.16

[jcbrill]

FYI, the MSVS batch file code for processing the vcvars_ver argument is:

@REM Support the VS 2015 Visual C++ Toolset
if "%__VCVARS_VERSION%" == "14.0" (
    goto :vcvars140_version
)

@REM If VCVARS_VERSION was not specified, then default initialize the environment
if "%__VCVARS_VERSION%" == "" (
    goto :check_platform
)

:check_vcvars_ver_exists
@REM If we've reached this point, we've detected an override of the toolset version.

@REM Check if full version was provided and the target directory exists. If so, we can proceed to environment setup.
if EXIST "%VSINSTALLDIR%\VC\Tools\MSVC\%__VCVARS_VERSION%" (
    goto :check_platform
)

@REM Check if a partial version was provided (e.g. MAJOR.MINOR only).  In this case,
@REM select the first directory we find that matches that prefix.
set __VCVARS_VER_TMP=
setlocal enableDelayedExpansion
for /F %%a IN ('dir "%VSINSTALLDIR%\VC\Tools\MSVC\" /b /ad-h /o-n') DO (
    set __VCVARS_DIR=%%a
    set __VCVARS_DIR_REP=!__VCVARS_DIR:%__VCVARS_VERSION%=_vcvars_found!
    if "!__VCVARS_DIR!" NEQ "!__VCVARS_DIR_REP!" (
        set "__VCVARS_VER_TMP=!__VCVARS_DIR!"
        goto :check_vcvars_ver_exists_end
    )
)
:check_vcvars_ver_exists_end 

If a full version number (e.g., "14.NN.NNNNN") is not passed via vcvars_ver, an attempt is made to make a partial match of the user's version number and the folder names under VC\Tools\MSVC.

The vc\tools\msvc directory contents are sorted by reverse name order. If the user's vcvars_ver argument matches ANY part of the folder name then it is considered to found.

In Bill's example above, using "-vcvars_ver=14.1" should work just as well and will match the "largest" version of 14.1X installed (i.e., "14.1X.nnnnn").

Due to the code above, caution should be exercised when passing a version number as a mistake may very well match: "-vcvars_ver=26" will match "14.26.28801".

[jcbrill]

I have not had time to test the following yet.

A quick-and-dirty hack would be to add an environment setting (e.g., "MSVC_VCVARS_VER") that will be passed to the vcvarsall batch file via "-vcvars_ver" argument.

In the msvc_find_valid_batch_script method in vc.py , an argument list is already being constructed. It might be possible to add the "-vcvars_ver" argument similar to the following:

    # Get just version numbers
    maj, min = msvc_version_to_maj_min(version)
    # VS2015+
    if maj >= 14:
        if env.get('MSVC_UWP_APP') == '1':
            # Initialize environment variables with store/universal paths
            arg += ' store'
    # VS2017+
    if maj >= 14.1:
        vcvars_ver = env.get('MSVC_VCVARS_VER')
        if vcvars_ver:
            vcvars_ver = vcvars_ver.strip()
            if vcvars_ver:
                arg += ' -vcvars_ver=' + vcvars_ver

Usage would be something along the lines of:

Environment(MSVC_VERSION="14.2", MSVC_VCVARS_VER="14.1")

If there is a problem with user's version string, the batch file execution should fail.

While this is quick-and-dirty and may work, there may be deeper issues involving the cache.

In the long run, a second variable may be necessary: MSVC_VERSION should be the installed version (e.g., 14.2, 14.1, etc) and the second variable should be a proper prefix of the msvc tool folder name. This would allow any tool version to be used and mirror the behavior of the MS batch files.

A user that has both 2017 and 2019 installed, may desire to use 2017 rather than the 2019 build tools for 2017. In this case specifying a MSVC_VERSION of "14.1" would mean the 2017 installation and not the 2019 installation even if the 2017 tools are installed.

[jcbrill]

Proof-of-concept code for supporting specific msvc toolset versions is shown below and attached. Testing has been limited.

An environment variable "MSVC_SPECIFIC_VERSION" is used to choose a specific toolset for a given "MSVC_VERSION".

The motivating example from above would be specified as follows for MSVC 2019:

env = Environment(MSVC_VERSION="14.2", MSVC_SPECIFIC_VERSION="14.1")
env.Program("hello", "hello.cpp")

A patch of the modified version of vc.py from the scons master is attached in scons-master-msvcver-patch.zip.

The modified version of vc.py from the scons master is attached in scons-master-msvcver-souce.zip along with a test folder containing a minimal SConstruct file used for testing.

The implementation:

• Prior to the existing code determining the default version to use for 2017 and later, the msvc installation folders are processed and an ordered list of toolset versions for each (host, target) pair is saved in a dictionary.
• When the "MSVC_SPECIFIC_VERSION" variable is defined for 2017 and later, the toolset version dictionary is searched for a match. The specific version must be a proper prefix of the toolset version (i.e., "startswith"). If a match is found, the formatted "-vcvars_ver=XX.XX.XXXXX" for the full version is returned and added to the batch file arguments. There is a special shortcut for "14.0" (2015).
• If the "MSVC_SPECIFIC_VERSION" variable is not defined or is empty, None is returned, and no argument is added for the specific version.
• The most likely place for failure is the host folder name mapping.
• While the changes were made against the master, it could probably be patched into the current production version without much effort.

New code:

# Specific toolset version support for 14.1 (VS2017) and later
# Store toolset version list by (host,target) for each _VCVER version > 14
_VCVER_SPECIFIC_VERSIONS = {}

# Map MSVC host/target directory names to (host,target) specification names
_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14 = {}

# Mapping of directory names to specification names done once during initialization
__SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14 = False

def _find_all_cl_in_vc_dir(env, vc_dir, msvc_version):
    """
    Find the locations of all cl.exe installed by Visual Studio/VCExpress.

    For versions 2017 and later, store individual toolset versions for each
    (host, target) pair discovered.  Save the vcvars_ver argument as well.
    """

    global __SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14

    ver_num = float(get_msvc_version_numeric(msvc_version))

    if ver_num > 14:

        if not __SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14:
            # map msvc host/target directory names to (host,target) specification names
            for vc_target_pair, vc_dir_pair inHOST_TARGET_TO_CL_DIR_GREATER_THAN_14.items():
                # case insensitive store and lookup
                _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_dir_pair[0].lower()] = vc_target_pair[0]
                _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_dir_pair[1].lower()] = vc_target_pair[1]
            __SETUP_HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14 = True

        vc_tool_d = {}

        # expected folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST\TARGET\cl.exe

        vc_msvc_path = os.path.join(vc_dir, 'Tools', 'MSVC')
        for vc_specific_version in os.listdir(vc_msvc_path):

            vc_tool_path = os.path.join(vc_msvc_path, vc_specific_version)
            if not os.path.isdir(vc_tool_path): continue

            # folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX
            debug('vc_specific_version: ' + vc_specific_version)

            vc_bin_path = os.path.join(vc_tool_path, 'bin')
            if not os.path.exists(vc_bin_path): continue

            for vc_host_dir in os.listdir(vc_bin_path):

                vc_host_path = os.path.join(vc_bin_path, vc_host_dir)
                if not os.path.isdir(vc_host_path): continue

                # folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST
                debug('vc_host_dir: ' + vc_host_dir)

                for vc_target_dir in os.listdir(vc_host_path):

                    vc_target_path = os.path.join(vc_host_path, vc_target_dir)
                    if not os.path.isdir(vc_target_path): continue

                    # folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST\TARGET
                    debug('vc_target_dir: ' + vc_target_dir)

                    cl_tool_path = os.path.join(vc_target_path, _CL_EXE_NAME)
                    if os.path.exists(cl_tool_path):

                        # map directory names to specication names
                        vc_host_target_name = _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_host_dir.lower()]
                        vc_target_target_name = _HOST_DIRNAME_TO_HOST_TARGETNAME_GREATER_THAN_14[vc_target_dir.lower()]

                        # folder structure: VC_DIR\Tools\MSVC\XX.XX.XXXXX\bin\HOST\TARGET\cl.exe
                        debug('found cl.exe: %s\\%s\\%s (%s, %s, %s)' % (
                            vc_specific_version, vc_host_dir, vc_target_dir, 
                            vc_specific_version, vc_host_target_name, vc_target_target_name)
                        )

                        # register specific version for target specification (host,target)
                        key = (vc_host_target_name, vc_target_target_name)
                        payload = (vc_specific_version, ' -vcvars_ver=' + vc_specific_version)

                        vc_tool_d.setdefault(key,[]).append(payload)

        # sort version number lists in descending order
        for vc_target_spec in vc_tool_d.keys():
            vc_tool_d[vc_target_spec] = sorted(vc_tool_d[vc_target_spec], key = lambda x: x[0], reverse=True)

        # register tool versions dictionary
        _VCVER_SPECIFIC_VERSIONS[msvc_version] = vc_tool_d

    return None

def _find_specific_version(env, msvc_version, vc_host, vc_target):
    """
    Find a specific toolset version.

    Returns vcvars_ver argument string or None
    """

    vc_specific_version = env.get("MSVC_SPECIFIC_VERSION", "").strip()
    if not vc_specific_version:
        return None

    if vc_specific_version == "14.0":
        # "14.0" is handled as a special case in vcvars.bat
        vcvars_ver = ' -vcvars_ver=' + vc_specific_version
        debug('vcvars_ver match found:%s %s' % (vc_specific_version, vcvars_ver))
        return vcvars_ver

    try:
        vc_tool_d = _VCVER_SPECIFIC_VERSIONS[msvc_version]
    except KeyError:
        debug('_VCVER_SPECIFIC_VERSIONS lookup failed:%s' % msvc_version)
        return None

    try:
        vc_tool_list = vc_tool_d[(vc_host, vc_target)]
    except KeyError:
        debug('vc_tool_d lookup failed:%s (%s,%s)' % (msvc_version, vc_host, vc_target))
        return None

    for version_t in vc_tool_list:
        specific_version, vcvars_ver = version_t
        if specific_version.startswith(vc_specific_version):
            debug('vcvars_ver match found:%s %s' % (vc_specific_version, vcvars_ver))
            return vcvars_ver

    debug('vcvars_ver lookup failed:%s (%s,%s) version %s' % (msvc_version, vc_host, vc_target, vc_specific_version))
    return None

Hook into _check_cl_exists_in_vc_dir(env, vc_dir, msvc_version):

    # make sure the cl.exe exists meaning the tool is installed
    if ver_num > 14:
        # find all vc toolset versions for 2017 and later
        _find_all_cl_in_vc_dir(env, vc_dir, msvc_version)

        # 2017 and newer allowed multiple versions of the VC toolset to be
        # installed at the same time. This changes the layout.
        # Just get the default tool version for now
        #TODO: support setting a specific minor VC version
        default_toolset_file = os.path.join(vc_dir, _VC_TOOLS_VERSION_FILE)

Hook into def msvc_find_valid_batch_script(env, version):

            # Get just version numbers
            maj, min = msvc_version_to_maj_min(version)
            # VS2015+
            if maj >= 14:
                if env.get('MSVC_UWP_APP') == '1':
                    # Initialize environment variables with store/UWP paths
                    arg = (arg + ' store').lstrip()
            # VS2017+
            if maj >= 14 and min >= 1:
                vcvars_ver = _find_specific_version(env, version, host_platform, tp)
                if vcvars_ver:
                    arg = (arg + vcvars_ver).lstrip()

scons-master-msvcver-patch.zip scons-master-msvcver-source.zip

[bdbaddog]

@jcbrill - please make a PR. large patches in issues are hard to manage/comment on/etc.. We can refine and/or at least pull your changes from your branch and continue work on them much more easily from a PR.

[mwichmann]

We might want to resurrect the MSVS_VERSION var, which was deprecated (right now forced to be the same as MSVC_VERSION).

[jcbrill]

Are there legacy issues with existing SConstruct/SConscript files if MSVS_VERSION is resurrected and the definition of MSVC_VERSION changes/expands?

[mwichmann]

Honestly, no clue. It feels like letting MSVC_VERSION also take a more specific version string wouldn't hurt anything, here's the current claim:

MSVC_VERSION
    Valid values for Windows are 14.2, 14.1, 14.1Exp, 14.0, 14.0Exp, 12.0, 12.0Exp, 11.0, 11.0Exp, 10.0, 10.0Exp, 9.0, 9.0Exp, 8.0, 8.0Exp, 7.1, 7.0, and 6.0.

What turning MSVS_VERSION back to relevant and allowed to specify a required VS version would do - it might affect things. I think it might still have a meaning in the vs module, the one that is used to generate project files but not actually hunt for usable compilers.

It was just a thought :)

[mwichmann]

Note to self (or rather to create a github xref)... tried to capture the detection flow in a README file added by #3690 (unmerged as of this writing). Would want to update that if the above changes are added.

[jcbrill]

A challenge with the current MSVC_VERSION definition is that "14.2", "14.1" and "14.1Exp" describe an msvs installation rather than an msvc toolset (14.16.27023) which can be present in all three.

For a user with all three versions (2019, 2017, 2017Exp) installed, an msvc version of 14.16.27023 is not unique. If a user specifies 14.16, should SCons use the 14.1, 14.1Exp, or 14.2 toolset?

My guess is that some users will want to explicitly specify the msvs installation and toolset.

Perhaps when migrating from an older version of msvs to a newer version of msvs and making sure the build processes are identical. Or perhaps a client/customer only has the express version installed and a developer would like to make sure there are no build issues between the two.

Still thinking about the problem...

[jcbrill]

I apologize in advance as I am time limited for the rest of the day today and tomorrow morning.

Notes to self while it is still fresh that a bug report that needs to be issued.

During initial testing of the proposed changes above, and after installing 14.1Exp, a few issues were discovered.

There are at least three problems with the msvc detection in the master and one of them is in 3.1.2:

• "14.1Exp" was added to the _VCVER list but was not added to the _VCVER_TO_VSWHERE_VER dictionary. This causes a key error when trying to look up the vswhere_ver and exits without finding the express version. Extending the dictionary results in the problem in the next item.

• When both 14.1 and 14.1Exp are installed, the current query for vswhere is unreliable. The express version is returned for both 14.1 and 14.1Exp. The first result returned is not necessarily the desired result.

  The problem has to to with using "-products * -version [15.0,16.0)" in the query which finds both the full version and the express version. With only -version [15.0, 16.0) the full version is found. With -products Microsoft.VisualStudio.Product.WDExpress the express version is found.

  One solution is fairly straightforward.

  
  _VCVER_TO_VSWHERE_VER = {
  '14.2'    : [ '-version', '[16.0, 17.0)' ],
  '14.1'    : [ '-version', '[15.0, 16.0)' ],
  '14.1Exp' : [ '-products', 'Microsoft.VisualStudio.Product.WDExpress' ],
  }

...

vswhere_cmd = [vswhere_path] + vswhere_ver + ['-property', 'installationPath']

- MSVC 6.0 is not detected due to the directory walk and checking if "cl.exe" exists in the file list.  Believe it or not, in msvc 6.0, the filename is "CL.EXE".  This is in 3.1.2 and master.  

  A quick test for cl.exe existing in the bin folder (i.e., under vc_dir) prior to the directory walk is successful for 7.1, 7.0, and 6.0.  If that ever fails, the case-sensitive comparison for 6.0 is a problem.
```py
        # quick test for bin directory
        cl_path = os.path.join(vc_dir, 'bin', _CL_EXE_NAME)
        if os.path.exists(cl_path):
            debug('get_installed_vcs ' + _CL_EXE_NAME + ' found %s' % cl_path)
            return True

This is somewhat embarrassing: as recently as last week I thought I was doing c preprocessor tests against 6.0.

Vintage 32-bit windows 7 test box (circa 2007) with the following now discovered by locally "modified" Scons:

installed_vcs:['14.2', '14.1', '14.1Exp', '14.0', '12.0', '11.0', '11.0Exp', '10.0', '9.0', '8.0', '7.1', '7.0', '6.0']

There are no boxes here with git installed nor anyone with git experience. Minor patches for the above can be provided quickly. In the short term, pull request(s) a little longer...

[jcbrill]

The issue with MSVC 6.0 mentioned above is a false negative:

• The initial code to find installed vcs (get_installed_vcs) fails to find msvc 6.0 due to the case sensitive name test (cl.exe == CL.EXE). The debug logging shows that no compiler is found for 6.0 and it does not show up in the installed versions list.
• However, when msvc_find_valid_batch_script is called later, it correctly finds the msvc 6.0 bin\vcvars32.bat file.
• The diagnostic for the installed version check indicates that msvc 6 is not installed but the build process will use actually use msvc 6.

As described above, 14.1 Express will not work as intended in the master and the obvious fix will lead to a less-obvious problem with the vswhere arguments.

The introduction of 14.1 Express also makes the original issue report a little more thorny as now there are at least three msvs installations that support the 14.1 toolset.

@bdbaddog or @mwichmann Should the 14.1 Express and msvc 6 false negative issues go through the mailing list or is it OK to file an issue directly?

Joe

[mwichmann]

For me, the 14.1Express support was a recent addition, and so "not fully debugged" isn't that surprising. My tests were on a VM that has only express, so I wouldn't have seen the overlap problems. Seems like including fixes for that as part of the other "finding" work is fine - but will wait for @bdbaddog to weigh in. I don't have disk space to test more combinations - my native Windows box is full (and old/slow), and the drive holding the VMs is also full - including 1TB of Windows VMs that do nothing but test scons combinations :)

[jcbrill]

I have attached a second prototype of specific version support. This is pre-alpha quality lacking adequate testing, documentation and error checking. It may very well only work on my computer.

This prototype accepts a specific toolset version via the MSVC_VERSION variable. Internally, the specific version is converted to the existing _VCVER format for the existing code. This very well may break code outside of vc.py (e.g., test code) expecting the current definitions.

Probably a better implementation is to define an internal-user variable with the specific version.

An additional variable was added "MSVC_PRODUCT" to specify the installed version when there are choices. For example, when 14.1 and 14.2 are both installed, a MSVC_VERSION="14.1" will use the 14.1 installation. With MSVC_VERSION="14.1" and MSVC_PRODUCT="14.2", the "14.2" installation will use the 14.16 tools. This is a bit tricky as early on the MSVC_VERSION need to be "rewritten".

This version will also use installed 14.2 for MSVC_VERSION="14.1" when there are no instances of 14.1 installed which is the point of this current issue.

The desire to expand the definition of MSVC_VERSION has been satisfied. Trying to adhere to the "Law of Least Astonishement", if the product for the MSVC_VERSION is installed that will be used. It can be over-ridden via the MSVC_PRODUCT variable.

Due to the nature of the implementation, there is experimental support for choosing the product installation type: Ent (Enterprise), Pro (Professional), Com (Community), Exp (Express), and BT (BuildTools). Examples are shown below.

vswhere is run once for all installations and the json output is processed for fields of interest. Unfortunately, since this is done so early I don't believe a user's vswhere variable will have any effect.

Three exceptions were added and raised for invalid data. As the specific version information is passed as an argument to the vcvars batch files, ignoring any errors can/will result in a successful compilation with possibly a different (e.g., default) toolset than the user specification. For now, the exceptions have been useful in identifying corner cases.

Support for 14.0 with the later tools is precarious at best. The Express version of 14.1 makes the problem more annoyingly difficult.

scons-master-msvcver-source-2.zip

Example test configurations:

env = Environment() # 14.2 Community

# 14.2 Installed [Community]

#env = Environment(MSVC_VERSION="14.2")        # 14.2 Community
#env = Environment(MSVC_VERSION="14.26")       # 14.2 Community
#env = Environment(MSVC_VERSION="14.26.28801") # 14.2 Community
#env = Environment(MSVC_VERSION="14.25")       # 14.2 Community
#env = Environment(MSVC_VERSION="14.25.28610") # 14.2 Community
#env = Environment(MSVC_VERSION="14.20")       # 14.2 Community
#env = Environment(MSVC_VERSION="14.20.27508") # 14.2 Community
#env = Environment(MSVC_VERSION="14.27")       # 14.2 Community: MSVCProductNotFound exception

# 14.1 [Fake] Not Installed [Community or Express]

#env = Environment(MSVC_VERSION="14.1")        # 14.2 Community
#env = Environment(MSVC_VERSION="14.16")       # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27")    # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27023") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27024") # 14.2 Community: MSVCProductNotFound exception

# 14.1 Installed [Community, Express]

#env = Environment(MSVC_VERSION="14.1")        # 14.1 Community [Ent->Pro->Com->Exp->BT]
#env = Environment(MSVC_VERSION="14.16")       # 14.1 Community
#env = Environment(MSVC_VERSION="14.16.27")    # 14.1 Community
#env = Environment(MSVC_VERSION="14.16.27023") # 14.1 Community
#env = Environment(MSVC_VERSION="14.15")       # 14.1 Community
#env = Environment(MSVC_VERSION="14.15.26726") # 14.1 Community
#env = Environment(MSVC_VERSION="14.14")       # 14.1 Community
#env = Environment(MSVC_VERSION="14.14.26428") # 14.1 Community
#env = Environment(MSVC_VERSION="14.16.27024") # 14.1 Community: MSVCProductNotFound exception

#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1")    # 14.1 Community
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Exp") # 14.1 Express
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Com") # 14.1 Community
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Ent") # 14.1 Enterprise:   MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1Pro") # 14.1 Professional: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.1BT")  # 14.1 BuildTools:   MSVCProductNotFound exception

#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2")    # 14.2 Community
#env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2Com") # 14.2 Community

#env = Environment(MSVC_VERSION="14.16",       MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27",    MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27023", MSVC_PRODUCT="14.2") # 14.2 Community
#env = Environment(MSVC_VERSION="14.16.27024", MSVC_PRODUCT="14.2") # 14.2 Community: MSVCProductNotFound exception

# 14.1 Express

#env = Environment(MSVC_VERSION="14.1Exp")         # 14.1 Express
#env = Environment(MSVC_VERSION="14.16Exp")        # 14.1 Express
#env = Environment(MSVC_VERSION="14.16.27Exp")     # 14.1 Express
#env = Environment(MSVC_VERSION="14.16.27023Exp")  # 14.1 Express
#env = Environment(MSVC_VERSION="14.16.27024Exp")  # 14.1 Express: MSVCProductNotFound exception

# 14.0 [Fake] Not Installed

#env = Environment(MSVC_VERSION="14.0")                         # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2")    # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Com") # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1")    # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Com") # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Exp") # 14.1 Express -> 14.0

# 14.0 Installed

#env = Environment(MSVC_VERSION="14.0")                         # 14.0 Community
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.0")    # 14.0 Community
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.0Exp") # 14.0 Community: MSVCProductInvalid [conflict]
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1")    # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Com") # 14.1 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Exp") # 14.1 Express -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2")    # 14.2 Community -> 14.0
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Com") # 14.2 Community -> 14.0

#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Ent") # 14.1 Enterprise: MSVCProductNotFound exception 
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1Pro") # 14.1 Professional: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.1BT")  # 14.1 BuildTools: MSVCProductNotFound exception

#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Ent") # 14.2 Enterprise: MSVCProductNotFound exception 
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2Pro") # 14.2 Professional: MSVCProductNotFound exception
#env = Environment(MSVC_VERSION="14.0", MSVC_PRODUCT="14.2BT")  # 14.2 BuildTools: MSVCProductNotFound exception

[jcbrill]

Once the 2019 build tools were actually installed, a problem with identifying "synthetic" versions (e.g., Enterprise, Professional, Community, BuildTools) when there are multiple versions installed was revealed.

The example configurations below actually address the problem in the original post.

The MSVC_PRODUCT="14.2BT" may not be necessary if there is only one instance of 14.2. This has not been tested yet.

Will move the source to git soon ...

scons-master-msvcver-source-3.zip

Example test configurations:

env = Environment(MSVC_VERSION="14.1", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools

#env = Environment(MSVC_VERSION="14.2",        MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.26",       MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.26.28801", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.25",       MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.25.28610", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.1",        MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.16",       MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.16.27023", MSVC_PRODUCT="14.2BT") # 14.2 BuildTools
#env = Environment(MSVC_VERSION="14.0",        MSVC_PRODUCT="14.2BT") # 14.2 BuildTools -> 14.0

[jcbrill]

I have a "working" prototype version of vc.py that unifies both #3265 and #3664.

The prototype is not production ready but does illustrate that both #3265 and #3664 can be addressed with the same implementation. There is a great deal of new code that needs to be fully tested, documented, and modified based on feedback. The ramifications to code outside of vc.py are unknown.

If desired, A PR can be issued to move this discussion. The reality is that integration is likely a ways off in the future.

3265 is an enhancement to allow a specific version of an msvc toolset. In #3265, there was a suggestion to allow specifying a specific version via the existing environment variable MSVC_VERSION.

3664 (this issue) involves detecting a toolset that is installed in a later product version (i.e., the 14.16 toolset via the 14.2 product line). In this case, the 14.1 toolset via the 14.2 Build Tools.

The prototype implementation is "wired-in" to the existing code base in a few select locations. The existing code base continues to operate on an MSVC_VERSION that is compatible with the _VCVER version list (e.g., "14.2"). A single internal environment variable and dictionary was added to maintain the specific version and product state.

There is a single hard-coded variable at the top of the source file that selectively enables/disables the specific version support. When disabled, the functionality should be exactly the same as the current master. The hooks into the existing code were intentionally minimized to ease updating from the master. The bulk of the new code was added at the bottom of the file.

The MSVC_VERSION format was expanded to allow:

• specifying an individual toolset at varying degrees of precision:
  • "14.1"
  • "14.16"
  • "14.16.27"
  • "14.16.27023"
• specifying individual product types (attached to a specific toolset or a product):
  • Enterprise: "14.26Ent"
  • Professional: "14.2Pro"
  • Community: "14.26.27Com"
  • Build Tools: "14.2BT"
  • Express: "14.1Exp"
• a right assignment operator ("->") that maps a specific toolset to a defined product version with an optional product type:
  • "14.0->14.2" use the the 14.0 toolset via a 2019 installation
  • "14.1->14.2BT" use the 14.1 toolset via a 2019 Build Tools installation
  • "14.1->14.2Com" use the 14.1 toolset via a 2019 Community installation

Without a product qualifier (e.g., "BT"), there is a subtle but significant change semantically: the product selected will be the newest/latest toolset that matches the user specification within installed toolsets for a given (host,target) combination. Effectively, without a product qualifier, any product type could be returned based on the installed toolsets. This only affects multiple installations of the same product version (e.g., 14.1 and 14.Exp and/or 14.2Com and 14.2BT).

This means that a user does not need to specify "14.1Exp" explicitly if there is a single 14.1 product installation. On the flip side, it also means that "14.1Exp" may be returned for a "14.1" request. In local testing, "14.1Exp" was returned for an "arm" target. In the local installations of 14.1 Community and 14.1 Express, the "newest" toolset for an arm target was in the 14.1 Express installation as it was installed recently.

With a product qualifier, a specific product version and type is used.

The motivating example for the current issue, is that no 14.1 products are installed and the 14.1 toolset is not found within the 14.2 Build Tools installation. In the prototype implementation, "MSVC_VERSION=14.1" will return an installed version of 14.1 if it exists or an installed version of 14.2 if the 14.16 toolset is installed for the host/target combination.

There are two attachments:

1. msvc-specific-142.txt contains edited testing output for the following SConstruct specification:

   # 14.1 is not installed, 14.2 Build Tools is installed
   env = Environment(MSVC_VERSION="14.1")

2. msvc-specific-141-142.txt contains edited testing output for the following SConstruct specifications:

   # 14.1 Installed [Community, Express], 14.2 Installed [Community, BuildTools]
   env1 = Environment(MSVC_VERSION="14.1")
   env2 = Environment(MSVC_VERSION="14.16.27Exp") # 14.16.27->14.1Exp
   env3 = Environment(MSVC_VERSION="14.1->14.2")
   env4 = Environment(MSVC_VERSION="14.1->14.2BT")

The internal version tables (all installed and by version), internal toolset dictionary state at four locations, and select SConstruct environment variables are displayed.

I realize everyone is busy trying get 4.0 out the door...

msvc-specific-142.txt msvc-specific-141-142.txt

[bdbaddog]

Yes. Please push a PR mark it "[WIP]" in the title.

On Wed, Jun 24, 2020 at 10:49 AM Joseph Brill notifications@github.com wrote:

I have a "working" prototype version of vc.py that unifies both #3265 https://github.com/SCons/scons/issues/3265 and #3664 https://github.com/SCons/scons/issues/3664.

The prototype is not production ready but does illustrate that both #3265 https://github.com/SCons/scons/issues/3265 and #3664 https://github.com/SCons/scons/issues/3664 can be addressed with the same implementation. There is a great deal of new code that needs to be fully tested, documented, and modified based on feedback. The ramifications to code outside of vc.py are unknown.

If desired, A PR can be issued to move this discussion. The reality is that integration is likely a ways off in the future.

3265 https://github.com/SCons/scons/issues/3265 is an enhancement to

allow a specific version of an msvc toolset. In #3265 https://github.com/SCons/scons/issues/3265, there was a suggestion to allow specifying a specific version via the existing environment variable MSVC_VERSION.

3664 https://github.com/SCons/scons/issues/3664 (this issue) involves

detecting a toolset that is installed in a later product version (i.e., the 14.16 toolset via the 14.2 product line). In this case, the 14.1 toolset via the 14.2 Build Tools.

The prototype implementation is "wired-in" to the existing code base in a few select locations. The existing code base continues to operate on an MSVC_VERSION that is compatible with the _VCVER version list (e.g., "14.2"). A single internal environment variable and dictionary was added to maintain the specific version and product state.

There is a single hard-coded variable at the top of the source file that selectively enables/disables the specific version support. When disabled, the functionality should be exactly the same as the current master. The hooks into the existing code were intentionally minimized to ease updating from the master. The bulk of the new code was added at the bottom of the file.

The MSVC_VERSION format was expanded to allow:

• specifying an individual toolset at varying degrees of precision:
  • "14.1"
  • "14.16"
  • "14.16.27"
  • "14.16.27023"
• specifying individual product types (attached to a specific toolset or a product):
  • Enterprise: "14.26Ent"
  • Professional: "14.2Pro"
  • Community: "14.26.27Com"
  • Build Tools: "14.2BT"
  • Express: "14.1Exp"
• a right assignment operator ("->") that maps a specific toolset to a defined product version with an optional product type:
  • "14.0->14.2" use the the 14.0 toolset via a 2019 installation
  • "14.1->14.2BT" use the 14.1 toolset via a 2019 Build Tools installation
  • "14.1->14.2Com" use the 14.1 toolset via a 2019 Community installation

Without a product qualifier (e.g., "BT"), there is a subtle but significant change semantically: the product selected will be the newest/latest toolset that matches the user specification within installed toolsets for a given (host,target) combination. Effectively, without a product qualifier, any product type could be returned based on the installed toolsets. This only affects multiple installations of the same product version (e.g., 14.1 and 14.Exp and/or 14.2Com and 14.BT).

This means that a user does not need to specify "14.1Exp" explicitly if there is a single 14.1 product installation. On the flip side, it also means that "14.1Exp" may be returned for a "14.1" request. In local testing, "14.1Exp" was returned for an "arm" target. In the local installations of 14.1 Community and 14.1 Express, the "newest" toolset for an arm target was in the 14.1 Express installation as it was installed recently.

With a product qualifier, a specific product version and type is used.

The motivating example for the current issue, is that no 14.1 products are installed and the 14.1 toolset is not found within the 14.2 Build Tools installation. In the prototype implementation, "MSVC_VERSION=14.1" will return an installed version of 14.1 if it exists or an installed version of 14.2 if the 14.16 toolset is installed for the host/target combination.

There are two attachments:

1. msvc-specific-142.txt contains edited testing output for the following SConstruct specification:

   14.1 is not installed, 14.2 Build Tools is installed

   env = Environment(MSVC_VERSION="14.1")

2. msvc-specific-141-142.txt contains edited testing output for the following SConstruct specifications:

   14.1 Installed [Community, Express], 14.2 Installed [Community, BuildTools]

   env1 = Environment(MSVC_VERSION="14.1") env2 = Environment(MSVC_VERSION="14.16.27Exp") # 14.16.27->14.1Exp env3 = Environment(MSVC_VERSION="14.1->14.2") env4 = Environment(MSVC_VERSION="14.1->14.2BT")

The internal version tables (all installed and by version), internal toolset dictionary state at four locations, and select SConstruct environment variables are displayed.

I realize everyone is busy trying get 4.0 out the door...

msvc-specific-142.txt https://github.com/SCons/scons/files/4827072/msvc-specific-142.txt msvc-specific-141-142.txt https://github.com/SCons/scons/files/4827073/msvc-specific-141-142.txt

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/SCons/scons/issues/3664#issuecomment-648970241, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAUCNB5IL26H5KMOZWX3ZTRYI4DFANCNFSM4NHLQUDA .

[jcbrill]

For posterity.

It has been deemed that this issue is not a bug and is the preferred behavior based on the response in https://github.com/SCons/scons/pull/3717#issuecomment-670822825:

One of the first principles of SCons..

The build should be repeatable regardless of user's environment.

So, if a user requests version X.1, then get version X.1, even if they have version X.2. If X.1 is not available, it should bail out.

We're not going to change a fundamental principle of SCons (reproducible builds) because you don't want to update your build scripts when there's new releases of the tools.

Now, if you were suggesting that we introduced a way for the build system to EXPLICITLY introduce flexibility, that would be acceptable.

But not without explicitly requesting it.

So MSVC_CHOOSE_VERSION(or even MSVC_VERSION if it's a callable) as a python function which gets passed the list of available versions and selects, that'd be fine.

But asking for 14.1 and accepting 14.2... nope.

BTW. I'm totally not saying that what you need SCons to do is unreasonable, just that it changes a fundamental SCons principle, and so wouldn't be merge-able without some explicit way to tell SCons to be flexible.

Curiously, the behavior deemed unacceptable is this reason for this issue and was laid out very clearly above prior to being requested in a PR.

But asking for 14.1 and accepting 14.2... nope.

Which is exactly the "bug" in the current issue.

Despite the fact that MSVC 2017 and MSVC 2019 are toolset oriented (i.e., a MSVC 2019 installation is a container for 14.1X, 14.2X, and to a lesser extent 14.0 toolsets), the desire is that the MSVC detection remain product oriented.

Prior to MSVS 2017, there was a 1:1 mapping between the msvc product and the msvc toolset. The toolset might be upgraded in an update or service pack but there was no way to select a specific toolset as side-by-side installations were not supported.

With MSVC 2017 and MSVC 2019, an SCons request of "14.1" or "14.2" is a toolset request. The current msvc behavior uses the default toolset which is generally the newest installed toolset version when the "--vcvars_ver" argument is not specified for the vcvars batch file.

For example, "14.1" (MSVC 2017) is likely to return "14.16.27023". Effectively, the product request is a toolset request similar to "find the newest toolset version that startswith('14.1')". However, "14.2" (MSVC 2019) will return a toolset based on the last update which may vary across users and computers.

The actual toolset version used for 2017 and 2019 is based on end-user's installation options and frequency of updates. While two different users may be using the same product (e.g., 2019) they may not be using the same toolset. Similarly, the installed toolset version is not necessary the same for all host/target combinations in a given installation.

The current implementation will use a default toolset within a product across different users and environments, which may not be the same, but will not use a toolset across products.

For whatever reason (e.g., enterprise and/or customer requirements), if a build depends on the 14.1X toolset it hardly seems as important in which product container it is installed (given a simple selection rule). The same issue will arise when the next MSVC product is released and end-users desire to use the 14.2X toolset combination.

There are two schools of thought:

• If you want to use the 14.1X toolset in any version other that MSVC 2017: a user must specify the product and the toolset. This is the product-centric view.

• If you want to use the 14.1X toolset: automatically select MSVC 2017 if it is installed, otherwise select the newest product with the toolset installed. An explicit product specification is optional. This is the toolset-centric view.

  For the majority of end-users, there is likely only one msvc installation anyway so there is no ambiguity nor an explicit need to specify a product. This current issue falls into this camp.

A fundamental shift from a product-centric view to a toolset-centric view for the msvc tools would go a long way in supporting the msvc tools in the future and reduce the burden on an end-user.

The point of this particular issue is that a toolset is desired and not a necessarily product/toolset.

The build should be repeatable regardless of user's environment.

If the selected product for the requested toolset is supported by SCons, this principle should hold. The binary tools may be different. The binary tools are likely already different within the same product across users and computers.

It does not seem to be any different than the same build scripts using an installed version of mingw gcc 9.3 on one computer and using an installed version of mingw gcc 8.3 on another computer simply due to a lag in updates.

While this is certainly a noble tenet, in practice I'm not sure this always holds across two or more users in a windows environment or even across two or more windows boxes.

For example, the addition of cygwin to the default tools path caused some local builds to fail. The solution was to explicitly specify the tools used thus bypassing the effect of cygwin being added to the path. However, if the same build scripts were developed on a box without cygwin installed and sent to a customer who had cygwin installed the build would have failed.

[bdbaddog]

@jcbrill - No it has not been ruled that this is not a bug. If it had been I would have closed it with comments.

Please stop overreacting.

[jcbrill]

@bdbaddog I apologize if you believe that I'm overreacting. It is impossible to tell how much thought you may have put into supporting this issue and specific versions in general from one or two line comments here and there. I am merely trying to document some hard earned insight for your benefit in the future and for others who may attempt a solution.

[bdbaddog]

@jcbrill - As you're apparently unwilling to join any of the normal forms of discourse for this project, it's difficult to discuss this with you. Yes. I believe you are overreacting. I (and no volunteers here) work on this project 24/7. We do the best we can and are rarely rewarded with thanks and more often with complaints that bugs reported didn't immediately get our undivided attention until they were successfully resolved. Unfortunately that's just not possible. Listing the number of days/weeks an issue has been open is really only fair when you are PAYING for support. Which no-one is for this project.

[jcbrill]

@bdbaddog I understand completely. That is why actual solutions were provided that were worked on full time until completion. To be clear, I am not in need of any fixes, enhancements or reporting bugs. Merely trying to knock a couple off the list. Is there a mechanism where we can discuss this offline?

[bdbaddog]

@jcbrill - scons-devel mailing list and #devel on our discord server are the typical places. Which I believe I've mentioned before. If not sorry for the oversight.

[bryanwweber]

@jcbrill is this issue resolved by #4131? Either by adding some configuration or just by default? I couldn't quite tell from the discussion. Thanks!

[jcbrill]

4131 does not address this particular issue. However, it is possible to use a specific toolset via recently added features. The possible solutions can be considered bridge mechanisms until first-class support can be added.

Brief aside: #4131 closed a loophole where when only an older Express version of Visual Studio was installed on a 64-bit host, the initialization of the default tools failed to detect the Express installation. When the Express installation was the only VS installation, a build would fail without an explicit MSVC_VERSION specification. The initial detection code and the code that determines the batch file to invoke were distinctly different. #4131 effectively refactored the code so that the initial detection and the batch file determination were using the same internal data structures (more or less). Some run-time code concerned with special-case handling could be removed as a result.

4106 describes three enhancements that each provide a user an "escape hatch" to make all features of the msvc batch files available within SCons until first-class support can be added (if ever). As of last night, two of the three methods have been implemented.

4106 enhancements:

1. MSVC_USE_SCRIPT_ARGS (implemented in #4102 2022-Mar-07)
2. MSVC_SCRIPT_ARGS (not implemented yet)
3. MSVC_USE_SETTINGS (implemented in #4125 2022-May-15)

Tangentially related are the changes to the SCons cache file (#4111 2022-May-15).

The recent discussion in #4149 covers some of the same ground as this issue with the notable exception that I appeared to have changed teams :) Nothing like arguing both sides of an issue...

Until first-class support for build tools and/or toolsets is available, there are a couple of low-energy methods now available to achieve the end result. Perhaps the best method (MSVC_SCRIPT_ARGS) is not available at this time but is shown below anyway. These methods work well in a controlled build environment.

It is possible to employ the first method (MSVC_USE_SCRIPT + MSVC_USE_SCRIPT_ARGS) and the third method (MSVC_USE_SETTINGS) as the returned values of user-defined function(s) defined in a script installed in one of the scons site folders. I am doing this for testing purposes. While beyond the scope of the discussion at hand, it is incredibly useful nonetheless.

A quick and dirty hack to the main branch providing MSVC_SCRIPT_ARGS functionality was used to demonstrate all three methods below.

The example uses the 14.16.27023 toolset installed in VS2022 Community (14.3) on a 64-bit host and producing 64-bit executables.

Using the quick-and-dirty hacked version, the following are shown below

• The SConstruct file used (SConstruct-3664)
• The output from running the batch file and the SConstruct file
• The batch file and batch file invocation used
• The SCons cache entry for methods 1 and 2

Questions and comments are always welcome.

Notes:

• --vcvars_ver= can contain a full (e.g., 14.16.27023) or partial (e.g., 14.16) toolset specification.
• A single - can be used as well: -vcvars_ver=

SConstruct-3664:

import sys

env_list = []

def make_environment(**kwargs):
    global env_list
    build_n = len(env_list) + 1
    build = '_build{:03d}'.format(build_n)
    print('Build:', build, kwargs, file=sys.stderr)
    VariantDir(build, '.', duplicate=0)
    env=Environment(**kwargs)
    env.Program(build + '/hello', [build + '/hello.c'])
    env_list.append(env)
    return env

# method 1: MSVC_USE_ARGS [not implemented yet]
env1 = make_environment(
    MSVC_VERSION ='14.3',
    MSVC_SCRIPT_ARGS = '--vcvars_ver=14.1',
    HOST_ARCH = 'amd64',
    TARGET_ARCH = 'amd64',
)

# method 2: MSVC_USE_SCRIPT and MSVC_USE_SCRIPT_ARGS [implemented]
env2 = make_environment(
    MSVC_VERSION ='14.3',  # generally a good idea (14.3 or 14.1?)
    MSVC_USE_SCRIPT = "C:\\Software\\MSVS-2022-143-Com\\VC\\Auxiliary\\Build\\vcvars64.bat",
    MSVC_USE_SCRIPT_ARGS = "--vcvars_ver=14.1",
    HOST_ARCH = 'amd64',   # unnecessary
    TARGET_ARCH = 'amd64', # unnecessary
)

# method 3: MSVC_USE_SETTINGS [implemented]
#           MSVC_USE_SETTINGS dictionary taken from the scons cache for method 2
env3 = make_environment(
    MSVC_VERSION ='14.3',  # generally a good idea (14.3 or 14.1?)
    MSVC_USE_SETTINGS = {
        "INCLUDE": [
          "C:\\Software\\MSVS-2017-141-Com\\VC\\Tools\\MSVC\\14.16.27023\\ATLMFC\\include",
          "C:\\Software\\MSVS-2017-141-Com\\VC\\Tools\\MSVC\\14.16.27023\\include",
          "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.6.1\\include\\um",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\ucrt",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\shared",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\um",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\winrt",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\cppwinrt"
        ],
        "LIB": [
          "C:\\Software\\MSVS-2017-141-Com\\VC\\Tools\\MSVC\\14.16.27023\\ATLMFC\\lib\\x64",
          "C:\\Software\\MSVS-2017-141-Com\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x64",
          "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.6.1\\lib\\um\\x64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.22000.0\\ucrt\\x64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.22000.0\\um\\x64"
        ],
        "LIBPATH": [
          "C:\\Software\\MSVS-2017-141-Com\\VC\\Tools\\MSVC\\14.16.27023\\ATLMFC\\lib\\x64",
          "C:\\Software\\MSVS-2017-141-Com\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x64",
          "C:\\Software\\MSVS-2017-141-Com\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x86\\store\\references",
          "C:\\Program Files (x86)\\Windows Kits\\10\\UnionMetadata\\10.0.22000.0",
          "C:\\Program Files (x86)\\Windows Kits\\10\\References\\10.0.22000.0",
          "C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319"
        ],
        "PATH": [
          "C:\\Software\\MSVS-2017-141-Com\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX64\\x64",
          "C:\\Software\\MSVS-2017-141-Com\\Common7\\IDE\\VC\\VCPackages",
          "C:\\Software\\MSVS-2017-141-Com\\Common7\\IDE\\CommonExtensions\\Microsoft\\TestWindow",
          "C:\\Software\\MSVS-2017-141-Com\\Common7\\IDE\\CommonExtensions\\Microsoft\\TeamFoundation\\Team Explorer",
          "C:\\Software\\MSVS-2017-141-Com\\MSBuild\\15.0\\bin\\Roslyn",
          "C:\\Software\\MSVS-2017-141-Com\\Team Tools\\Performance Tools\\x64",
          "C:\\Software\\MSVS-2017-141-Com\\Team Tools\\Performance Tools",
          "C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Common\\VSPerfCollectionTools\\\\x64",
          "C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Common\\VSPerfCollectionTools\\",
          "C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v10.0A\\bin\\NETFX 4.6.1 Tools\\x64\\",
          "C:\\Program Files (x86)\\Windows Kits\\10\\bin\\10.0.22000.0\\x64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\bin\\x64",
          "C:\\Software\\MSVS-2017-141-Com\\\\MSBuild\\15.0\\bin",
          "C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319",
          "C:\\Software\\MSVS-2017-141-Com\\Common7\\IDE\\",
          "C:\\Software\\MSVS-2017-141-Com\\Common7\\Tools\\",
          "C:\\Windows\\System32",
          "C:\\Windows\\System32\\Wbem",
          "C:\\Windows\\System32\\WindowsPowerShell\\v1.0",
          "C:\\Software\\MSVS-2017-141-Com\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin",
          "C:\\Software\\MSVS-2017-141-Com\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\Ninja"
        ],
        "VSCMD_ARG_app_plat": [
          "Desktop"
        ],
        "VCINSTALLDIR": [
          "C:\\Software\\MSVS-2017-141-Com\\VC\\"
        ],
        "VCToolsInstallDir": [
          "C:\\Software\\MSVS-2017-141-Com\\VC\\Tools\\MSVC\\14.16.27023\\"
        ]
      },
      HOST_ARCH = 'amd64',   # unnecessary
      TARGET_ARCH = 'amd64', # unnecessary
)

Output (manually modified for brevity):

Enabling scons cache ... 
Removing scons_msvc_cache.json ... 
Removing build folders ... 
Running scons ( 9:25:41.91) ...
scons: Reading SConscript files ...
Build: _build001 {'MSVC_VERSION': '14.3', 'MSVC_SCRIPT_ARGS': '--vcvars_ver=14.1', 'HOST_ARCH': 'amd64', 'TARGET_ARCH': 'amd64'}
Build: _build002 {'MSVC_VERSION': '14.3', 'MSVC_USE_SCRIPT': 'C:\\Software\\MSVS-2022-143-Com\\VC\\Auxiliary\\Build\\vcvars64.bat', 'MSVC_USE_SCRIPT_ARGS': '--vcvars_ver=14.1', 'HOST_ARCH': 'amd64', 'TARGET_ARCH': 'amd64'}
Build: _build003 {'MSVC_VERSION': '14.3', 'MSVC_USE_SETTINGS': {'***MANUALLY REMOVED***'}, 'HOST_ARCH': 'amd64', 'TARGET_ARCH': 'amd64'}
scons: done reading SConscript files.
scons: Building targets ...
scons: building associated VariantDir targets: _build001 _build002 _build003
cl /Fo_build003\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build003\hello.exe _build003\hello.obj
cl /Fo_build002\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build002\hello.exe _build002\hello.obj
cl /Fo_build001\hello.obj /c hello.c /nologo
hello.c
link /nologo /OUT:_build001\hello.exe _build001\hello.obj
scons: done building targets.
Scons finished ( 9:25:47.82)
Removing build folders ... 
Disabling scons cache ... 
Finished 

Batch file invocation: run_useargs 3664 1>_output-3664.txt 2>&1:

@set "$SCONS_ROOT=..\jbrill-msvc-useargs"
@if "%PYEXE%"=="" @call E:\Python\python-embed.bat 3.6.8 64
@echo Enabling scons cache ... 1>&2
@set "SCONS_CACHE_MSVC_CONFIG=1"
@echo Removing scons_msvc_cache.json ... 1>&2
@if exist %USERPROFILE%\scons_msvc_cache.json @del %USERPROFILE%\scons_msvc_cache.json 1>nul 2>nul
@echo Removing build folders ... 1>&2
@call :clean-dir
@echo Running scons (%TIME%) ...
@%PYEXE% %$SCONS_ROOT%\scripts\scons.py --sconstruct=sconstruct-%1 %2
@echo Scons finished (%TIME%)
@echo Removing build folders ... 1>&2
@call :clean-dir
@echo Disabling scons cache ... 1>&2
@set "SCONS_CACHE_MSVC_CONFIG="
@echo Finished 1>&2
@set "$SCONS_ROOT="
@goto :eof

:clean-dir
    @for /d %%G in (".\_build*") do @(
        @del /f /s /q "%%~G" 1>nul 2>nul
        @rmdir /q "%%~G" 1>nul 2>nul
    ) 
    @if exist .sconsign.dblite @del .sconsign.dblite 1>nul 2>nul
    @goto :eof

When enabled, the SCons cache contains the script, arguments, and output for each unique invocation. In the example shown below, key contains a two-element list containing the script and the script argument and data contains the processed msvc batch file environment used by SCons.

The key can be used to define SCONS_USE_SCRIPT and SCONS_USE_SCRIPT_ARGS.

The data can be used to define SCONS_USE_SETTINGS.

SCons cache entry for method 1 and method 2:

...
  {
    "key": [
      "C:\\Software\\MSVS-2022-143-Com\\VC\\Auxiliary\\Build\\vcvars64.bat",
      "--vcvars_ver=14.1"
    ],
    "data": {
      "INCLUDE": [
        "C:\\Software\\MSVS-2022-143-Com\\VC\\Tools\\MSVC\\14.16.27023\\ATLMFC\\include",
        "C:\\Software\\MSVS-2022-143-Com\\VC\\Tools\\MSVC\\14.16.27023\\include",
        "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\include\\um",
        "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.22000.0\\ucrt",
        "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\shared",
        "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\um",
        "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\winrt",
        "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.22000.0\\\\cppwinrt"
      ],
      "LIB": [
        "C:\\Software\\MSVS-2022-143-Com\\VC\\Tools\\MSVC\\14.16.27023\\ATLMFC\\lib\\x64",
        "C:\\Software\\MSVS-2022-143-Com\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x64",
        "C:\\Program Files (x86)\\Windows Kits\\NETFXSDK\\4.8\\lib\\um\\x64",
        "C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.22000.0\\ucrt\\x64",
        "C:\\Program Files (x86)\\Windows Kits\\10\\\\lib\\10.0.22000.0\\\\um\\x64"
      ],
      "LIBPATH": [
        "C:\\Software\\MSVS-2022-143-Com\\VC\\Tools\\MSVC\\14.16.27023\\ATLMFC\\lib\\x64",
        "C:\\Software\\MSVS-2022-143-Com\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x64",
        "C:\\Software\\MSVS-2022-143-Com\\VC\\Tools\\MSVC\\14.16.27023\\lib\\x86\\store\\references",
        "C:\\Program Files (x86)\\Windows Kits\\10\\UnionMetadata\\10.0.22000.0",
        "C:\\Program Files (x86)\\Windows Kits\\10\\References\\10.0.22000.0",
        "C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319"
      ],
      "PATH": [
        "C:\\Software\\MSVS-2022-143-Com\\VC\\Tools\\MSVC\\14.16.27023\\bin\\HostX64\\x64",
        "C:\\Software\\MSVS-2022-143-Com\\Common7\\IDE\\VC\\VCPackages",
        "C:\\Software\\MSVS-2022-143-Com\\Common7\\IDE\\CommonExtensions\\Microsoft\\TestWindow",
        "C:\\Software\\MSVS-2022-143-Com\\Common7\\IDE\\CommonExtensions\\Microsoft\\TeamFoundation\\Team Explorer",
        "C:\\Software\\MSVS-2022-143-Com\\MSBuild\\Current\\bin\\Roslyn",
        "C:\\Software\\MSVS-2022-143-Com\\Team Tools\\Performance Tools\\x64",
        "C:\\Software\\MSVS-2022-143-Com\\Team Tools\\Performance Tools",
        "C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Common\\VSPerfCollectionTools\\vs2019\\\\x64",
        "C:\\Program Files (x86)\\Microsoft Visual Studio\\Shared\\Common\\VSPerfCollectionTools\\vs2019\\",
        "C:\\Program Files (x86)\\Microsoft SDKs\\Windows\\v10.0A\\bin\\NETFX 4.8 Tools\\x64\\",
        "C:\\Program Files (x86)\\Windows Kits\\10\\bin\\10.0.22000.0\\\\x64",
        "C:\\Program Files (x86)\\Windows Kits\\10\\bin\\\\x64",
        "C:\\Software\\MSVS-2022-143-Com\\\\MSBuild\\Current\\Bin\\amd64",
        "C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319",
        "C:\\Software\\MSVS-2022-143-Com\\Common7\\IDE\\",
        "C:\\Software\\MSVS-2022-143-Com\\Common7\\Tools\\",
        "C:\\Windows\\System32",
        "C:\\Windows\\System32\\Wbem",
        "C:\\Windows\\System32\\WindowsPowerShell\\v1.0",
        "C:\\Software\\MSVS-2022-143-Com\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin",
        "C:\\Software\\MSVS-2022-143-Com\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\Ninja",
        "C:\\Software\\MSVS-2022-143-Com\\Common7\\IDE\\VC\\Linux\\bin\\ConnectionManagerExe"
      ],
      "VSCMD_ARG_app_plat": [
        "Desktop"
      ],
      "VCINSTALLDIR": [
        "C:\\Software\\MSVS-2022-143-Com\\VC\\"
      ],
      "VCToolsInstallDir": [
        "C:\\Software\\MSVS-2022-143-Com\\VC\\Tools\\MSVC\\14.16.27023\\"
      ]
    }
  }
...

[bryanwweber]

Thanks for the detailed update @jcbrill!

[jcbrill]

Errata: references to MSVC_USE_ARGS in the post above will be MSVC_SCRIPT_ARGS when implemented. Will edit the above shortly.

[mwichmann]

So struggling what to do with this issue. The original description suggests a bug: you install the 16.x whatever-we-call it (product? framework? something?), and then use the installer to select three toolsets, 14.0, 14.1 and 14.2. We now have two toolsets that don't have the naturally matching product, but rather all under one product. If we just don't know how to deal with "older" toolsets at the moment, we might flip this to an "enhancement" request, but the point seems to be that 14.0 and 14.2 can be selected, but not 14.1; a recently linked issue suggests something similar where 14.3 and 14.2 can be selected but not 14.1. Sorry by now the discussion has gotten so long I'm asking if there's a TL;DR reset that can be added here....

[bdbaddog]

@jcbrill and @mwichmann Let me recap and correct me if wrong. Due to changes in MSVS, you can install other versions of MSVC (the compiler) under MSVS so (for example) 2031 MSVC compilers could be installed under MSVS 2041

Currently we lack a way to request this without changing the meaning of existing MSVC_ variables in a possibly incompatible way. So given that, this is effectively a new feature request, thus enhancement would be correct.

Perhap MSVC_TOOLSET_VERSION could be added, which if not defined msvc tool would only select minor versions of the request MSVC_VERSION, and if defined and MSVC_VERSION is not defined would select the newest version of that toolset installed in any installled MSVS, and if both installed then the toolset isntalled in the MSVS install matchin MSVC_VERSION?

Does that make sense?

[jcbrill]

The original description suggests a bug: you install the 16.x whatever-we-call it (product? framework? something?), and then use the installer to select three toolsets, 14.0, 14.1 and 14.2. We now have two toolsets that don't have the naturally matching product, but rather all under one product.

More or less correct.

Being pedantic: we now have one toolset that doesn't have naturally matching products, but rather all but 14.0 under one product. While 14.0 can be installed in 2017 (14.1), 2019 (14.2), and 2022 (14.3), the 14.0 tools are installed to their own MSVS 2015 folder (if not already installed) and populate the registry like a normal 2015 installation. For 2017-2019, there is a special batch file that queries the registry for the 2015 (14.0) location. This is why 14.0 and 14.2 worked but 14.1 did not. 14.0 was found through the registry and 14.2 was found via vswhere.

If we just don't know how to deal with "older" toolsets at the moment, we might flip this to an "enhancement" request, but the point seems to be that 14.0 and 14.2 can be selected, but not 14.1; a recently linked issue suggests something similar where 14.3 and 14.2 can be selected but not 14.1. Sorry by now the discussion has gotten so long I'm asking if there's a TL;DR reset that can be added here.

Ignore 14.0 for the moment. Each version for 2017 and later allows earlier build tools to be installed. So 2022 (14.3) allows 14.2, 14.1, and 14.0 to be installed. 14.0 is found via the registry. 14.2 and 14.1 are inaccessible from the 2022 (14.3) "container". Basically the "newest" toolset is the current definition of MSVC_VERSION and the "oldest" toolset is its own installation. There are N-2 inaccessible toolsets possible.

Currently we lack a way to request this without changing the meaning of existing MSVC_ variables in a possibly incompatible way. So given that, this is effectively a new feature request, thus enhancement would be correct.

Is it a bug or a enhancement? Its easy to argue both sides. I used to be in the bug camp but am tilting slightly toward the enhancement camp. The fundamental problem is that for 2015 and earlier, there was a 1:1 mapping between MSVC_VERSION and product. For 2015 and later, the MSVC_VERSION is really the product in which the version was introduced rather than all products that may contain the version.

The current data structures coupled with legacy implications aren't built to handle the change in semantics. For this reason, I would lean towards enhancement.

PR #4149 is exactly the same same as this issue.

Perhap MSVC_TOOLSET_VERSION could be added, which if not defined msvc tool would only select minor versions of the request MSVC_VERSION, and if defined and MSVC_VERSION is not defined would select the newest version of that toolset installed in any installled MSVS, and if both installed then the toolset isntalled in the MSVS install matchin MSVC_VERSION?

This is possible. I am preparing to make a working user-space solution (scons site) public shortly which could be a basis for discussion and is how I test some concepts prior to introduction into SCons.

While the concepts are pretty straightforward, the msvc installation needs to be walked for versions/hosts/targets. As more "degrees-of-freedom" are added (e.g., preview/release, community/professional/enterprise/buildtools, 2022/2019/2017), validation/consistency between newly added variables and legacy variables turns into a non-trivial problem.

The easiest solution, while possibly distasteful, involves defining new variables and deprecating MSVC_VERSION as a user defined variable. Someday, this would allow the MSVC_VERSION to be redefined internally as the toolset rather than the product. MSVC_VERSION is used internally for determining options, formats, etc in a couple of places. In these cases, it probably should be using the toolset rather than the product definition.

[jcbrill]

@bdbaddog and @mwichmann

Perhaps the best argument for "enhancement" is that even when using msvc from the command-line, it takes two pieces of information to use any "non-native" toolset: the path to the batch file (the product) and the toolset version argument.

For example:

# VS2022 Community with default v143 build tools
C:\Software\MSVS-2022-143-Com\VC\Auxiliary\Build\vcvarsall.bat amd64

#VS2022 Community with v142 build tools
C:\Software\MSVS-2022-143-Com\VC\Auxiliary\Build\vcvarsall.bat amd64 -vcvars_ver=14.2

#VS2022 Community with v141 build tools
C:\Software\MSVS-2022-143-Com\VC\Auxiliary\Build\vcvarsall.bat amd64 -vcvars_ver=14.1

#VS2022 Community with v140 build tools
C:\Software\MSVS-2022-143-Com\VC\Auxiliary\Build\vcvarsall.bat amd64 -vcvars_ver=14.0

Note: it may be possible that only an older toolset is installed which, while possibly confusing, could make the default toolset different from the newest installed with the product (e.g., only the v142 toolset is installed in 2022).

If it only took one piece of information, there would not be a problem. If vswhere gave us toolsets rather than products it would certainly be easier.

As @bdbaddog pointed out, MSVC_SCRIPT_ARGS would be the quickest and lowest-energy solution.

# VS2022 Community with default v143 build tools
Environment(MSVC_VERSION='14.3')

#VS2022 Community with v142 build tools
Environment(MSVC_VERSION='14.3', MSVC_SCRIPT_ARGS='-vcvars_ver=14.2')

#VS2022 Community with v141 build tools
Environment(MSVC_VERSION='14.3', MSVC_SCRIPT_ARGS='-vcvars_ver=14.1')

#VS2022 Community with v140 build tools
Environment(MSVC_VERSION='14.3', MSVC_SCRIPT_ARGS='-vcvars_ver=14.0')

[jcbrill]

Lengthy informational/educational post ahead...

VS2017 introduced side-by-side toolset versions. Multiple toolset versions could installed simultaneously and were made available to users via the batch file argument -vcvars_ver. All of the VS2017 toolset version numbers begin with the prefix 14.1. The VS2017 installer also provided a checkbox to install the v140 tools. The batch file argument -vcvars_ver=14.0 could be used from the command-line to build with the v140 toolsets via VS2017.

3265 is an enhancement issue that would provide the ability to specify a full toolset version.

From the outside, it appears that the VS2015 build tools are a component of a VS2017 installation. This is not the case. The installer is installing the VS2015 build tools as a standalone installation separate from the VS2017 installation. The VS2015 and VS2017 installations are independent: it is tantamount to manually downloading and installing the VS2015 build tools and then downloading and installing VS2017 without checking the v140 options box. There is a special VS2017 batch file for the v140 tools that queries the registry for the VS2015 installation path.

SCons correctly detects VS2017 via vswhere and VS2015 via the registry. When installing the v140 build tools via VS2017, there is an illusion that the 14.0 builds are performed via the VS2017 installation. This is not the case. The environment paths for 14.0 do not contain any references to the VS2017 installation; while the v140 build tools via the VS2017 batch file will add ancillary paths for the VS2017 installation.

The ramifications of the separate product installations did not appear until VS2019 was released. All of the VS2019 toolset version numbers begin with the prefix 14.2. The VS2019 installer allows the v141 and v140 toolsets to be installed. The v141 toolsets are installed in the VS2019 installation folder while the v140 toolsets are installed as a separate product as described above. SCons does not have first class support for specifying toolset versions starting in VS2017. With the introduction of VS2019, for the first time a major toolset (v141) was inaccessible based on a single installation. An additional issue is that it "appears" that 14.0 works while 14.1 does not. From a user perspective this is true. In reality, Microsoft changed the rules and SCons has not caught up yet.

Using 14.0 based on installing the v140 build tools via VS2017/VS2019/VS2022 is somewhat misleading: the installation of VS2015 is a stand-alone product and is not using the VS2017/VS2019/VS2022 installation. Looked at this way, SCons has never worked in the manner in which it appears from the outside due to the lack of toolset version support and the manner in which v140 is installed.

To illustrate the subtle ramifications, VS2022 Preview was installed in a VMWare virtual machine with the v140 build tools. Three (3) builds were attempted: 14.0 succeeds, (2) 14.3 fails due to not detecting the preview version, and (3) the v140 build tools passed as an argument to the VS2022 preview batch file via MSVC_USE_SCRIPT and MSVC_USE_SCRIPT_ARGS. Had the v140 build tools been a component of the VS2022 installation, the 14.0 build would have failed. As it is an independent installation, the build succeeds.

Shown below are: (1) the SConstruct file used, (2) the output fragments from running SCons, (3) the SCons cache file for the two successful builds, and (4) fragments from the MSCommon debug log. The run was performed with the current "warnfix" branch and makes use of MSVC_NOTFOUND_POLICY.

Note the difference in the resulting PATH components in the SCons cache file between the two successful builds.

File contents:

1. SConstruct file

   Notes:

   • VS2015: MSVC_VERSION='14.0'
   • VS2022: MSVC_VERSION='14.3'
   • v140 via VS2022: MSVC_USE_SCRIPT= and MSVC_USE_SCRIPT_ARGS=

   import SCons.Tool.MSCommon.vc as vc
   
   build_n = 0
   env_list = []
   def environment(**kwargs):
      global build_n
      global env_list
      build_n += 1
      build = '_build{:03d}'.format(build_n)
      print('\nBuild:', build, kwargs)
      VariantDir(build, '.', duplicate=0)
      env=Environment(**kwargs)
      env.Program(build + '/hello', [build + '/hello.c'])
      env_list.append(env)
      return env
   
   # SCons VCS
   print("\nSCons Detected VCS =", vc.get_installed_vcs())
   
   # VS2015
   environment(MSVC_VERSION='14.0')
   
   # VS2022 Preview
   try:
      environment(MSVC_VERSION='14.3', MSVC_NOTFOUND_POLICY='Error')
   except vc.MSVCVersionNotFound as e:
      print('\nMSVC_VERSION = {}: {}'.format('14.3', str(e)))
   
   # VS2022 Preview 14.0: amd64 -vcvars_ver=14.0 
   environment(
      MSVC_VERSION='14.0', 
      MSVC_USE_SCRIPT=r'C:\Program Files\Microsoft Visual Studio\2022\Preview\VC\Auxiliary\Build\vcvarsall.bat',
      MSVC_USE_SCRIPT_ARGS=['amd64', '-vcvars_ver=14.0']
   )
   
   print()

2. Output Fragment

   Notes:

   • MSVC_VERSION='14.0' builds successfully.
   • MSVC_VERSION='14.3' is not detected by SCons.
   • MSVC_USE_SCRIPT= and MSVC_USE_SCRIPT_ARGS= for the v140 tools via VS2022 Preview builds successfully.

   ...
   scons: Reading SConscript files ...
   
   SCons Detected VCS = ['14.0']
   
   Build: _build001 {'MSVC_VERSION': '14.0'}
   
   Build: _build002 {'MSVC_VERSION': '14.3', 'MSVC_NOTFOUND_POLICY': 'Error'}
   
   MSVC_VERSION = 14.3: MSVC version '14.3' was not found.
    Visual Studio C/C++ compilers may not be set correctly.
    Installed versions are: ['14.0']
   
   Build: _build003 {'MSVC_VERSION': '14.0', 'MSVC_USE_SCRIPT': 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Auxiliary\\Build\\vcvarsall.bat', 'MSVC_USE_SCRIPT_ARGS': ['amd64', '-vcvars_ver=14.0']}
   
   scons: done reading SConscript files.
   scons: Building targets ...
   scons: building associated VariantDir targets: _build001 _build002 _build003
   cl /Fo_build003\hello.obj /c hello.c /nologo
   hello.c
   link /nologo /OUT:_build003\hello.exe _build003\hello.obj
   cl /Fo_build001\hello.obj /c hello.c /nologo
   hello.c
   link /nologo /OUT:_build001\hello.exe _build001\hello.obj
   scons: done building targets.
   ...

3. SCons cache

   Notes:

   • The VS2015 installation is independent of the VS2022 installation.
   • The first entry is for MSVC_VERSION='14.0'. \ The PATH components refer to the VS2015 installation and do not include any VS2022 components.
   • The second entry is for the VS2022 preview script and building with the '14.0' tools. \ The PATH components refer to the VS2015 installation and include VS2022 components.

   [
    {
      "key": [
        "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\vcvarsall.bat",
        "amd64"
      ],
      "data": {
        "INCLUDE": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\INCLUDE",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\ATLMFC\\INCLUDE",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\ucrt",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\shared",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\um",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\winrt"
        ],
        "LIB": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\LIB\\amd64",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\ATLMFC\\LIB\\amd64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.19041.0\\ucrt\\x64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.19041.0\\um\\x64"
        ],
        "LIBPATH": [
          "C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\LIB\\amd64",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\ATLMFC\\LIB\\amd64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\UnionMetadata",
          "C:\\Program Files (x86)\\Windows Kits\\10\\References",
          "\\Microsoft.VCLibs\\14.0\\References\\CommonConfiguration\\neutral"
        ],
        "PATH": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\BIN\\amd64",
          "C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\Common7\\IDE",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\Common7\\Tools",
          "C:\\Program Files (x86)\\Windows Kits\\10\\bin\\x64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\bin\\x86",
          "C:\\Windows\\System32",
          "C:\\Windows\\System32\\Wbem",
          "C:\\Windows\\System32\\WindowsPowerShell\\v1.0"
        ],
        "VSCMD_ARG_app_plat": [],
        "VCINSTALLDIR": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\"
        ],
        "VCToolsInstallDir": []
      }
    },
    {
      "key": [
        "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\VC\\Auxiliary\\Build\\vcvarsall.bat",
        "amd64 -vcvars_ver=14.0"
      ],
      "data": {
        "INCLUDE": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\ATLMFC\\include",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\include",
          "C:\\Program Files (x86)\\Windows Kits\\10\\include\\10.0.19041.0\\ucrt",
          "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.19041.0\\\\shared",
          "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.19041.0\\\\um",
          "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.19041.0\\\\winrt",
          "C:\\Program Files (x86)\\Windows Kits\\10\\\\include\\10.0.19041.0\\\\cppwinrt"
        ],
        "LIB": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\ATLMFC\\lib\\amd64",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\lib\\amd64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\lib\\10.0.19041.0\\ucrt\\x64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\\\lib\\10.0.19041.0\\\\um\\x64"
        ],
        "LIBPATH": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\ATLMFC\\lib\\amd64",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\lib\\amd64",
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\lib\\store\\references",
          "C:\\Program Files (x86)\\Windows Kits\\10\\UnionMetadata\\10.0.19041.0",
          "C:\\Program Files (x86)\\Windows Kits\\10\\References\\10.0.19041.0",
          "C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319"
        ],
        "PATH": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\bin\\amd64",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\Common7\\IDE\\CommonExtensions\\Microsoft\\TestWindow",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\Common7\\IDE\\CommonExtensions\\Microsoft\\TeamFoundation\\Team Explorer",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\MSBuild\\Current\\bin\\Roslyn",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\Team Tools\\Performance Tools\\x64",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\Team Tools\\Performance Tools",
          "C:\\Program Files (x86)\\Windows Kits\\10\\bin\\10.0.19041.0\\\\x64",
          "C:\\Program Files (x86)\\Windows Kits\\10\\bin\\\\x64",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\\\MSBuild\\Current\\Bin\\amd64",
          "C:\\Windows\\Microsoft.NET\\Framework64\\v4.0.30319",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\Common7\\IDE\\",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\Common7\\Tools\\",
          "C:\\Windows\\System32",
          "C:\\Windows\\System32\\Wbem",
          "C:\\Windows\\System32\\WindowsPowerShell\\v1.0",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\CMake\\bin",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\Common7\\IDE\\CommonExtensions\\Microsoft\\CMake\\Ninja",
          "C:\\Program Files\\Microsoft Visual Studio\\2022\\Preview\\Common7\\IDE\\VC\\Linux\\bin\\ConnectionManagerExe"
        ],
        "VSCMD_ARG_app_plat": [
          "Desktop"
        ],
        "VCINSTALLDIR": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\"
        ],
        "VCToolsInstallDir": [
          "C:\\Program Files (x86)\\Microsoft Visual Studio 14.0\\VC\\"
        ]
      }
    }

4. MSCommon Debug Fragment

   Notes:

   • The VS2022 Preview installation is not detected by SCons.
   • The VS2015 build tools installation is detected by SCons via the registry.

   00000ms:MSCommon/vc.py:get_installed_vcs#971: trying to find VC 14.3
   00015ms:MSCommon/vc.py:find_vc_pdir_vswhere#705: VSWHERE: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe
   00015ms:MSCommon/vc.py:find_vc_pdir_vswhere#710: running: ['C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vswhere.exe', '-version', '[17.0, 18.0)', '-property', 'installationPath']
   00031ms:MSCommon/vc.py:find_vc_pdir_vswhere#710: running: ['C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vswhere.exe', '-version', '[17.0, 18.0)', '-products', 'Microsoft.VisualStudio.Product.BuildTools', '-property', 'installationPath']
   00062ms:MSCommon/vc.py:find_vc_pdir#764: no VC found for version '14.3'
   00062ms:MSCommon/vc.py:find_vc_pdir#780: no VC registry key ''
   00062ms:MSCommon/vc.py:get_installed_vcs#981: return None for ver 14.3
   00062ms:MSCommon/vc.py:get_installed_vcs#971: trying to find VC 14.2
   00062ms:MSCommon/vc.py:find_vc_pdir_vswhere#705: VSWHERE: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe
   00062ms:MSCommon/vc.py:find_vc_pdir_vswhere#710: running: ['C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vswhere.exe', '-version', '[16.0, 17.0)', '-property', 'installationPath']
   00094ms:MSCommon/vc.py:find_vc_pdir_vswhere#710: running: ['C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vswhere.exe', '-version', '[16.0, 17.0)', '-products', 'Microsoft.VisualStudio.Product.BuildTools', '-property', 'installationPath']
   00124ms:MSCommon/vc.py:find_vc_pdir#764: no VC found for version '14.2'
   00124ms:MSCommon/vc.py:find_vc_pdir#780: no VC registry key ''
   00124ms:MSCommon/vc.py:get_installed_vcs#981: return None for ver 14.2
   00124ms:MSCommon/vc.py:get_installed_vcs#971: trying to find VC 14.1
   00124ms:MSCommon/vc.py:find_vc_pdir_vswhere#705: VSWHERE: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe
   00124ms:MSCommon/vc.py:find_vc_pdir_vswhere#710: running: ['C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vswhere.exe', '-version', '[15.0, 16.0)', '-property', 'installationPath']
   00140ms:MSCommon/vc.py:find_vc_pdir_vswhere#710: running: ['C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vswhere.exe', '-version', '[15.0, 16.0)', '-products', 'Microsoft.VisualStudio.Product.BuildTools', '-property', 'installationPath']
   00172ms:MSCommon/vc.py:find_vc_pdir#764: no VC found for version '14.1'
   00172ms:MSCommon/vc.py:find_vc_pdir#780: no VC registry key ''
   00172ms:MSCommon/vc.py:get_installed_vcs#981: return None for ver 14.1
   00172ms:MSCommon/vc.py:get_installed_vcs#971: trying to find VC 14.1Exp
   00172ms:MSCommon/vc.py:find_vc_pdir_vswhere#705: VSWHERE: C:\Program Files (x86)\Microsoft Visual Studio\Installer\vswhere.exe
   00172ms:MSCommon/vc.py:find_vc_pdir_vswhere#710: running: ['C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vswhere.exe', '-version', '[15.0, 16.0)', '-products', 'Microsoft.VisualStudio.Product.WDExpress', '-property', 'installationPath']
   00188ms:MSCommon/vc.py:find_vc_pdir#764: no VC found for version '14.1Exp'
   00188ms:MSCommon/vc.py:find_vc_pdir#780: no VC registry key ''
   00188ms:MSCommon/vc.py:get_installed_vcs#981: return None for ver 14.1Exp
   00188ms:MSCommon/vc.py:get_installed_vcs#971: trying to find VC 14.0
   00188ms:MSCommon/vc.py:find_vc_pdir#782: found VC in registry: C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\
   00188ms:MSCommon/vc.py:get_installed_vcs#975: found VC 14.0
   00188ms:MSCommon/vc.py:get_host_target#501: HOST_ARCH:None
   00234ms:MSCommon/vc.py:get_host_target#509: TARGET_ARCH:None
   00234ms:MSCommon/vc.py:_check_cl_exists_in_vc_dir#869: host_platform amd64, target_platform None host_target_list [('amd64', 'amd64'), ('x86', 'amd64'), ('amd64', 'x86'), ('x86', 'x86'), ('amd64', 'arm'), ('x86', 'arm'), ('x86', 'ia64')]
   00234ms:MSCommon/vc.py:_check_cl_exists_in_vc_dir#919: host platform amd64, target platform amd64 for version 14.0
   00234ms:MSCommon/vc.py:_check_cl_exists_in_vc_dir#931: checking for cl.exe at C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\cl.exe
   00234ms:MSCommon/vc.py:_check_cl_exists_in_vc_dir#934: found cl.exe

[bryanwweber]

@mwichmann The original description suggests a bug

@bdbaddog Currently we lack a way to request this without changing the meaning of existing MSVC_ variables in a possibly incompatible way. So given that, this is effectively a new feature request, thus enhancement would be correct.

@jcbrill From a user perspective this is true. In reality, Microsoft changed the rules and SCons has not caught up yet.

As a user of SCons, this presents as a bug :smile: I can see from a developer perspective, though, how this is an enhancement request. Please let me know if I can be of any assistance in resolving this!

[jcbrill]

@bryanwweber In the short-term, would the following work for your use case?

# VS2019 using v141 toolset
Environment(MSVC_VERSION='14.2', MSVC_SCRIPT_ARGS='-vcvars_ver=14.1')

Passing user-defined additional script argument(s) is on the immediate short list...

[bryanwweber]

@jcbrill If it allows me to select an installed toolset, then that would be fine 😊

[jcbrill]

Potential notes of interest...

The msvc "default" toolset is not necessarily the latest toolset installed. This has implications when a toolset version is specified using only one minor digit (e.g., MSVC_TOOLSET_VERSION='14.3' or MSVC_SCRIPT_ARGS='-vcvars_ver=14.3').

In current code exploration for implementation options, explicitly defining MSVC_TOOLSET_VERSION=None will return the same toolset that the msvc batch files would return. When using MSVC_SCRIPT_ARGS, the toolset specification should be omitted entirely.

Local installation and summary results for exploration tests:

VS2022\VC\Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt
    14.31.31103

VS2022\VC\Auxiliary\Build\Microsoft.VCToolsVersion.default.txt
    14.32.31326

Summary:

    14.31.31103   Environment()
    14.31.31103   Environment(MSVC_TOOLSET_VERSION=None)

    14.32.31326*  Environment(MSVC_TOOLSET_VERSION='14.3')
    14.32.31326*  Environment(MSVC_SCRIPT_ARGS=['-vcvars_ver=14.3'])

    14.31.31103   Environment(MSVC_TOOLSET_VERSION='14.31')
    14.31.31103   Environment(MSVC_SCRIPT_ARGS=['-vcvars_ver=14.31'])

    14.32.31326   Environment(MSVC_TOOLSET_VERSION='14.32')
    14.32.31326   Environment(MSVC_SCRIPT_ARGS=['-vcvars_ver=14.32'])

VS2022\Common7\Tools\vsdevcmd\ext\vcvars.bat

@echo     -vcvars_ver=version : Version of VC++ Toolset to select
@echo            ** [Default]   : If -vcvars_ver=version is NOT specified, the toolset specified by
@echo                             [VSInstallDir]\VC\Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt will be used.
@echo            ** 14.0        : VS 2015 (v140) VC++ Toolset (installation of the v140 toolset is a prerequisite)
@echo            ** 14.xx       : VS 2017 or VS 2019 VC++ Toolset, if that version is installed on the system under 
@echo                             [VSInstallDir]\VC\MSVC\Tools\[version].  Where '14.xx' specifies a partial 
@echo                             [version]. The latest [version] directory that matches the specified value will 
@echo                             be used.
@echo            ** 14.xx.yyyyy : VS 2017 or VS 2019 VC++ Toolset, if that version is installed on the system under 
@echo                             [VSInstallDir]\VC\MSVC\Tools\[version]. Where '14.xx.yyyyy' specifies an 
@echo                             exact [version] directory to be used.
@echo            ** 14.xx.VV.vv : VS 2019 C++ side-by-side toolset package identity alias, if the SxS toolset has been installed on the system.
@echo                             Where '14.xx.VV.vv' corresponds to a SxS toolset
@echo                                 VV = VS Update Major Version (e.g. "16" for VS 2019 v16.9)
@echo                                 vv = VS Update Minor version (e.g. "9" for VS 2019 v16.9)
@echo                             Please see [VSInstallDir]\VC\Auxiliary\Build\[version]\Microsoft.VCToolsVersion.[version].txt for mapping of 
@echo                             SxS toolset to [VSInstallDir]\VC\MSVC\Tools\ directory. 

VS2022 Determine default toolset version:

@REM Add MSVC
set "__VCVARS_DEFAULT_CONFIG_FILE=%VCINSTALLDIR%Auxiliary\Build\Microsoft.VCToolsVersion.default.txt"

@REM We will "fallback" to Microsoft.VCToolsVersion.default.txt (latest) if Microsoft.VCToolsVersion.v143.default.txt does not exist.
if EXIST "%VCINSTALLDIR%Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt" (
    if "%VSCMD_DEBUG%" GEQ "2" @echo [DEBUG:ext\%~nx0] Microsoft.VCToolsVersion.v143.default.txt was found.
    set "__VCVARS_DEFAULT_CONFIG_FILE=%VCINSTALLDIR%Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt"

) else (
    if "%VSCMD_DEBUG%" GEQ "1" @echo [DEBUG:ext\%~nx0] Microsoft.VCToolsVersion.v143.default.txt was not found. Defaulting to 'Microsoft.VCToolsVersion.default.txt'.
)

[mwichmann]

This stuff is mind-bending...

VS2022\VC\Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt  :    14.31.31103
VS2022\VC\Auxiliary\Build\Microsoft.VCToolsVersion.default.txt           :    14.32.31326

but

@echo            ** [Default]   : If -vcvars_ver=version is NOT specified, the toolset specified by
@echo                             [VSInstallDir]\VC\Auxiliary\Build\Microsoft.VCToolsVersion.v143.default.txt will be used.

So it prefers v143.default.txt to default.txt? Does this happen automatically, or is this to be able to force "I just installed a new toollset, but I want the previous one to remain the default until I decide to switch"?

[jcbrill]

So it prefers v143.default.txt to default.txt? Does this happen automatically, or is this to be able to force "I just installed a new toollset, but I want the previous one to remain the default until I decide to switch"?

It appears to happen automatically. The "14.32.XXXXX" toolsets appear to be those from VS2022 preview which may explain the difference. It looks like the thinking might be: use the latest "release" default toolset and you have to explicitly specify that you want a later (preview?) toolset. I could be wrong.

Where it gets sticky is that when specifying "14.3", the toolset folders are sorted in descending order and the first match wins. In this case, the latest/preview version.

Starting in VS2019, the buildtools (e.g., v142, v143) default file is used if it exists and the fallback is the "regular" default file. In VS2017, only the default text file exists.

Sigh...

[mwichmann]

Yeah, I have this too, on my "lots of stuff installed" system. It's not the Preview bits causing it, they're in a different path. I have these:

• 2022 default.txt: 14.32.31263
• 2022 v143.default.txt: 14.31.31103
• 2022Pre default.txt: 14.33.31517
• 2022Pre v143.default.txt: 14.33.31517
• 2019 default.txt: 14.29.30133
• 2019 v142.default.txt: 14.29.30133

for whatever that means....

[jcbrill]

It's not the Preview bits causing it, they're in a different path

You are correct. I looked in the wrong folder: preview toolset versions appear to be "14.33" (14.33.31517).

I can't explain why v143.default and default are different.

[jcbrill]

What is strange is that the VS2022 installer shows 14.30 (17.0), 14.31 (17.1), and 14.32 (17.2) and that 17.2.4 is installed.

[jcbrill]

@mwichmann the msvc cache proved to be an invaluable debugging aid in determining toolset version and sdk version used for a build and that they match what was expected.

[jcbrill]

@bryanwweber With the recent merge of #4174, there are now two methods in the main branch which allow an installed toolset to be used for a build. However, there are limitations...

At present, there are two pieces of information required in order to use a specific toolset version: the msvc product (i.e., MSVC_VERSION) and the msvc toolset version (MSVC_TOOLSET_VERSION).

The msvc toolset version is not used during selection.

If MSVC_VERSION is not specified and MSVC_TOOLSET_VERSION is specified, then the toolset version applies to the default msvc product (i.e., the newest product installed). If the toolset version requested is not found/installed, then a MSVCToolsetVersionNotFound exception is raised.

The internal ms readme file (SCons/Tool/MSCommon/Readme.rst) has been updated: \ https://github.com/SCons/scons/tree/master/SCons/Tool/MSCommon

In particular, the default msvc toolset version section as discussed above may be relevant: \ https://github.com/SCons/scons/tree/master/SCons/Tool/MSCommon#62default-toolset-version

Examples:

# VS2022 with 14.2X.YYYYY toolset
env = Environment(MSVC_VERSION='14.3', MSVC_TOOLSET_VERSION='14.2')

# Default VS with 14.2X.YYYYY toolset
env = Environment(MSVC_TOOLSET_VERSION='14.2')

# Default VS with 14.32.YYYYY toolset
env = Environment(MSVC_TOOLSET_VERSION='14.32')

# VS2019 with specific toolset which is not the latest
env = Environment(MSVC_VERSION='14.2', MSVC_TOOLSET_VERSION='14.28.29333')

With an exception raised when the msvc toolset version is not found, it is possible to "query" by msvc version and msvc toolset version in user-space for a desired toolset.

Since this appears useful from an end-user standpoint, an experimental function msvc_query_version_toolset was added that is a proxy for using the msvc toolset version for selection.

The docstring is:

def msvc_query_version_toolset(version=None, prefer_newest=True):
    """
    Returns an msvc version and a toolset version given a version
    specification.

    This is an EXPERIMENTAL proxy for using a toolset version to perform
    msvc instance selection.  This function will be removed when
    toolset version is taken into account during msvc instance selection.

    Search for an installed Visual Studio instance that supports the
    specified version.

    When the specified version contains a component suffix (e.g., Exp),
    the msvc version is returned and the toolset version is None. No
    search if performed.

    When the specified version does not contain a component suffix, the
    version is treated as a toolset version specification. A search is
    performed for the first msvc instance that contains the toolset
    version.

    Only Visual Studio 2017 and later support toolset arguments.  For
    Visual Studio 2015 and earlier, the msvc version is returned and
    the toolset version is None.

    Args:

        version: str
            The version specification may be an msvc version or a toolset
            version.

        prefer_newest: bool
            True:  prefer newer Visual Studio instances.
            False: prefer the "native" Visual Studio instance first. If
                   the native Visual Studio instance is not detected, prefer
                   newer Visual Studio instances.

    Returns:
        tuple: A tuple containing the msvc version and the msvc toolset version.
               The msvc toolset version may be None.

    Raises:
        MSVCToolsetVersionNotFound: when the specified version is not found.
        MSVCArgumentError: when argument validation fails.
    """

This function takes a version, either an msvc version or an msvc toolset version, and returns the associated msvc version and msvc toolset version. By default, it prefers newer versions of Visual Studio.

Examples:

from SCons.Tool.MSCommon import msvc_query_version_toolset

# Assume: VS2022 installed with 14.29.30133 toolset
#         VS2019 installed with 14.29.30133 toolset and 14.28.29910 toolset

# VS2022 with 14.29.30133 toolset
msvc_version, msvc_toolset_version = msvc_query_version_toolset('14.2', prefer_newest=True)
env = Environment(MSVC_VERSION=msvc_version, MSVC_TOOLSET_VERSION=msvc_toolset_version)

# VS2019 with latest 14.29.30133 toolset
msvc_version, msvc_toolset_version = msvc_query_version_toolset('14.2', prefer_newest=False)
env = Environment(MSVC_VERSION=msvc_version, MSVC_TOOLSET_VERSION=msvc_toolset_version)

# VS2019 with 14.28.29910 toolset
msvc_version, msvc_toolset_version = msvc_query_version_toolset('14.28.29910', prefer_newest=True)
env = Environment(MSVC_VERSION=msvc_version, MSVC_TOOLSET_VERSION=msvc_toolset_version)

# VS2019 with 14.28.29910 toolset
msvc_version, msvc_toolset_version = msvc_query_version_toolset('14.28.29910', prefer_newest=False)
env = Environment(MSVC_VERSION=msvc_version, MSVC_TOOLSET_VERSION=msvc_toolset_version)

Any and all feedback, positive and negative, is appreciated. I am happy to field any questions.