search

SCons / scons

SCons - a software construction tool
http://scons.org
MIT License
2.03k stars 311 forks source link

SCons not defining VCINSTALLDIR or VSINSTALLDIR in env['MSVS'] per documentation #2992

Open bdbaddog opened 6 years ago

bdbaddog commented 6 years ago

This issue was originally created at: 2015-02-13 15:22:03. This issue was reported by: dirkbaechle. dirkbaechle said at 2015-02-13 15:22:03

As reported by Andrew C. Morrow, the 'MSVS' Environment variable contents appear to be out of sync with their documentation. The SCons 2.3.4 docs state:

MSVS
When the Microsoft Visual Studio tools are initialized, they set up this dictionary with the following keys:

And then lists several keys, including PLATFORMSDKDIR.

However the documented variables do not actually appear to get set. The string PLATFORMSDKDIR, for instance, appears nowhere in the SCons 2.3.4 sources.

dirkbaechle said at 2015-02-13 15:22:34

Added keyword

mwichmann commented 4 years ago

On a fully setup Visual Studio environment, it populates the MSVS dictionary only with two keys: PROJECTSUFFIX and SOLUTIONSUFFIX. Looks like quite a bit of the rest of the MS* construction variable setting differs from the manpage too.

acmorrow commented 4 years ago

When this gets addressed, I think it would also be handy to introduce a VSWHERE SCons variable. This could be set at environment construction to tell SCons which vswhere.exe to use (since it can be installed in different places). Alternatively, if not set, and SCons locates it itself, it should set VSWHERE, so that user build systems can also make use of VSWHERE without needing to implement their own search.

mwichmann commented 4 years ago

Okay, for the first part of this, why would it matter which one you use? Supplying a path that we wouldn't have looked at otherwise seems fine, although it appears if you have any product that can be scanned with vswhere, it will have installed vswhere in the known places.

acmorrow commented 4 years ago

I thought only VS 2017+ installed vswhere? Imagine that I have a VS 2015 only machine, and I want to use vswhere to locate the runtime redistributables. It would be nice to be able to pull down the chocolatey version, and then specify VSWHERE=... on my command line (having made a Variable decl for it in my SConstruct). On other platforms, where SCons will find vswhere, I don't need to do this (SCons would populate it for me from the expected location), but I can still read out the discovered location for vswhere from VSWHERE. So any tooling I need that wants vswhere can be made independent of its actual install location. The same code will work with VS 2017 and VS 2019. Another reason would be if the "system" version of vswhere had a bug and you could get the newer chocolatey version and override which one SCons would use to fix a bug. Overall, it just seems smart to be able to let the user have control over this as an input parameter, the same way MSVC_VERSION works. Or any other number of situations where the default search for vswhere proves to not work as advertised, this gives the user a path forward.

bdbaddog commented 4 years ago

vswhere can detect some "legacy" versions older than 2017. So there is value.

acmorrow commented 4 years ago

Or, look it as future proofing for when Microsoft in their wisdom decides to move vswhere somewhere new in a future version.

mwichmann commented 4 years ago

Fair enough.

As to it working on pre-2017 - as I said in discord, my system that has 2015/2017/2019 all installed doesn't have vswhere "see" anything about the 2015 edition. Updated: with the -legacy flag it does, although it presents only three bits of information about a legacy release - should be enough to work with.

acmorrow commented 4 years ago

Given that this feature appears to have been broken for some time, I think it is worthwhile considering what we would like the fix to look like. I'm not sure that simply re-introducing the prior MSVS dictionary is the right fix.

Let's look at each in turn:

Finally, I think I'd propose hoisting all of these out of the MSVS dictionary. Having these things inside a dictionary isn't great, because it makes it difficult to set them via the command line and Variables objects.

My suggestion is that we end up with top level variables as follows:

I'm open to suggestions on what to do with the .NET stuff.

acmorrow commented 4 years ago

@bdbaddog - Any thoughts on my comments above? And should we tag it for 4.0? It seems worth fixing - there was definitely a regression somewhere along the line, and exposing the installation directories is handy for things like locating the vcredist files.

bdbaddog commented 4 years ago

May or may not make it for 4.0.0 but tagging for now.

bdbaddog commented 2 years ago

Or, look it as future proofing for when Microsoft in their wisdom decides to move vswhere somewhere new in a future version.

Note VSWHERE env variable has been implemented for some time at present.

jcbrill commented 1 year ago

Populating the MSVS dictionary with most of the values listed in the documentation appears to have been removed prior to the release of scons-2.0.0-final.0. The function responsible for populating the dictionary was present in scons-1.2.0 and is shown below.

It appears that starting with SCons 2.0, most of the detection routines were moved into dedicated modules. Some of the dictionary elements are available now via (possibly internal) module functions.

All of the content appears to be "pure detection" via register queries and file system examination rather than the result of running the vcvars batch files. The vcvars batch files produce many of the variables of interest but are not necessarily preserved in the resulting environment. For example, when running the vcvars batch files, VCINSTALLDIR is propagated to the resulting ENV dictionary while VSINSTALLDIR is not.

Note: scons-1.2.0 appears to have supported vs versions 6.0 (and possibly earlier), 7.0 (2002), 7.1 (2003), and 8.0 (2005).

VC/VS:

.NET Framework:

Platform SDK:

Detection functionality largely moved to independent source modules:

scons-1.2.0/engine/SCons/Tool/msvs.py

get_msvs_install_dirs function (lines: 1340-1521)

def get_msvs_install_dirs(version = None, vs8suite = None):
    """
    Get installed locations for various msvc-related products, like the .NET SDK
    and the Platform SDK.
    """

    if not SCons.Util.can_read_reg:
        return {}

    if not version:
        versions = get_visualstudio_versions()
        if versions:
            version = versions[0] #use highest version by default
        else:
            return {}

    version_num, suite = msvs_parse_version(version)

    K = 'Software\\Microsoft\\VisualStudio\\' + str(version_num)
    if (version_num >= 8.0):
        if vs8suite == None:
            # We've been given no guidance about which Visual Studio 8
            # suite to use, so attempt to autodetect.
            suites = get_visualstudio8_suites()
            if suites:
                vs8suite = suites[0]

        if vs8suite == 'EXPRESS':
            K = 'Software\\Microsoft\\VCExpress\\' + str(version_num)

    # vc++ install dir
    rv = {}
    if (version_num < 7.0):
        key = K + r'\Setup\Microsoft Visual C++\ProductDir'
    else:
        key = K + r'\Setup\VC\ProductDir'
    try:
        (rv['VCINSTALLDIR'], t) = SCons.Util.RegGetValue(SCons.Util.HKEY_LOCAL_MACHINE, key)
    except SCons.Util.RegError:
        pass

    # visual studio install dir
    if (version_num < 7.0):
        try:
            (rv['VSINSTALLDIR'], t) = SCons.Util.RegGetValue(SCons.Util.HKEY_LOCAL_MACHINE,
                                                             K + r'\Setup\Microsoft Visual Studio\ProductDir')
        except SCons.Util.RegError:
            pass

        if not rv.has_key('VSINSTALLDIR') or not rv['VSINSTALLDIR']:
            if rv.has_key('VCINSTALLDIR') and rv['VCINSTALLDIR']:
                rv['VSINSTALLDIR'] = os.path.dirname(rv['VCINSTALLDIR'])
            else:
                rv['VSINSTALLDIR'] = os.path.join(SCons.Platform.win32.get_program_files_dir(),'Microsoft Visual Studio')
    else:
        try:
            (rv['VSINSTALLDIR'], t) = SCons.Util.RegGetValue(SCons.Util.HKEY_LOCAL_MACHINE,
                                                             K + r'\Setup\VS\ProductDir')
        except SCons.Util.RegError:
            pass

    # .NET framework install dir
    try:
        (rv['FRAMEWORKDIR'], t) = SCons.Util.RegGetValue(SCons.Util.HKEY_LOCAL_MACHINE,
            r'Software\Microsoft\.NETFramework\InstallRoot')
    except SCons.Util.RegError:
        pass

    if rv.has_key('FRAMEWORKDIR'):
        # try and enumerate the installed versions of the .NET framework.
        contents = os.listdir(rv['FRAMEWORKDIR'])
        l = re.compile('v[0-9]+.*')
        installed_framework_versions = filter(lambda e, l=l: l.match(e), contents)

        def versrt(a,b):
            # since version numbers aren't really floats...
            aa = a[1:]
            bb = b[1:]
            aal = string.split(aa, '.')
            bbl = string.split(bb, '.')
            # sequence comparison in python is lexicographical
            # which is exactly what we want.
            # Note we sort backwards so the highest version is first.
            return cmp(bbl,aal)

        installed_framework_versions.sort(versrt)

        rv['FRAMEWORKVERSIONS'] = installed_framework_versions

        # TODO: allow a specific framework version to be set

        # Choose a default framework version based on the Visual
        # Studio version.
        DefaultFrameworkVersionMap = {
            '7.0'   : 'v1.0',
            '7.1'   : 'v1.1',
            '8.0'   : 'v2.0',
            # TODO: Does .NET 3.0 need to be worked into here somewhere?
        }
        try:
            default_framework_version = DefaultFrameworkVersionMap[version[:3]]
        except (KeyError, TypeError):
            pass
        else:
            # Look for the first installed directory in FRAMEWORKDIR that
            # begins with the framework version string that's appropriate
            # for the Visual Studio version we're using.
            for v in installed_framework_versions:
                if v[:4] == default_framework_version:
                    rv['FRAMEWORKVERSION'] = v
                    break

        # If the framework version couldn't be worked out by the previous
        # code then fall back to using the latest version of the .NET
        # framework
        if not rv.has_key('FRAMEWORKVERSION'):
            rv['FRAMEWORKVERSION'] = installed_framework_versions[0]

    # .NET framework SDK install dir
    if rv.has_key('FRAMEWORKVERSION'):
        # The .NET SDK version used must match the .NET version used,
        # so we deliberately don't fall back to other .NET framework SDK
        # versions that might be present.
        ver = rv['FRAMEWORKVERSION'][:4]
        key = r'Software\Microsoft\.NETFramework\sdkInstallRoot' + ver
        try:
            (rv['FRAMEWORKSDKDIR'], t) = SCons.Util.RegGetValue(SCons.Util.HKEY_LOCAL_MACHINE,
                key)
        except SCons.Util.RegError:
            pass

    # MS Platform SDK dir
    try:
        (rv['PLATFORMSDKDIR'], t) = SCons.Util.RegGetValue(SCons.Util.HKEY_LOCAL_MACHINE,
            r'Software\Microsoft\MicrosoftSDK\Directories\Install Dir')
    except SCons.Util.RegError:
        pass

    if rv.has_key('PLATFORMSDKDIR'):
        # if we have a platform SDK, try and get some info on it.
        vers = {}
        try:
            loc = r'Software\Microsoft\MicrosoftSDK\InstalledSDKs'
            k = SCons.Util.RegOpenKeyEx(SCons.Util.HKEY_LOCAL_MACHINE,loc)
            i = 0
            while 1:
                try:
                    key = SCons.Util.RegEnumKey(k,i)
                    sdk = SCons.Util.RegOpenKeyEx(k,key)
                    j = 0
                    name = ''
                    date = ''
                    version = ''
                    while 1:
                        try:
                            (vk,vv,t) = SCons.Util.RegEnumValue(sdk,j)
                            # TODO(1.5):
                            #if vk.lower() == 'keyword':
                            #    name = vv
                            #if vk.lower() == 'propagation_date':
                            #    date = vv
                            #if vk.lower() == 'version':
                            #    version = vv
                            if string.lower(vk) == 'keyword':
                                name = vv
                            if string.lower(vk) == 'propagation_date':
                                date = vv
                            if string.lower(vk) == 'version':
                                version = vv
                            j = j + 1
                        except SCons.Util.RegError:
                            break
                    if name:
                        vers[name] = (date, version)
                    i = i + 1
                except SCons.Util.RegError:
                    break
            rv['PLATFORMSDK_MODULES'] = vers
        except SCons.Util.RegError:
            pass

    return rv

generate function (lines: 1732-1790, fragment: 1766-1776)

def generate(env):
...
    try:
        version = get_default_visualstudio_version(env)
        # keep a record of some of the MSVS info so the user can use it.
        dirs = get_msvs_install_dirs(version)
        env['MSVS'].update(dirs)
    except (SCons.Util.RegError, SCons.Errors.InternalError):
        # we don't care if we can't do this -- if we can't, it's
        # because we don't have access to the registry, or because the
        # tools aren't installed.  In either case, the user will have to
        # find them on their own.
        pass
...

scons-master/Scons/Tool/MSCommon/netframework.py

Notes:

netframework module (lines 33-76):

...
# Original value recorded by dcournapeau
_FRAMEWORKDIR_HKEY_ROOT = r'Software\Microsoft\.NETFramework\InstallRoot'
# On SGK's system
_FRAMEWORKDIR_HKEY_ROOT = r'Software\Microsoft\Microsoft SDKs\.NETFramework\v2.0\InstallationFolder'

def find_framework_root():
    # XXX: find it from environment (FrameworkDir)
    try:
        froot = read_reg(_FRAMEWORKDIR_HKEY_ROOT)
        debug("Found framework install root in registry: %s", froot)
    except OSError:
        debug("Could not read reg key %s", _FRAMEWORKDIR_HKEY_ROOT)
        return None

    if not os.path.exists(froot):
        debug("%s not found on fs", froot)
        return None

    return froot

def query_versions():
    froot = find_framework_root()
    if froot:
        contents = os.listdir(froot)

        l = re.compile('v[0-9]+.*')
        versions = [e for e in contents if l.match(e)]

        def versrt(a,b):
            # since version numbers aren't really floats...
            aa = a[1:]
            bb = b[1:]
            aal = aa.split('.')
            bbl = bb.split('.')
            # sequence comparison in python is lexicographical
            # which is exactly what we want.
            # Note we sort backwards so the highest version is first.
            return (aal > bbl) - (aal < bbl)

        versions.sort(versrt)
    else:
        versions = []

    return versions
...

scons-master/SCons/Tool/MSCommon/sdk.py

...
def get_installed_sdks():
    global InstalledSDKList
    global InstalledSDKMap
    debug('get_installed_sdks()')
    if InstalledSDKList is None:
        InstalledSDKList = []
        InstalledSDKMap = {}
        for sdk in SupportedSDKList:
            debug('trying to find SDK %s', sdk.version)
            if sdk.get_sdk_dir():
                debug('found SDK %s', sdk.version)
                InstalledSDKList.append(sdk)
                InstalledSDKMap[sdk.version] = sdk
    return InstalledSDKList
...
def get_sdk_by_version(mssdk):
    if mssdk not in SupportedSDKMap:
        raise SCons.Errors.UserError("SDK version {} is not supported".format(repr(mssdk)))
    get_installed_sdks()
    return InstalledSDKMap.get(mssdk)
...
def get_default_sdk():
    """Set up the default Platform/Windows SDK."""
    get_installed_sdks()
    if not InstalledSDKList:
        return None
    return InstalledSDKList[0]
...
mwichmann commented 1 year ago

One possibility is to deprecate this variable with an eye to removing. It seems pretty clear the intent has been to migrate away from looking at things in terms of Visual Studio but in terms of Visual C++, which more accurately describes what we're calling to build C/C++ code. And there's no particular support for C# and other languages whose projects VS knows how to manage, and msbuild knows how to build. My understanding is that the only direct VS/msbuild knowledge we have is in generating project/solution files.

Another possibility is to put back a few of these as makes sense, in case there's any value to being able to query them, or, put them back in a new information structure, more aligned to how we do things now. The docs are updated for 4.5 to say there's not much available. The added bit looks like this, hopefully that's an adequate description:

Visual Studio 2017 and later do not use the registry for primary storage of this information, so typically for these versions only PROJECTSUFFIX and SOLUTIONSUFFIX will be set.