search

SCons / scons

SCons - a software construction tool
http://scons.org
MIT License
2.06k stars 314 forks source link

MSVC Version 14.1 not found with 14.2 installed #3664

Open bryanwweber opened 4 years ago

bryanwweber commented 4 years ago

Describe the bug I have VS2019 installed, with the VC++ 14.2, 14.1, and 14.0 build tools selected in the installer. When MSVC_VERSION="14.1" is specified, SCons cannot find the 14.1 build tools. When MSVC_VERSION="14.2" or 14.0 are specified, SCons works as expected.

Required information

env = Environment(MSVC_VERSION="14.1")
env.Program("hello", "hello.cpp")

Further information:

[
  {
    "instanceId": "7876f29c",
    "installDate": "2020-02-04T16:37:05Z",
    "installationName": "VisualStudio/16.6.0+30114.105",
    "installationPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise",
    "installationVersion": "16.6.30114.105",
    "productId": "Microsoft.VisualStudio.Product.Enterprise",
    "productPath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\Common7\\IDE\\devenv.exe",
    "state": 4294967295,
    "isComplete": true,
    "isLaunchable": true,
    "isPrerelease": false,
    "isRebootRequired": false,
    "displayName": "Visual Studio Enterprise 2019",
    "description": "Scalable, end-to-end solution for teams of any size",
    "channelId": "VisualStudio.16.Release",
    "channelUri": "https://aka.ms/vs/16/release/channel",
    "enginePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\resources\\app\\ServiceHub\\Services\\Microsoft.VisualStudio.Setup.Service",
    "releaseNotes": "https://go.microsoft.com/fwlink/?LinkId=660893#16.6.0",
    "thirdPartyNotices": "https://go.microsoft.com/fwlink/?LinkId=660909",
    "updateDate": "2020-05-20T17:41:11.1443849Z",
    "catalog": {
      "buildBranch": "d16.6",
      "buildVersion": "16.6.30114.105",
      "id": "VisualStudio/16.6.0+30114.105",
      "localBuild": "build-lab",
      "manifestName": "VisualStudio",
      "manifestType": "installer",
      "productDisplayVersion": "16.6.0",
      "productLine": "Dev16",
      "productLineVersion": "2019",
      "productMilestone": "RTW",
      "productMilestoneIsPreRelease": "False",
      "productName": "Visual Studio",
      "productPatchVersion": "0",
      "productPreReleaseMilestoneSuffix": "7.0",
      "productSemanticVersion": "16.6.0+30114.105",
      "requiredEngineVersion": "2.6.2108.39667"
    },
    "properties": {
      "campaignId": "",
      "channelManifestId": "VisualStudio.16.Release/16.6.0+30114.105",
      "nickname": "",
      "setupEngineFilePath": "C:\\Program Files (x86)\\Microsoft Visual Studio\\Installer\\vs_installershell.exe"
    }
  }
]
bryanwweber commented 2 years ago

Thanks @jcbrill I'll try it out on our GitHub Actions runs and see how it goes!

bdbaddog commented 2 years ago

Thanks @jcbrill I'll try it out on our GitHub Actions runs and see how it goes!

Any updates on this and/or using 4.4.0?

bryanwweber commented 1 year ago

I've not been able to get this to work on GH hosted runners. Unfortunately, I don't have a local Windows machine for testing.

For reference, I'm working in this PR: https://github.com/Cantera/cantera/pull/1392, I'm using SCons 4.4.0, and the software on the windows-2019 image* includes:

The specific changes I'm making are in the SConstruct file. I tried adding

Environment(MSVC_VERSION="14.3")

but SCons couldn't find that version of MSVC (makes sense, 2019 is installed). I also tried

Environment(MSVC_VERSION="14.2", MSVC_TOOLSET_VERSION="14.3")

but SCons complained that the toolset version cannot be greater than the MSVC_VERSION. So I think I'm misconfiguring something, because it seems like this should just work™️. I've pinged some other folks on the project who have Windows boxes to see if they can test locally.

* I'm using windows-2019 because windows-2022 only includes the 14.32 toolset

jcbrill commented 1 year ago

The following was done rather quickly, I apologize in advance for any erroneous information.

Unless I'm missing something, the v143 (14.3x) toolsets are not available in a 2019 installation.

The table on the windows-2019 image page seems to indicate that the v142, v141, and v140 toolsets are installed in VS2019. It looks like the 14.3X runtimes are installed as well.

This is consistent with the SCons message in the first log:

scons: warning: MSVC version '14.3' was not found.
  Visual Studio C/C++ compilers may not be set correctly.
  Installed versions are: ['14.2', '14.0']

Based on the installation table for the windows-2019 image and the error message above, the message appears to be consistent/correct.

VS2019 ('14.2') is detected and VS2015 ('14.0') is detected due to the v140 build tools being installed from VS2019 which is found independently via the registry.

If the above is correct, then neither MSVC_VERSION or MSVC_TOOLSET_VERSION can have the value '14.3' (i.e., VS2022 is not installed and the v143 build tools are not installed).

The following should work on the windows-2019 image for VS2019;

Based on the table linked above, the windows-2022 image for VS2022 appears to contain the v143, v142, and v141 build tools.

The following should work on the windows-2022 image for VS2022:

If possible, try MSVC_VERSION='14.3' and MSVC_TOOLSET_VERSION='14.2' with the windows-2022 image for the v142 build tools in VS2022.

I hope this helps.

bryanwweber commented 1 year ago

Thanks @jcbrill I totally misread that table in the GitHub docs. I set it up the way you suggested on both their windows-2019 and windows-2022 images, and things seem to be working exactly the way I'd expect! https://github.com/Cantera/cantera/actions/runs/3094836889

This is great news for SCons and I think this issue is resolved as far as the initial report is concerned. I'll let you or @bdbaddog decide whether there's still more action items from this issue and whether those are being tracked here or elsewhere, to actually close this issue. Thanks again for the help!

mwichmann commented 1 year ago

May be unrelated, but my setup is still failing the MSVC_USE_SETTINGS test (Could not find MSVC compiler 'cl') - all the other failures are gone with the last PR that went in. @jcbrill ?

jcbrill commented 1 year ago

May be unrelated, but my setup is still failing the MSVC_USE_SETTINGS test (Could not find MSVC compiler 'cl') - all the other failures are gone with the last PR that went in.

@mwichmann I will start investigating (time limited today though).

jcbrill commented 1 year ago

@mwichmann The MSVC_USE_SETTINGS test passes locally with the latest SCons master both with and without the msvc cache enabled using python 3.6.

Any chance there was a msvc/toolset update and the cache is out-of-date?

mwichmann commented 1 year ago

of course... but weren't we somewhat immune to that now?

mwichmann commented 1 year ago

still fails if I move aside the old cache

jcbrill commented 1 year ago

of course... but weren't we somewhat immune to that now?

Yes, but it is not foolproof.

The "VCToolsInstallDir" folder is checked for existence (if the list is not empty). It may be possible that the folder exists but is empty. Unlikely, but possible.

still fails if I move aside the old cache

groan.

Could you try running just the MSVC_USE_SETTINGS test from the scons root at your convenience?

Set debug log, cache file, and run problematic test:

(_env-311-64-scons) S:\SCons\scons-master>set SCONS_MSCOMMON_DEBUG=%CD%\mydebug.txt
(_env-311-64-scons) S:\SCons\scons-master>set SCONS_CACHE_MSVC_CONFIG=%CD%\mycache.json
(_env-311-64-scons) S:\SCons\scons-master>python runtest.py .\test\msvc\MSVC_USE_SETTINGS.py

and post the debug log and cache file:

mydebug.txt
mycache.json
jcbrill commented 1 year ago

@mwichmann Is it possible there is a compiler executable ('cl.exe') found on the windows system path?

One of the tests in MSVC_USE_SETTINGS passes an empty dictionary and expects a warning to be issued.

As we discussed in another thread, the find_program_path(env, 'cl') invocation (I believe erroneously) inspects the windows system path.

If 'cl.exe' is not in the generated environment but IS on the system path, the warning will not be issued and the test would fail as the expected warning text is not produced.

mwichmann commented 1 year ago

yeah we've been talking about the mismatch between windows path and scons path elsewhere too - I'm having trouble with the D compilers on Windows, as well as Intel Fortran, because of that.

jcbrill commented 1 year ago

I just tried a test by running the windows batch file from the command prompt and then running the single test:

(_env-368-64-scons) S:\SCons\scons-master>C:\\Software\\MSVS-2022-143-Com\\VC\\Auxiliary\\Build\\vcvars64.bat
**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.6.4
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
[vcvarsall.bat] Environment initialized for: 'x64'

(_env-368-64-scons) S:\SCons\scons-master>python runtest.py -f ..\scons-tests\msvc-tests-jcb.txt
1/1 (100.00%) S:\SCons\_env-368-64-scons\Scripts\python.exe test\MSVC\MSVC_USE_SETTINGS.py
Missing expected input from output:
Could not find MSVC compiler 'cl'
output =========================================================================

FAILED test of S:\SCons\scons-master\scripts\scons.py
        at line 350 of S:\SCons\scons-master\testing\framework\TestCommon.py (must_contain_all)
        from line 60 of test\MSVC\MSVC_USE_SETTINGS.py

(_env-368-64-scons) S:\SCons\scons-master>
mwichmann commented 1 year ago

Here are the two files... the cache I had to call .txt instead because of github's silly limitations. And yes, that looks like the same error.

msdebug.txt scons_msvc_cache.txt

jcbrill commented 1 year ago

I think a cl.exe was found on the system path.

The log contains the following:

00311ms:MSCommon/vc.py:msvc_setup_env#1518: use_settings {}
00357ms:MSCommon/vc.py:msvc_exists#1546: version=None, return=True

If no cl.exe is found the log should look like:

00386ms:MSCommon/vc.py:msvc_setup_env#1518: use_settings {}
00394ms:MSCommon/vc.py:msvc_setup_env#1532: did not find cl.exe

Which triggers the warning expected by the test.

In SCons\Tool\MSCommon\vc.py (starting at line 1530), try changing the search function from:

    # final check to issue a warning if the compiler is not present
    if not find_program_path(env, 'cl'):
        debug("did not find %s", _CL_EXE_NAME)
        ...

to:

    # final check to issue a warning if the compiler is not present
    if not env.WhereIs('cl'):
        debug("did not find %s", _CL_EXE_NAME)
        ...

The test should pass.

jcbrill commented 1 year ago

Prior discussion concerning cl.exe found on system path but not in environment starting here: https://github.com/SCons/scons/issues/4312#issuecomment-1605440471

mwichmann commented 1 year ago

I think a cl.exe was found on the system path.

yes - I just looked over at the screen and it seems I opened a "Developer Command Prompt for VS 2022" there, rather than my normal bare cmd prompt. The test passes if I do the latter.

mwichmann commented 1 year ago

... needless to say, that shouldn't affect the test pass/fail status... Seems like msvc tool does plenty of work now to find legitimate copies in the way we want, and we should probably stop doing what we're doing with that find_program_path call.

jcbrill commented 1 year ago

... needless to say, that shouldn't affect the test pass/fail status... Seems like msvc tool does plenty of work now to find legitimate copies in the way we want, and we should probably stop doing what we're doing with that find_program_path call.

I agree.

I'm not convinced that env.WhereIs('cl.exe') is the best approach as the environment path is checked after prepending the 'restricted' msvc environment path.

It is possible the user modified the passed in environment (e.g., env['ENV']['PATH']=os.environ['PATH']) prior to the msvc call. The desired cl.exe could still be missing but, analogous the current problem, a spurious 'cl.exe' could be found in the env['ENV']['PATH'].

The check should probably use only the generated dictionary PATH elements rather than the resultant env['ENV']['PATH'].

My $0.02 cents.

Edit: add literal text for readability.

ghost commented 4 months ago

how do we solve this now?

mwichmann commented 4 months ago

There was a lot of discussion here, which makes for some messy reading, but the original poster did comment that things seemed to be working eventually - see https://github.com/SCons/scons/issues/3664#issuecomment-1253141077

If you're having problems finding the compiler, it would be useful to get a few more details...

ghost commented 4 months ago

There was a lot of discussion here, which makes for some messy reading, but the original poster did comment that things seemed to be working eventually - see #3664 (comment)

If you're having problems finding the compiler, it would be useful to get a few more details...

Thanks!

jcbrill commented 4 months ago

Some additional examples of using an older toolset are also here: https://github.com/SCons/scons/discussions/4412#discussioncomment-7034182