search

deplinenoise / tundra

Tundra is a code build system that tries to be accurate and fast for incremental builds
MIT License
438 stars 75 forks source link

Rework of MSVC tools #341

Closed leidegre closed 11 months ago

leidegre commented 11 months ago

I got annoyed by the slowness associated with the vcvarsall.bat invocation (which is totally my fault) so I investigated what the VsDevCmd.bat script does (the script responsible for setting up the Developer Command Prompt for Visual Studio).

VsDevCmd.bat just calls the bat files in these locations

But for the VC tools the only two scripts that we care about are core\winsdk.bat and ext\vcvars.bat. Unsurprisingly these take about 160 milliseconds to complete. So I ported them into a new msvc-latest.lua tools script and DAG regen is now down to ~40 milliseconds and this is without a vcvars_cache.

I have made some concessions. I have...

I made some changes to the MSVC config options. The old names still work but I though these aligned more nicely with everything else. So I would have to update the docs (add a new section after vs2008)!

Build {
  Configs = {
    {
      Tools = {
        {
          "msvc-latest",
          Path = "C:\\Program Files (x86)\\Microsoft Visual Studio", -- optional
          Version = "2019", -- optional
          Product = "BuildTools", -- optional
          HostArch = "x64", -- optional
          TargetArch = "x64", -- optional
          WindowsSdkVersion = "10.0.22621.0", -- optional
          VcToolsVersion = "14.29.30133" -- optional
        }
      }
    }
  }
}

I call this tool msvc-latest because when Microsoft eventually releases VS20xx this should keep on working. I'm hoping that this will be forward compatible and that it can be used to replace msvc-vscommon-next.lua. It will however emit a warning for untested Visual Studio versions.

Maybe something like this:

msvc-vs2017 -> msvc-latest with Version = "2017" predefined
msvc-vs2019 -> msvc-latest with Version = "2019" predefined
msvc-vs2022 -> msvc-latest with Version = "2022" predefined

and theoretically:
msvc-latest with Version = "2024" (should just work)

Thus deleting msvc-vscommon-next.lua and msvc-vswild.lua (I don't think any sensible person uses this)

I haven't tested this thoroughly yet but it appears to work just fine, as-is. I've tried to include very helpful error messages and made it output the exact version you end up using if you don't specify one, so that if you want to, you can more easily lock in a specific SDK and compiler version. It might look like this:

> tundra2 -f -G
3 valid build tuples
Generating DAG for win64-msvc-release-default
  WindowsSdkVersion : 10.0.22621.0
  VcToolsVersion    : 14.29.30133
Generating DAG for win64-msvc-debug-default
  WindowsSdkVersion : 10.0.22621.0
  VcToolsVersion    : 14.29.30133
Generating DAG for win64-msvc-production-default
  WindowsSdkVersion : 10.0.22621.0
  VcToolsVersion    : 14.29.30133
save_dag_data: 3 bindings, 29 accessed files
compiling mmapable DAG data..
*** Build success (0.04 seconds)

Note: these go away if you specify a version. They only show up if you don't ask for a specific version. If you ask for a specific version and we cannot find it, you will get an error.

I'm planning on adding support for ATLMFC and app platforms however, these will be opt-in.

I'm also planning a msvc-clang.lua tool. Since Microsoft now ships clang with Visual Studio. Adding this is very little effort (given everything else here) and I think it's really nice to easily be able to use clang on Windows with Tundra.

I would very much appreciate some feedback on this. Like naming, direction, overall scope. Maybe some brave soul who can test the new tooling script before merging and so that we don't risk breaking stuff?

leidegre commented 11 months ago

I added the following information to the docs.

My plan is to do some more testing tomorrow. If all goes well I should be able to add the app platforms, MFC/ATL and clang over the weekend and maybe then we can merge this?

deplinenoise commented 11 months ago

Nice work, let me take this for a test drive.

deplinenoise commented 11 months ago

I like this, it works well for me. Ready for me to merge this or do you want to touch anything up?

leidegre commented 11 months ago

Yes, there are a few things I need to test. It looks like the VS 2022 BuildTools are still being loaded from \Program Files (x86) but the other VS 2022 products are loaded from \Program Files. ¯\_(ツ)_/¯

I have VS 2017 - VS 2022 and various products installed and some samples that I just want to run though to verify everything. Hoping to get that done over the weekend and then I'll just take this PR out of draft mode and you can squash merge everything with a nice summary?

Also to be perfectly clear, I will also delete msvc-common-next.lua and msvs-vcwild.lua and migrate msvc-vs2017..msvc-vs2022 to msvc-latest.

leidegre commented 11 months ago

Okay, here's the gist

DAG regen takes about 10 milliseconds per configuration. In a build with 12 configurations tundra2 -f -G takes 0.12 seconds.

Here's what this PR doesn't do

I will be submitting another PR for msvc-clang tools based on this.

Unless someone has anything else to add, feel free to merge this.

leidegre commented 11 months ago

Okay, I'm done I just found a silly bug and forgot to cleanup a file. No more force pushing.