Allow to run external scripts/tools at configure time

[Leandros]

Since FASTBuild already only parses the .bff files once and generates a database from it, it would be nice to allow running a script only on regeneration. This would allow for more advanced configurations, without having to resort back to environment variables (which is currently the best solution).

Though, this would also need some sort of mechanism to get input from these scripts back.

As an example: I would run a python script at configure time, to detect where the Visual Studio is installed and would pass the information to FASTBuild.

[ffulin]

For your Visual Studio detection example, I think you would want it to auto-detect a newly installed Visual Studio after it had been run for the first time. That would require the script to run every time, which is not something I want to allow since it's a sure-fire way to kill build performance.

You can actually already detect the Visual Studio install as follows if you want:

#if exists(VS150COMNTOOLS)
    #import VS150COMNTOOLS
    .Compiler = '$VS150COMNTOOLS$/../../VC/bin/cl.exe'
#else
    #if exists(VS140COMNTOOLS)
        #import VS140COMNTOOLS
        .Compiler = '$VS140COMNTOOLS$/../../VC/bin/cl.exe'
    #else
        #error No compiler found // #error is not actually a thing (yet) but still stops parsing :)
    #endif
#endif

FASTBuild tracks which environment variables you references in the .bff and will re-parse if they change (so installing new versions is handled correctly) with the above.

I guess this might be just one example - do you have other use-cases in mind?

[hacst]

New visual studio versions since 2017 cannot be detected this way. The old environment variables have been removed. You now have to use external tools or COM to query for the toolset. See https://blogs.msdn.microsoft.com/vcblog/2017/03/06/finding-the-visual-c-compiler-tools-in-visual-studio-2017/ for more information on how they want you to do it now.

[ffulin]

@hacst Thanks for that info. It sadly seems that you are right.

I'm open to suggestions about how to extend the ability of FASTBuild to detect compilers (like even allowing it to use the COM API potentially) or solving this is a completely different way. The main concern is to not add something into the critical compilation path.

(Note that I personally recommend installing compilers into source control, which is what I do and why I didn't realize MS "broke" this (also it seems like it worked in at least some form of the VS2017 preview))

[hacst]

The code for vswhere is on github (https://github.com/Microsoft/vswhere). Shouldn't be hard to figure out what they are doing under the hood. The most flexible way probably would still be to let fastbuild run executables to populate variables. I guess that you could then handle them similar to environment variables (as in output change triggers re-parse).

(I agree with you that vendoring the compiler and platform SDK is highly preferable for any environment that wants consistent builds for developers and build machines. Unfortunately there's no way to do so I know of that is officially supported by Microsoft. Of course I can just grab stuff from the install and bang on it until it works but Visual Studio won't know what I am doing and I'm probably violating some MS license by "modifying and redistributing". For the toolset there are now nuget packages from MS but outside of VS those aren't much help either. It feels like they make it harder and harder to make sure you are actually using what you intended to with every release these days.)