How do I set the compiler to use for a package and all its dependencies?

[DanL1234]

Background to issue: https://github.com/mlpack/mlpack/issues/2971

In the above, the advice was to compile armadillo on its own with the cl.exe compiler and the right compile flags - with much reading, I think I can do this within armadillo's CMakeLists.txt by setting:

set(CMAKE_C_COMPILER "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64" )

(or, other ways, as https://gitlab.kitware.com/cmake/community/-/wikis/FAQ#how-do-i-use-a-different-compiler)

However, I dont think that resolves my problem as:

1. There may be a plethora of other dependencies for mlpack, all of which will need the cl.exe compiler to be defined - then this becomes a piece-wise, hugely manual process
2. To my mind, there must be a more elegant solution to ensure a compiler is specified at the vcpkg top level, &
3. As vcpkg installs a package and all its dependencies from a single command (download, unzip, compile as per inlcuded cmakelists file instructions), I'm not sure even how to break into that process and change the compile settings and re-compile via vcpkg while ensuring my modified CMakeList.txt file remains unaltered.

What I think (in my ignorance) there should be a way to say to vcpkg at the "top level", "use this compiler" in the same way as you can, say:

set(VCPKG_CXX_FLAGS "/openmp- ${VCPKG_CXX_FLAGS}") (in the triplet)

Ie. it makes no sense to me that you can set compiler flags but not the compiler - as compiler flags are not universal; they apply to specific compilers. (as outlined in the history bug/quesiton link at the top)

Nb0. https://vcpkg.readthedocs.io/en/latest/users/triplets/ - says, quote, "Triplet is a standard term used in cross compiling as a way to completely capture the target environment (cpu, os, compiler, runtime, etc) in a single convenient name.")

Nb1. I have toyed with the idea of adding a "set(CMAKE_C_COMPILER ..." setting in the vcpkg x64-windows triplet, but it's just a guess and I have no way of knowing what effect this would have, if it would be passed down-the-cmake-dependency-hierarchy, or if its the right thing to do. The vcpkg flags are outlined in the above link - none appear to allow compiler specification)

Nb2. Each time I try something different, removing/deleting vcpkg, re-download/installing so it's fresh (& repeatable) and then compiling the packages takes 30 - 50 mins. Consequently, I've not tried too many things - rather, I have spent time reading & reading & reading on the topic, hoping to find the precise advice to allow me to build my packages and dependencies with a single compiler.

Nb3. I've used google to search on all possible related keywords over about a week and cannot find anything that touches on enforcing cl.exe (or any other compiler). Most destinations have been places like here, or stackoverflow or similar.

I've got to the point where I'm sure I must be missing something really obvious/fundamental. Please help!!

[JackBoosY]

First, vcpkg will find the latest Visual Studio on this machine and use it to build the port in Windows. In other words, any port in Windows will be built using the latest Visual Studio on your machine.

Secondly, if you have multiple versions of Visual Studio installed on your machine at the same time, if you want to choose to use a lower version of Visual Studio, you can add set(VCPKG_PLATFORM_TOOLSET "${TOOLSET_VER}")(TOOLSET_VER="v140" / "v141" / "v142") to the triplet file. At this time, all ports are built using the version of Visual Studio you set.

If you want to use different Visual Studio versions to build different ports in the same vcpkg, you can set the cmake code in the previous item at the top of the portfile.cmake. However, due to the use of different Visual Studio versions, we do not guarantee the validity.

[DanL1234]

Hi Jack, thank you very much for your reply.

I have been digging away in the background and now have part of my answer:

I believe I have found an issue in Vcpkg where different compilers are used within a single build of multiple dependencies (this statement may be worded better by those with more knowledge) .

In my ignorance of Vcpkg, I saw this "issue" and assumed that it was normal for different dependencies to call up different compilers. Thus, my original quesiton of how to specify the compiler.

In my case, installing mlpack:x64-windows would call the following compilers:

cl (from Visual Studio 2019, latest updates as of today, no other Visual Studios - it's a new PC) gcc

I found this out by trying to pass the "/openmp-" flag to cl.exe for the whole mlpack project and dependencies (through VCPKG_CXX_FLAGS & VCPKG_C_FLAGS of x64-windows triplet) -> and found that gcc failed with "error: /openmp-: No such file or directory" (despite all other compile logs showing cl being used)

Ie. All dependencies seemed to build okay with the openmp flag up until the "lapack-reference:x64-windows" dependency, which called gcc. (See attached fail log)

I now understand that "lapack-reference:x64-windows" dependency is calling gcc for access to fortran compilation, but still (erroneously) passes the C compile flags.

I'm now trying to figure out what a "port" is ( I guess, it's short for "portfile") - and how I might use it to prevent C / CXX compile flags causing issues with gcc fortran compilers. Still a bit stumped to be honest.

[DanL1234]

config-x64-windows-out.log

[JackBoosY]

Some ports (libraries) use cross-platform to build themselves on Windows, so it is reasonable to use different compilers for different ports. But if this issue occurs on the master, it must be a bug.

As you can see:

    C:\vcpkg\downloads\tools\msys2\d0837232c10def01\mingw64\bin\gcc.exe   /openmp- -o CMakeFiles/cmTC_c5dca.dir/testCCompiler.c.obj -c testCCompiler.c
    gcc.exe: error: /openmp-: No such file or directory

Normally, we are based on the cmake build system, and other build systems (gcc, clang, etc.) will extract the C_FLAGS / CXX_FLAGS of the cmake build system to set their own environment variables. And it's automatically.

[DanL1234]

Hi Jack, I'll have to leave it to you (or those with better understanding than mine) as I do not understand the topic well enough to say if this is a bug.

What I can say is that I can clone a fresh copy of vcpkg, bootstrap it, then update the x64-windows triplet's C / CXX flags to include "\openmp-", then when I call ".\vcpkg install mlpack:x64-windows" it quits with the error you see when it gets to the "lapack" dependency. From the log I attach/you quote from, this looks like it is taking the C / CXX flags intended for cl.exe and passing them along to gcc for the fortran compilation.

If this is intended, then that is not very friendly to the user. (And it has taken me - a complete novice in this - around 4 weeks to understand, and I dont really have a solution yet)

On an aside, anyone with similar issue / questions to mine - if you follow this procedure, you dont see the same issues: (Although i have yet to get the packages to work together yet!!! (A different issue, I think))

1. freshly clone vcpkg and bootstrap it.
2. .\vcpkg install lapack:x64-windows (this performs the install / build with no C / CXX flags being passed to the gcc / fortran compiler)
3. modify the C / CXX flags as you desire
4. .\vcpkg install mlpack:x64-windows (this looks to build all remaining / outstanding packages with the flags desired and does not recompile the earlier code such as lapack)

The above is taken from the logs created in the build process, and details whether (in my case) \openmp- was used or not.

[JackBoosY]

@DanL1234 Can you provide me a sample project or sample modificiation for mlpack?

[DanL1234]

Hi Jack, sorry for the delay in responding - I'm unable to work on this full time.

Please find attached a VC solution (zipped) and a x64-windows triplet. I have no other modified files.

Let me know what you think. Cheers.

ConsoleApplication1.zip x64-windows.zip

[JackBoosY]

@DanL1234 See my comment in https://github.com/mlpack/mlpack/issues/2971#issuecomment-870345237.