Add MSYSTEM to VCPKG_ENV_PASSTHROUGH for mingw triplets

[RealLitb]

Describe the bug MSYS2 uses MSYSTEM to decide what the correct value is for uname to return. This is very important because it influences makefile systems, in my case python3's autoconf (check https://github.com/msys2-contrib/cpython-mingw/issues/109 ).

If MSYSTEM is not forwarded to the build systems, uname will report a pure msys/cygwin environment. The binaries that will be built then will often work and be native windows binaries, at least for GNU autotools, since VCPKG passes --build=x86_64-pc-mingw32 which makes --host take that triplet aswell (afaik). This signifies a non-cross build, hence uname should return information that's coherent with the host system. But if MSYSTEM is missing, that's not true. Hence the above linked build failure for python3.

Environment

• Windows

[dg0yt]

Basically we don't need uname because it tells us the system we are running on (vcpkg term: host, and msys is okay). The python build will need the system we are building for (vcpkg term: target).

Can you find another example than python? Pristine cpython doesn't support mingw. Vcpkg's capability to cross-build python are incomplete. I looked at https://github.com/msys2-contrib/cpython-mingw recently, but this something to pick up after #31228, #31621, #31727.

[RealLitb]

You are correct that uname tells us the system we are running on. I think for Msys, there are multiple possible system environments that you could run on (I'm a beginner with MSYS/Mingw, but I think I've enough coverage to understand there's an issue here).

• If you wanna have a linux environment, you can set MSYSTEM to MSYS. Then all programs you invoke take, output and work with linux paths and use the cygwin libraries to be as near to POSIX as possible.
• If you wanna have a native windows environment set MSYSTEM to one of {UCRT,CLANG,MINGW}{32,64} and many binaries then become native windows binaries and don't use cygwin.

The compilers will change together with that.. if you choose MSYSTEM to be UCRT64, then the compiler links target windows programs against UCRT64, as far as I understand. Check https://www.msys2.org/docs/environments/ for details.

The uname function works together with that scheme and returns MINGW{32, 64} if it's a windows environment, and MSYS if not. Check https://github.com/msys2/msys2-runtime/blob/msys2-3.4.6/winsup/cygwin/uname.cc#L28 .

Why is it relevant? So while the configure script runs with bash in a MSYS environment, the tools like the compiler and some others are native windows programs (native in the sense they process real windows paths and produce native windows programs). Not only that, but if you tell the configure script that buildsystem is equal to target system (i.e giving --build=... and not giving --host or giving the same value), then the script is free to use uname to learn more about the target system. That's why the value of MSYSTEM is important.

Possibly other packages fail to build aswell, or build and fail at runtime (actually python.exe built, and then failed to run).. Mingw is not tested with the official CI system, as far as I know.

[dg0yt]

I build regularly with mingw on MSYS2. If a port fails to build or run but should work, report it. But again, python is not a suitable example. It is unsupported.

And again, uname is not relevant for the (vcpkg) target. However, some package are very picky about the (vcpkg) host.

MSYSTEM doesn't really select the compiler. However if you set MSYSTEM and then start a login shell, it will add that system's dir to PATH, thus determining which binary will be g++.

This doesn't take away any <prefix>-g++ (cross-)compilers. The vcpkg mingw CMake toolchain chooses such compilers with prefix: https://github.com/microsoft/vcpkg/blob/04d50defc7ead2841b593c358f9b15e03ea4bf0e/scripts/toolchains/mingw.cmake#L24-L25 And the maintainer functions try to pass these settings into other build systems.

[RealLitb]

I understand if you don't want to fix it right now and risk breaking packages that build right now.

As for python, cpython-mingw builds fine for me within vcpkg, after I set MSYSTEM to make configure happy, and after I moved my vcpkg from my Desktop to C:\ in order to make paths shorter than 260 characters (setup.py generates very deep folder hierarchies...). I will raise the path names issue with cpython-mingw - the MSVC version of python buildfiles already has a manifest for python to support long paths, I think they can reuse that.

I also tried to cross build cpython-mingw on linux for windows with vcpkg in the past. While the python-package itself built apparently, the CMake FindPython module caused problems, because CMake tries to discover various things about python by executing it at build time... of course that's a bad thing if you are cross building.

[dg0yt]

Ideally cross-build to MinGW should work from Linux and from MSYS environement - both represent the POSIX world. You ask for the MSYSTEM change to facilitate native builds from MinGW environments. But ideally the build would also work without an existing msys2 installation, just with mingw tools from other providers. vcpkg already acquires msys2 tools. And yes, this python mingw support must be compemented with corresponding changes to vcpkg's wrapper for FindPython.cmake.

Please take the X-Y-problem out of this issue. Make it clear that you want python3 for mingw, or make another case for MSYSTEM.

[RealLitb]

I think you make a good point about it working without an existing msys2 installation. After I got build failures for Qt, I replaced my Msys2 installation (which only had the toolchain in my case..) with another distribution of Mingw-W64 (https://github.com/niXman/mingw-builds-binaries/releases). The Msys2 GCC simply is too recent, and it's not possible to pin the msys2 package system (pacman) to a specific GCC version. You would have to manually download packages from the MSys2 repo and resolve any dependencies.. which is what VCPKG does to get at least some amount of version stability.

Msys2 is used by VCPKG itself, in order to execute the configure shell scripts and resulting Makefiles. Maybe exposing the MSYSTEM thing in the triplet like I proposed isn't ideal. An alternative would be to just set the environment variable in vcpkg_configure_make (which could then be an option on that function that ports could use..).

[RealLitb]

I now see that CONFIGURE_ENVIRONMENT_VARIABLES is an option for vcpkg_configure_make. For python maybe that's already enough to make it work, because as far as I can see, it executes uname at configure time only and then pins the value it got from there. From there it takes its way into "sys.platform" and reaches python scripts.

[dg0yt]

I now see that CONFIGURE_ENVIRONMENT_VARIABLES is an option for vcpkg_configure_make.

You don't need that. The portfile can set(ENV{MSYSTEM} whatever).

[JonLiu1993]

This issue hasn’t been updated in 3 month, if it is still an issue, please reopen this issue.