mstorsjo / llvm-mingw

An LLVM/Clang/LLD based mingw-w64 toolchain
Other
1.93k stars 186 forks source link

Application with (w)WinMain entrypoint should not allocate console #96

Open vitalyster opened 4 years ago

vitalyster commented 4 years ago

With this simple program:

#include <stdio.h>
#include <windows.h>

int WINAPI wWinMain(HINSTANCE instance, HINSTANCE prev_instance, LPWSTR cmd_line, int show_cmd) {
    wprintf(L"Test\n");
}

MSVC compiler/linker does not allocate console (adds subsystem:windows automatically?) for programs with (w)WinMain entrypoint by default. However both llvm-mingw and mingw64 require to add subsystem flags explicitly.

mstorsjo commented 4 years ago

LLD is able to infer the subsystem, but that's disabled when operating in mingw mode, to match the behaviour of ld.bfd (GNU binutils ld), so that the mingw linkers behave the same. If one were to remove https://github.com/llvm/llvm-project/blob/3e24242/lld/COFF/Driver.cpp#L577-L578, it might infer the subsystem automatically (untested).

Even if the subsystem is inferred automatically, you'd still need to pass -municode when linking, to get the right entry point for those cases - this is different from how it works with MSVC (where the executable entry points are handled by more magic in the linker, while GCC and the mingw toolchains require the compiler driver to pick between the entry points between main and wmain, and WinMain and wWinMain.

eabase commented 2 years ago

Yeah, I was getting:

ld.lld: error: undefined symbol: WinMain
>>> referenced by ../crt\crt0_c.c:18
>>>               libmingw32.a(lib64_libmingw32_a-crt0_c.o):(main)
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)

adding -municode to the compile line fixed it!