Support custom MSVC build tools (msvc-wine, PortableBuildTools)

star-hengxing commented 1 week ago

Is your feature request related to a problem? Please describe.

related toolchain:

related pr:

https://github.com/xmake-io/xmake/pull/5823

Describe the solution you'd like

Cross-compile Window programs on Linux based on msvc-wine.
Use custom msvc toolchain.

Describe alternatives you've considered

No response

Additional context

No response

star-hengxing commented 1 week ago

Host Windows + PortableBuildTools

msvc

$ xmake f -c --sdk=C:/BuildTools
$ xmake -v
[ 50%]: compiling.release src\main.cpp
C:\BuildTools\VC\Tools\MSVC\14.41.34120\bin\Hostx64\x64\cl.exe -c -nologo /EHsc -Fobuild\.objs\test\windows\x64\release\src\main.cpp.obj src\main.cpp
[ 75%]: linking.release test.exe
C:\BuildTools\VC\Tools\MSVC\14.41.34120\bin\Hostx64\x64\link.exe -nologo -dynamicbase -nxcompat -machine:x64 -out:build\windows\x64\release\test.exe build\.objs\test\windows\x64\release\src\main.cpp.obj
[100%]: build ok, spent 0.312s

clang-cl

$ xmake f -c --sdk=C:/BuildTools --toolchain=clang-cl
$ xmake -v
[ 50%]: cache compiling.release src\main.cpp
C:\Users\star\scoop\apps\llvm\current\bin\clang-cl.exe -c -m64 /EHsc -Fobuild\.objs\test\windows\x64\release\src\main.cpp.obj src\main.cpp
[ 75%]: linking.release test.exe
C:\BuildTools\VC\Tools\MSVC\14.41.34120\bin\Hostx64\x64\link.exe -nologo -dynamicbase -nxcompat -machine:x64 -out:build\windows\x64\release\test.exe build\.objs\test\windows\x64\release\src\main.cpp.obj
[100%]: build ok, spent 0.281s

clang

$ xmake f -c --sdk=C:/BuildTools --toolchain=clang
$ xmake -v
[ 50%]: cache compiling.release src\main.cpp
C:\Users\star\scoop\apps\llvm\current\bin\clang -c -Qunused-arguments -m64 --target=x86_64-windows-msvc -fexceptions -fcxx-exceptions -o build\.objs\test\windows\x64\release\src\main.cpp.obj src\main.cpp
[ 75%]: linking.release test.exe
C:\Users\star\scoop\apps\llvm\current\bin\clang++ -o build\windows\x64\release\test.exe build\.objs\test\windows\x64\release\src\main.cpp.obj -m64 --target=x86_64-windows-msvc
[100%]: build ok, spent 0.235s

star-hengxing commented 1 week ago

Host Linux + msvc-wine

msvc

$ xmake f -c -p windows -a x64 --sdk=~/msvc
checking for Microsoft C/C++ Compiler (x64) version ... 19.41.34120
$ xmake -v
checking for cl.exe ... /home/star/msvc/bin/x64/cl.exe
checking for the c++ compiler (cxx) ... cl.exe
[ 50%]: cache compiling.release src/main.cpp
/home/star/msvc/bin/x64/cl.exe -c -nologo /EHsc -Fobuild/.objs/test/windows/x64/release/src/main.cpp.obj src/main.cpp
checking for /home/star/msvc/bin/x64/cl.exe ... ok
checking for flags (cl_sourceDependencies) ... no
checking for link.exe ... /home/star/msvc/bin/x64/link.exe
checking for the linker (ld) ... link.exe
[ 75%]: linking.release test.exe
/home/star/msvc/bin/x64/link.exe -nologo -dynamicbase -nxcompat -machine:x64 -out:build/windows/x64/release/test.exe build/.objs/test/windows/x64/release/src/main.cpp.obj
checking for cl.exe ... /home/star/msvc/bin/x64/cl.exe
checking for the c compiler (cc) ... cl.exe
[100%]: build ok, spent 2.002s

clang

clang + msvc-wine requires lld-link for link, and some distributions of clang packages don't set an alias for lld-link, so we need to specify the toolchain version explicitly.

$ xmake f -c -p windows -a x64 --sdk=~/msvc --toolchain=clang
$ xmake -v
[ 50%]: cache compiling.release src/main.cpp
/usr/bin/clang -c -Qunused-arguments -m64 --target=x86_64-windows-msvc -fexceptions -fcxx-exceptions -o build/.objs/test/windows/x64/release/src/main.cpp.obj src/main.cpp
warning: unknown warning option '-Wno-gnu-line-marker'; did you mean '-Wno-gnu-case-range'? [-Wunknown-warning-option]
1 warning generated.
[ 75%]: linking.release test.exe
/usr/bin/clang++ -o build/windows/x64/release/test.exe build/.objs/test/windows/x64/release/src/main.cpp.obj -m64 -fuse-ld=lld-link --target=x86_64-windows-msvc
clang: error: unable to execute command: Executable "lld-link" doesn't exist!
clang: error: linker command failed with exit code 1 (use -v to see invocation)
error: execv(/usr/bin/clang++ -o build/windows/x64/release/test.exe build/.objs/test/windows/x64/release/src/main.cpp.obj -m64 -fuse-ld=lld-link --target=x86_64-windows-msvc) failed(1)

$ xmake f -c -p windows -a x64 --sdk=~/msvc --toolchain=clang-13
checking for LLVM Clang C/C++ Compiler (x64) version ... 14.0.0-1ubuntu1.1
$ xmake -v
checking for clang-13 ... /usr/bin/clang-13
checking for the c++ compiler (cxx) ... clang-13
[ 50%]: cache compiling.release src/main.cpp
/usr/bin/clang-13 -c -Qunused-arguments -m64 --target=x86_64-windows-msvc -fexceptions -fcxx-exceptions -o build/.objs/test/windows/x64/release/src/main.cpp.obj src/main.cpp
checking for /usr/bin/clang-13 ... ok
checking for flags (-MMD -MF) ... ok
checking for flags (-fdiagnostics-color=always) ... ok
warning: unknown warning option '-Wno-gnu-line-marker'; did you mean '-Wno-gnu-case-range'? [-Wunknown-warning-option]
1 warning generated.
checking for clang++-13 ... /usr/bin/clang++-13
checking for the linker (ld) ... clang++-13
[ 75%]: linking.release test.exe
/usr/bin/clang++-13 -o build/windows/x64/release/test.exe build/.objs/test/windows/x64/release/src/main.cpp.obj -m64 -fuse-ld=lld-link-13 --target=x86_64-windows-msvc
checking for clang-13 ... /usr/bin/clang-13
checking for the c compiler (cc) ... clang-13
[100%]: build ok, spent 0.502s

star-hengxing commented 1 week ago

msvc-wine with xrepo:

add_requires("fmt[shared]")

$ xmake f -c -p windows -a x64 --sdk=~/msvc -v
note: install or modify (m) these packages (pass -y to skip confirm)?
in xmake-repo:
  -> fmt 11.0.2 [shared:y, runtimes:"MT"]
please input: y (y/n/m)
y
checking for ping ... /usr/bin/ping
pinging the host(github.com) ... 86 ms
checking for unzip ... /usr/bin/unzip
/usr/bin/unzip -o fmt-11.0.2.zip -d source.tmp
checking for cl.exe ... /home/star/msvc/bin/x64/cl.exe
checking for the c++ compiler (cxx) ... cl.exe
checking for cl.exe ... /home/star/msvc/bin/x64/cl.exe
checking for the c compiler (cc) ... cl.exe
checking for ml64.exe ... /home/star/msvc/bin/x64/ml64.exe
checking for the assember (as) ... ml64.exe
checking for link.exe ... /home/star/msvc/bin/x64/link.exe
checking for the static library archiver (ar) ... link.exe
checking for link.exe ... /home/star/msvc/bin/x64/link.exe
checking for the linker (ld) ... link.exe
checking for link.exe ... /home/star/msvc/bin/x64/link.exe
checking for the shared library linker (sh) ... link.exe
checking for cmake ... /home/star/.xmake/packages/c/cmake/3.30.2/924bf70cd2964eea80fdac45c84f12e4/bin/cmake
/home/star/.xmake/packages/c/cmake/3.30.2/924bf70cd2964eea80fdac45c84f12e4/bin/cmake -DFMT_TEST=OFF -DFMT_DOC=OFF -DFMT_FUZZ=OFF -DCMAKE_CXX_VISIBILITY_PRESET=default -DBUILD_SHARED_LIBS=ON -DCMAKE_BUILD_TYPE=Release -DFMT_UNICODE=ON -DCMAKE_INSTALL_PREFIX=/home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3 -DCMAKE_INSTALL_LIBDIR:PATH=lib -G Ninja -DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreaded -DCMAKE_COMPILE_PDB_OUTPUT_DIRECTORY=pdb -DCMAKE_SYSTEM_NAME=Windows -DCMAKE_AR=/home/star/msvc/bin/x64/link.exe -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH -DCMAKE_FIND_USE_INSTALL_PREFIX=0 -DCMAKE_FIND_ROOT_PATH=~/msvc -DCMAKE_FIND_USE_CMAKE_SYSTEM_PATH=0 -DCMAKE_C_COMPILER=/home/star/msvc/bin/x64/cl.exe -DCMAKE_STATIC_LINKER_FLAGS= -DHAVE_FLAG_SEARCH_PATHS_FIRST=0 -DCMAKE_CXX_COMPILER=/home/star/msvc/bin/x64/cl.exe -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=BOTH -DCMAKE_ASM_COMPILER=/home/star/msvc/bin/x64/ml64.exe -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH "-DCMAKE_STATIC_LINKER_FLAGS_RELEASE=  -nodefaultlib:msvcrt.lib" "-DCMAKE_MODULE_LINKER_FLAGS_RELEASE=  -nodefaultlib:msvcrt.lib" "-DCMAKE_SHARED_LINKER_FLAGS_RELEASE=/machine:x64 /INCREMENTAL:NO -nodefaultlib:msvcrt.lib" "-DCMAKE_EXE_LINKER_FLAGS_RELEASE=/machine:x64 /INCREMENTAL:NO -nodefaultlib:msvcrt.lib" /home/star/.xmake/cache/packages/2411/f/fmt/11.0.2/source
-- CMake version: 3.30.2
-- The CXX compiler identification is MSVC 19.41.34120.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/star/msvc/bin/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- {fmt} version: 11.0.2
-- Build type: Release
-- Configuring done (3.1s)
-- Generating done (0.0s)
CMake Warning:
  Manually-specified variables were not used by the project:

    CMAKE_ASM_COMPILER
    CMAKE_C_COMPILER
    CMAKE_FIND_ROOT_PATH
    CMAKE_FIND_ROOT_PATH_MODE_LIBRARY
    CMAKE_FIND_ROOT_PATH_MODE_PACKAGE
    HAVE_FLAG_SEARCH_PATHS_FIRST

-- Build files have been written to: /home/star/.xmake/cache/packages/2411/f/fmt/11.0.2/source/build_20415532
checking for ninja ... /usr/bin/ninja
/usr/bin/ninja install -C /home/star/.xmake/cache/packages/2411/f/fmt/11.0.2/source/build_20415532 -j 3
ninja: Entering directory `/home/star/.xmake/cache/packages/2411/f/fmt/11.0.2/source/build_20415532'
[3/4] Install the project...
-- Install configuration: "Release"
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/lib/fmt.lib
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/bin/fmt.dll
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/args.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/base.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/chrono.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/color.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/compile.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/core.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/format.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/format-inl.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/os.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/ostream.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/printf.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/ranges.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/std.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include/fmt/xchar.h
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/lib/cmake/fmt/fmt-config.cmake
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/lib/cmake/fmt/fmt-config-version.cmake
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/lib/cmake/fmt/fmt-targets.cmake
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/lib/cmake/fmt/fmt-targets-release.cmake
-- Installing: /home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/lib/pkgconfig/fmt.pc
checking for xmake::fmt ... fmt 11.0.2
{
  links = {
    "fmt"
  },
  license = "MIT",
  linkdirs = {
    "/home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/lib"
  },
  version = "11.0.2",
  sysincludedirs = {
    "/home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/include"
  },
  libfiles = {
    "/home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/lib/fmt.lib",
    "/home/star/.xmake/packages/f/fmt/11.0.2/20415532ae914056bbaccd1abe29c2d3/bin/fmt.dll"
  },
  shared = true,
  static = true,
  cxxflags = "/utf-8",
  defines = "FMT_LIB_EXPORT"
}

checking for /home/star/msvc/bin/x64/cl.exe ... ok
checking for flags (/utf-8) ... ok
checking for flags (-MT) ... ok
checking for flags (cl_external_includedir) ... no
checking for flags (cl_external_includedir_experimental) ... no
> checking for c++ includes(fmt/format.h)
> checking for c++ links(fmt)
> checking for c++ snippet(test)
  => install fmt 11.0.2 .. ok

waruqi commented 1 week ago

Thanks! 👍

Issues-translate-bot commented 1 week ago

Bot detected the issue body's language is not English, translate it automatically.

Thanks! 👍

waruqi commented 1 week ago

xmake run -> call wine on non-windows

https://github.com/xmake-io/xmake/pull/5849

Zcaic commented 1 week ago

强，我测试了一下，windows下的PortableBuildTools，能够成功编译了，但是也有一些小问题，在vscode生成的compile_commands.json似乎有些问题，这里clangd会显示头文件In included file: 'crtdbg.h' file not foundclang(pp_file_not_found)，我查看了一下compile_command.json文件，里面的头文件似乎只包含了一个，但是，我查看了PortableBuildTools文件夹下的设置环境变量的脚本devcmd.ps1，它的头文件目录有好几个。

star-hengxing commented 1 week ago

@Zcaic 再试试，应该没问题了

Issues-translate-bot commented 1 week ago

Bot detected the issue body's language is not English, translate it automatically.

@Zcaic Try again, it should be fine

Zcaic commented 1 week ago

👍，刚测试了一下，输出的compile_commands.json 没问题了，clangd能够正确识别，不会有头文件报错的的问题了。👍

Issues-translate-bot commented 1 week ago

Bot detected the issue body's language is not English, translate it automatically.

👍, I just tested it, and the output compile_commands.json is no problem. clangd can correctly identify it, and there will be no problem of header file error. 👍

ririyeye commented 1 week ago

还是有点问题再帮忙看下 https://github.com/xmake-io/xmake/issues/4501#issuecomment-2492995236

Issues-translate-bot commented 1 week ago

Bot detected the issue body's language is not English, translate it automatically.

There is still something wrong. Please help me. https://github.com/xmake-io/xmake/issues/4501#issuecomment-2492995236

waruqi commented 4 days ago

现在支持了远程包拉取，即使当前 win 环境没有 msvc，也能自动安装和构建。

https://github.com/xmake-io/xmake/pull/5880 https://github.com/xmake-io/xmake-repo/pull/5820

add_rules("mode.debug", "mode.release")
add_requires("msvc")

target("test")
    set_kind("binary")
    add_files("src/*.c")
    set_toolchains("@msvc")
    --set_toolchains("clang-cl@msvc")

PS D:\projects\personal\xmake\tests\projects\package\toolchain_msvc> xmake
checking for platform ... windows
checking for architecture ... x64
checking for Microsoft Visual Studio (x64) version ... 2022
checking for Microsoft C/C++ Compiler (x64) version ... 19.41.33923
note: install or modify (m) these toolchain packages first (pass -y to skip confirm)?
in local-repo:
  -> msvc 14.42.17+12 [host]
please input: y (y/n/m)

  => install msvc 14.42.17+12 .. ok
checking for Microsoft C/C++ Compiler (x64) version ... 19.42.34433
[ 50%]: compiling.release src\main.c
[ 75%]: linking.release test.exe
[100%]: build ok, spent 0.703s

heheda123123 commented 4 days ago

太强了

Issues-translate-bot commented 4 days ago

Bot detected the issue body's language is not English, translate it automatically.

Too strong

xmake-io / xmake