Is it possible to compile cxxjit in Windows ?

[tapika]

Started Visual Studio's Developer Command Prompt for VS 2019:

First I've tried to compile cxxjit using command like this:

cmake -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=ON -DLLVM_USE_SPLIT_DWARF=ON -DCMAKE_INSTALL_PREFIX=C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\install -DLLVM_ENABLE_PROJECTS="llvm;clang;openmp" C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm

But when building it (ninja -j 7), it gives some odd error about DWARF symbols.

Suspect that I'm compiling on windows, and DWARF has something to do with ELF file format, which is linux.

Ok, so I thought - do I need that define ? Well, I've disabled it, and built using following command line again:

cmake -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_SHARED_LIBS=ON -DCMAKE_INSTALL_PREFIX=C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\install -DLLVM_ENABLE_PROJECTS="llvm;clang;openmp" C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm

Then I'm getting compilation errors like this:

[3/3782] Building CXX object lib\Demangle\CMakeFiles\LLVMDemangle.dir\MicrosoftDemangle.cpp.obj FAILED: lib/Demangle/CMakeFiles/LLVMDemangle.dir/MicrosoftDemangle.cpp.obj C:\PROGRA~2\MIB055~1\2019\ENTERP~2\VC\Tools\MSVC\1427~1.291\bin\Hostx86\x86\cl.exe /nologo /TP -DGTEST_HAS_RTTI=0 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_FILE_OFFSET_BITS=64 -D_HAS_EXCEPTIONS=0 -D_LARGEFILE_SOURCE -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib\Demangle -IC:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm\lib\Demangle -Iinclude -IC:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm\include /DWIN32 /D_WINDOWS /Zc:inline /Zc:strictStrings /Oi /Zc:rvalueCast /W4 -wd4141 -wd4146 -wd4180 -wd4244 -wd4258 -wd4267 -wd4291 -wd4345 -wd4351 -wd4355 -wd4456 -wd4457 -wd4458 -wd4459 -wd4503 -wd4624 -wd4722 -wd4800 -wd4100 -wd4127 -wd4512 -wd4505 -wd4610 -wd4510 -wd4702 -wd4245 -wd4706 -wd4310 -wd4701 -wd4703 -wd4389 -wd4611 -wd4805 -wd4204 -wd4577 -wd4091 -wd4592 -wd4319 -wd4709 -wd4324 -w14062 -we4238 /MD /Zi /O2 /Ob1 /DNDEBUG /EHs-c- /GR- /showIncludes /Folib\Demangle\CMakeFiles\LLVMDemangle.dir\MicrosoftDemangle.cpp.obj /Fdlib\Demangle\CMakeFiles\LLVMDemangle.dir\ /FS -c C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm\lib\Demangle\MicrosoftDemangle.cpp C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm\include\llvm/Demangle/MicrosoftDemangleNodes.h(258): error C2039: 'string': is not a member of 'std' C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.27.29110\include\array(20): note: see declaration of 'std' C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm\include\llvm/Demangle/MicrosoftDemangleNodes.h(258): error C3646: 'toString': unknown override specifier C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm\include\llvm/Demangle/MicrosoftDemangleNodes.h(258): error C2059: syntax error: '(' C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm\include\llvm/Demangle/MicrosoftDemangleNodes.h(258): error C2238: unexpected token(s) preceding ';' [8/3782] Building CXX object lib\Support\CMakeFiles\LLVMSupport.dir\AMDGPUMetadata.cpp.obj ninja: build stopped: subcommand failed.

Are llvm or clang some forks from official repositories, maybe because of that does not compile ?

Is it possible to get windows / visual studio support up-to-date ?

[tapika]

Short summary of what I have discovered by myself:

BUILD_SHARED_LIBS=ON cannot be used, as it turns every library into shared object library. From windows perspective this is no-no, as not every dll can expose small grained functions, and because of this no .lib's are generated.

This issue is being discussed in here: http://lists.llvm.org/pipermail/llvm-dev/2018-September/thread.html#126160

And I guess one approach is to compile clang normally (BUILD_SHARED_LIBS=OFF) - and that approach will work out of box, or dynamically - then it's possible to use LLVM_LINK_LLVM_DYLIB=ON - but it does not work because of cmake block in this file:

https://github.com/llvm/llvm-project/blob/master/llvm/tools/llvm-shlib/CMakeLists.txt#L14

(Also mentioned on forum)

[tapika]

This git repo seems to have bit older llvm & clang, and when compiling it with Visual studio - manifest as an error, similar to this one:

https://developercommunity.visualstudio.com/content/problem/1162322/inconsistent-behavior-between-llvm-and-std-impleme.html

Can be walkarounded by using -DHAVE_STD_IS_TRIVIALLY_COPYABLE=0 option.

[tapika]

In some cases test building fails (not sure because of which flags), but generally it's bit faster if you just disable a lot of tests builds, e.g. using parameters like:

-DLLVM_INCLUDE_EXAMPLES=OFF 
-DCLANG_INCLUDE_TESTS=0
-DLLVM_BUILD_TESTS=0
-DLLVM_BUILD_TOOLS=0
-DCLANG_BUILD_TOOLS=0
-DLLVM_BUILD_UTILS=0
-DCLANG_INCLUDE_DOCS=0 

[tapika]

Concerning: -DLLVM_USE_SPLIT_DWARF=ON

option - for windows it's not possible to use it - either because of compilation or linking errors. But I'm not sure whether some code dwarf::* in llvm/lib/IR/AsmWriter.cpp is needed or can be just disabled using #ifdef _WIN32 - haven't reached point of trying this out.

[tapika]

@davidbolvansky I've checked your commit, made 7.8.2018, in file: https://github.com/llvm/llvm-project/blob/master/llvm/tools/llvm-shlib/CMakeLists.txt#L14 - by commenting out those lines - build works also for Visual studio, but there is no __declspec(dllexport) used in any of class, as well there is no .def file for LLVM.dll. Dll does gets produces, but like you guess - it's not producing any .lib file, because not exported entries in .dll.

Is there any way it's possible to enable compilation to LLVM.dll also for windows - would be interested to try out newest [[clang::jit]] compilation on windows.

On even in theory this missing __declspec(dllexport) / .def file could be handled ?

[tapika]

Apparently there exists a python script, which is able to extract all symbol function names located in .lib file, which in a turn can be used as input file to dynamic library building:

python C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm\utils\extract_symbols.py --mangling microsoft C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm_build\lib\LLVMDemangle.lib C:\PrototypingQuick\cxxjit\llvm-project-cxxjit\llvm_build\lib\LLVMDlltoolDriver.lib --tools dumpbin >lib.exports

llvm\tools\llvm-shlib\CMakeLists.txt:

  target_link_libraries(LLVM PRIVATE ${LIB_NAMES})
-->
  add_llvm_symbol_exports(LLVM ${CMAKE_CURRENT_SOURCE_DIR}/llvm.exports)

and by adding this line it's possible to generate .def file from .exports file and then dll compiles with exported static library.

Maybe I need to recombine everything in cmake to make it fully functional.

[tapika]

Raised this ticket: https://developercommunity.visualstudio.com/content/problem/1215422/unresolved-external-symbol-when-linking-with-def-f.html At the moment closed from Microsoft side.