Load-time dynamic linking on Windows

[HertzDevil]

We already have two use cases for load-time dynamic linking on Windows: distributing the LLVM libraries, and linking to MPIR without license issues. (Run-time dynamic linking would be equivalent to porting Crystal::Loader to Windows.) #11573 shows that this is indeed possible, but we are still far from supporting it in the compiler itself. Here are some thoughts:

• Import libraries for DLLs are very different from static libraries; they cannot share the same names, so if $ORIGIN\lib is already used for the static libraries, we might need to find a different place for the import libraries, and the @[Link] annotations would have to be adjusted only on Windows depending on flag?(:static).
• For distribution the DLLs will most likely be located in the same directory as the compiler; it is standard practice to do so, and works for portable installations. As long as that same Crystal is available in %PATH%, so are the DLLs, but we might want to have more control over the DLL search order.
• Linking to the CRT dynamically means that the program will not run without the VC++ redistributable package. This might complicate packaging.
• Static libraries can already link to the CRT dynamically using /MD as well, or it can choose not to care with /Zl, which should be fine as our preludes already provide either libcmt or msvcrt. On the contrary, DLLs should not link to the CRT statically. This means there are at least 3 flavors of linking:
  • All compiler libraries are static, compiled with /MT (this is the current situation, and will likely be what is implied by the --static compiler flag eventually)
  • All compiler libraries are static, compiled with /MD
  • All compiler libraries are dynamic, compiled with /MD (this is what -Dpreview_dll implies)
• For at least one /MD scenario we should have a separate CI job that builds everything. A complete Crystal distribution, however, will probably include both static and dynamic libraries.
• We should also decide whether the compiler itself will be linked statically or dynamically.

[HertzDevil]

It looks like LLVM's CMakeLists.txt doesn't support BUILD_SHARED_LIBS on MSVC: https://llvm.org/docs/CMake.html#llvm-related-variables

It does have a LLVM_BUILD_LLVM_C_DYLIB variable which produces Release\lib\LLVM-C.lib and Release\bin\LLVM-C.dll. I do not know whether this DLL is sufficient for our LibLLVM. I am even less sure whether llvm_ext.obj would work.

[HertzDevil]

Related: #9278

[HertzDevil]

13436 will provide us a way to implement our own RPATH-style custom DLL searching, by calling LoadLibraryExA multiple times with different absolute paths, instead of once with an unqualified name. That would be good for crystal run and similar because it means we don't have to "install" the DLLs into a fixed location, and even Crystal itself doesn't need to be in %PATH%.

dumpbin /exports LLVM-C.dll shows that it provides everything in LibLLVM already except for the unused bindings in #13438. The problem is llvm_ext.cc; it depends on LLVM's C++ interfaces, so it must link against LLVM statically, which means a Crystal compiler distributed with just LLVM-C.dll cannot be used to rebuild Crystal. Removing the DIBuilder stuffs is easy, the rest not so much.

The top question is still figuring out how to distribute the static libraries and the DLL import libraries side-by-side. Generating import libraries from the DLLs themselves and placing them in the cache directory is probably not a viable option.

[HertzDevil]

Idea: if --static is provided, Crystal will manually look up foo-static.lib followed by foo.lib, otherwise Crystal tries foo-dynamic.lib and then foo.lib. This will solve the CRYSTAL_LIBRARY_PATH issue, but it still requires all libraries under --static to link against the static CRT (/MT), because that remains controlled by the same flag:

https://github.com/crystal-lang/crystal/blob/98c091ee9dd5a0e75b3dd47b642bd170b4c84a39/src/lib_c.cr#L1-L4

In the future this would become {{ flag?(:static) ? "libucrt" : "ucrt" }}. Ditto for the CRT startup library.

After #13436 we will perform an actual library search anyway, because we need to open the library files to search for the DLL imports. Two consequences are that we don't even need to pass /LIBPATH to cl.exe at all, and that we could produce a compiler error prior to linking if the library cannot be found.

[HertzDevil]

We have fully switched to dynamic linking and I think we could consider this issue finished once the following TODO is also addressed:

https://github.com/crystal-lang/crystal/blob/cdd9ccf460641ee17a603c67ff9d23aa1199f14f/src/compiler/crystal/loader/msvc.cr#L158-L161