Add support for modules

[GPMueller]

https://clang.llvm.org/docs/Modules.html

[GPMueller]

C++20 modules are currently not yet available in clang.

However, they have become available in gcc trunk (wiki and docs). This means if issue #123 were fully implemented, we would already be able to develop this feature on the gcc-toolchain.

[GPMueller]

Note, Bazel appears to now support building modules with clang: https://buildingblock.ai/cpp20-modules-bazel Note in particular their flags

compile:
-std=c++20
-fmodules
-fno-implicit-modules
-fbuiltin-module-map
-stdlib=libc++

link:
-stdlib=libc++

[GPMueller]

As demonstrated on the merged PR #129, it is possible to create custom toolchains now, meaning we could test modules compilation with gcc now.

A note regarding build stages: We currently separately generate dependency-files and then compile. We could add a module target type, which does this to figure out the module dependencies, but I believe even in the context of building modules, non-module targets probably don't need two build stages.

[GPMueller]

Some links for reference

• https://github.com/compiler-explorer/compiler-explorer/issues/1892
• https://www.modernescpp.com/index.php/c-20-modules
• https://godbolt.org/z/zs8KPWfz9
• https://quuxplusone.github.io/blog/2019/11/07/modular-hello-world
• https://badlydrawnrod.github.io/posts/2021/12/12/cpp-modules
• https://blog.ecosta.dev/en/tech/cpp-modules-with-clang
• https://cppcast.com/distributing-cpp-modules/

[naturallymitchell]

@GPMueller, have you had any more recent experience with modules?

[GPMueller]

Hi @naturallymitchell. Unfortunately not - did something get updated on clang's side? If it should be possible to do a little modules demo with clang + clang-build by now, I'd like to give it another try.

[kassane]

Some links for reference

• https://github.com/xmake-io/xmake/pull/3255

[GPMueller]

CMake is progressing: https://www.kitware.com/import-cmake-c20-modules

[GPMueller]

It completely went past me that clang-15 added standard C++ modules support 🎉 https://releases.llvm.org/15.0.0/tools/clang/docs/StandardCPlusPlusModules.html

There's now a clang-15-based modules examples at

• https://github.com/camio/modules-example (I tested and was able to build and run it)
• https://gist.github.com/DanielaE/39e270adb6ab27d12c61e1491d355d30

@Nohs Looks like we could finally start creating and testing a modules-build mechanism.

[GPMueller]

With relatively few modifications I'm able to build the trivial case (see PR #139), but the following example from the clang docs doesn't seem to be possible without either

• either the user specifying the dependencies between module partition files, letting clang-build ensure the correct build order and serializing parts that cannot be done in parallel
• or clang-build parsing module source files manually, in order to determine their dependency graph
• or requiring an update to clang itself, giving the -MMD/--write-user-dependencies flags the ability to also output import dependencies instead of only #include dependencies

The 3rd option, i.e. clang determining the dependencies, is the solution I'm hoping for, as I believe that would be the most stable and toolable solution. While clang-build would still have to reconstruct the dependency-graph out of multiple dependency files, working with what the compiler sees seems the most reliable to me.

[kassane]

I don't know if anyone had a similar problem with LLVM-linker (LLD).

Tests on zig c++ (clang 15 + lld builtins) https://gist.github.com/kassane/7e9a2da137e13eb6e1dbab726693bdb7

$> zig c++ -std=c++20 -fmodules -fbuiltin-module-map -fmodules-ts -Xclang -emit-module-interface -c moduleTest.cpp -o moduleTest.pcm
LLD Link... ld.lld: error: /home/kassane/.cache/zig/o/ec639bf9d5e7c589c4e222dd65d28017/moduleTest.o:995: unclosed quote

[naturallymitchell]

@GPMueller

https://github.com/joblobob/ray/tree/feature/anewhope (from: https://www.reddit.com/r/cpp/comments/1255t31/c20_modules_support_for_cmake/ )

[GPMueller]

Thanks @naturallymitchell that might be a useful reference in the future.

IIRC CMake achieves its modules support by constructing the module dependencies itself, which I don't want to do in clang-build. I'm waiting to see whether clang will provide a way to output module dependencies, analogous to (or included in) the include-dependencies it can output using the -MMD/--write-user-dependencies flag.

[GPMueller]

By now the LLVM docs actually describe how to get the module dependencies using clang-scan-deps -format=p1689: https://clang.llvm.org/docs/StandardCPlusPlusModules.html#discovering-dependencies (see P1689 for the file format description).

See also this CMake blog post for the invocations CMake generates: https://www.kitware.com/import-cmake-the-experiment-is-over

[1/8] "/Users/hoffman/Work/modules/llvm16-inst/bin/clang-scan-deps" -format=p1689 -- /Users/hoffman/Work/modules/llvm16-inst/bin/clang++   -std=c++20 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk -x c++ /Users/hoffman/Work/modules/blog/main.cxx -c -o CMakeFiles/hello.dir/main.cxx.o -MT CMakeFiles/hello.dir/main.cxx.o.ddi -MD -MF CMakeFiles/hello.dir/main.cxx.o.ddi.d > CMakeFiles/hello.dir/main.cxx.o.ddi
[2/8] "/Users/hoffman/Work/modules/llvm16-inst/bin/clang-scan-deps" -format=p1689 -- /Users/hoffman/Work/modules/llvm16-inst/bin/clang++   -std=c++20 -arch arm64 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk -x c++ /Users/hoffman/Work/modules/blog/foo.cxx -c -o CMakeFiles/foo.dir/foo.cxx.o -MT CMakeFiles/foo.dir/foo.cxx.o.ddi -MD -MF CMakeFiles/foo.dir/foo.cxx.o.ddi.d > CMakeFiles/foo.dir/foo.cxx.o.ddi

As clang-scan-deps apparently allows preprocessing and generating depfiles, it should now be straightforward to implement proper modules support for the LLVM toolchain.