Subdoc fails with missing modmap files

[danakj]

A recent change in Clang 18 has made subdoc fail to execute the clang tool through libclang, generating an error for every .cc file it attempts to compile:

Error while processing /home/runner/work/subspace/subspace/sus/num/i8_unittest.cc.
error: no such file or directory: '@sus/CMakeFiles/subspace_unittests.dir/mem/addressof_unittest.cc.o.modmap'
[2/99] /home/runner/work/subspace/subspace/sus/mem/addressof_unittest.cc

Subspace is not using modules so there's no modmaps expected.

https://github.com/llvm/llvm-project/pull/75142 looks like it could be the reason.

First appeared on the try run here: https://github.com/chromium/subspace/pull/436

[danakj]

Same error occurs in clang 17, seems to be something else. Maybe a cmake change?

[danakj]

Yeah it's CMake 3.28 which added C++20 module support. CMake's ninja generator puts something like this into the command line for each file:

@third_party\\googletest\\googlemock\\CMakeFiles\\gmock_main.dir\\src\\gmock-all.cc.obj.modmap

And libclang doesn't seem to know what to do with it.

[danakj]

No clang docs seem to mention what the @ means either :(

[danakj]

it's from here: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/8611/diffs

Which adds the @foo.modmap to the command line, which CL docs say is a command-file which has other commands in it? https://learn.microsoft.com/en-us/cpp/build/reference/cl-command-files?view=msvc-170

But the files don't exist for me after building actual binaries either.

The test expects @foo.modmap or -fmodule-mapper=foo.modmap and GCC docs mention the latter: https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Module-Mapper.html

[danakj]

Oh this blog post discusses it: https://nibblestew.blogspot.com/2023/10/the-road-to-hell-is-paved-with-good.html

Ninja does not support dynamic compiler argument generation. Which is good, because it makes things faster, simpler and more reliable. This does not stop CMake, which hacked it on top anyway. The compiler command they invoke looks like the following.

clang++ <some compiler flags> @path/to/somefile.modmap <more flags> The @file syntax means "read the contents of the file and pretend that it contains command line arguments". The file itself is created at build time with a scanner program. Here's what its contents look like.

-x c++-module
-fmodule-output=CMakeFiles/modtest.dir/M0.pcm
-fmodule-file=M1=CMakeFiles/modtest.dir/M1.pcm
-fmodule-file=M2=CMakeFiles/modtest.dir/M2.pcm
-fmodule-file=M3=CMakeFiles/modtest.dir/M3.pcm
-fmodule-file=M4=CMakeFiles/modtest.dir/M4.pcm
-fmodule-file=M5=CMakeFiles/modtest.dir/M5.pcm
-fmodule-file=M6=CMakeFiles/modtest.dir/M6.pcm
-fmodule-file=M7=CMakeFiles/modtest.dir/M7.pcm
-fmodule-file=M8=CMakeFiles/modtest.dir/M8.pcm
-fmodule-file=M9=CMakeFiles/modtest.dir/M9.pcm

This is a file that cmake generates before running the compile command. Since clang tools are not cmake... the file does not exist and the resulting compile_commands.json is broken for any other use. Oh no.

[danakj]

Since only CMake will generate these files during compilation, we can't use them obviously. I think our only option is to strip them out of the command line. This will work until code actually uses modules at which point CMake will not provide a command line in compile_commands.json that is suitable to reproduce building the C++ file. And we may need to get off of CMake or something.

[danakj]

Posted to the cmake discussion here: https://discourse.cmake.org/t/how-to-control-the-location-of-the-c-20-binary-module-interface-bmi-output-directory/7968/13

For now, we will strip out the modmap command-file from the command-line that CMake generates specifically by looking for @.*\.modmap.

[danakj]

A followup blog post was made about this issue specifically: https://nibblestew.blogspot.com/2023/12/even-more-breakage-in-c-module-world.html