C++20 modules are in: discussing a sane (experimental) design for Meson

[germandiagogomez]

Hello everyone.

I am particularly interested in this topic. CMkae guys already started something:https://www.reddit.com/r/cpp/comments/axnwiz/cmake_gcc_module_proofofconcept/

Though I do not know the proposal well enough to propose a particular design myself, I think it would be a good idea to kick off discussion on strategies and module mapping, at least in the context of gcc, now that Modules have been voted in.

I would propose that this thread output is an initial design proposal to give a first try on implementation with the main high level details and strategies:

• file mapping handling
• module scanning
• ninja (and others later) rules to generate...

[dcbaker]

I'm not sure I have a good enough grasp on how C++20 modules are supposed to work, does anyone have a link to a good overview of them?

[jpakkane]

There isn't one. There are three different implementation that are different. The standardisation committee has promised to create a technical specification on how this "should work" but no-one has to actually follow that, though they are strongly recommended to.

[dcbaker]

Sigh, I love committee hand waving. Does GCC or Clang have any documents on how their implementation is supposed to work?

I'm dreading trying to get this information out of the ICC guys.

[germandiagogomez]

Food for thought about how to organize things, this will be a series I guess: https://vector-of-bool.github.io/2019/03/10/modules-1.html

[germandiagogomez]

First attemp at module mapping: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1484r1.pdf

GNU Make modules support prototype: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1602r0.pdf

@jpakkane this seems to be the start of some reasonable mapping and a proof of concept implementation of scanning dependencies?

[andreaskem]

Hi,

some links about modules in GCC and Clang: https://gcc.gnu.org/wiki/cxx-modules https://clang.llvm.org/docs/Modules.html

This will also affect e.g., ninja: https://github.com/ninja-build/ninja/pull/1521

It might be worth keeping in mind that e.g., Fortran has had modules (and submodules) for a while. CMake handles those quite well (ninja still needs some patches, I think).

[dcbaker]

Unless the ninja patches go upstream I don't think we can rely on ninja for this. We'll also have to figure out what XCode and VS are going to do about modules, I don't know if it's better to rely on ninja doing it for us if VS or XCode makes us implement this ourselves. (It doesn't seem unlikely that msbuild will rely on VS explicitly declaring the module relationship in the XML.

Meson already handles fortran modules and submodules I think, without using anything from ninja. @scivision has done a lot of work in that area.

[jhasse]

Unless the ninja patches go upstream I don't think we can rely on ninja for this.

FYI: It's very very likely that those Ninja patches will go upstream soon (I'm planning to review the PR this month) and be released with 1.10.

[mathstuf]

Hi, I'm the author of the Reddit post linked in the description and the CMake C++20 module support, so I can answer any questions you might have. I have a repository with C++ module use cases that I'd be happy to have meson build support added to.

Meson already handles fortran modules and submodules

Does it support generated fortran sources though? That's basically what requires the ninja patches we made. I'll be adding a C++ example for this next week to the above mentioned repository.

[mathstuf]

There isn't one. There are three different implementation that are different. The standardisation committee has promised to create a technical specification on how this "should work" but no-one has to actually follow that, though they are strongly recommended to.

I've gotten verbal confirmation that the major compiler vendors would be fine with providing information specified in this thread (continued here and here). I have the patch for GCC. Clang is next on my list, and it sounds like MSVC will as well. EDG said they'd have to follow whatever the other three do anyways, so we get them and all their backends by their other-compiler emulation. If you have any input on the format specified there, feel free to drop me a line.

[GunpowderGuy]

any progress ? does the meson team know when we should we expect this feature to be released ? i would like to try experimental support as soon as it is available

[scivision]

Just to add another CMake perspective, with details of how CMakes implement Fortran modules and submodules and thoughts how they'd do C++ https://mathstuf.fedorapeople.org/fortran-modules/fortran-modules.html

[mathstuf]

I'll also note that the required Ninja features have landed and will be included in the 1.10 release.

[mensinda]

Here are my thoughts on supporting C++20 modules in Meson:

The main problem that has to be solved with modules is the mapping of module names/identifiers to source files and the resolution of dependencies. There are multiple proposals on how such resolution mechanisms can be implemented. They range from an on-demand compiler, build system IPC mechanism and batch compilation to scanning the source files during the build step.

The main problem I see with these proposals is that they all (some more than others) depend on additional functionality in the compiler. In an ideal world, the required feature would be implemented by every compiler with the same logic, same limitations, same JSON format, etc. However, this might not necessarily be the case. As a result, code that compiles perfectly fine with GCC and Clang might fail to compile with ICC. In the (granted unlikely) worst-case scenario, a module handler for every compiler has to be written and maintained.

Even if there is an interface that is supported by most compilers, a fallback solution is still required for the remaining compilers. This would also leave us with two module mechanisms to maintain. Additionally, there would also be no guarantee that code that works with the "official" mechanism would also work with the "fallback" mechanism.

My solution: Let the build system do the entire dependency management by only providing the "fallback" solution. At least for the initial support until there is a universally accepted solution.

Since depending on the compiler in any way doesn't work, both batch compilation and IPC are out. This leaves us with scanning the source code with a builtin/shared tool. I am not going to repeat what is already listed there. I want to argue why I think that this scanner shouldn't be implemented in the compiler.

Naturally, this scanning step can not cover all possible macro/module interactions. If such an import statement is encountered where it does not know what to do (unknown macro in #if, etc.) an error is produced, and the build is aborted.

This artificially restricts what you can do with C++20 modules (module lookup is implementation-defined anyway, so this should be fine). While this restricts what developers can do initially, it would greatly increase the portabilety of the code since every compiler that supports modules should support explicit module mapping.

That being said, there is already a relatively small preprocessor implementation in python, writing one for the scanning tool shouldn't be that hard. So something like this could be supported out of the box:

#define FOO fooMod
#define BAR(x) FOO ## : ## x
export module FOO;
import BAR(bar);

And expanding the subset of module declarations that are supported later on is always possible.

The only real problem is dealing with the automatic #include conversion by the compiler. However, it should be possible to work around this by having a whitelist of compilers where the rules for automatic #include conversion can be retrieved. For all other compilers, this feature is disabled.

Another advantage would be that we have full control over the scanning tool and can remove and add features as we please. With a direct compiler integration, we would have to ask all the compiler vendors to change something.

Of course, writing such a scanner wouldn't be trivial to build and would ideally be used by multiple build systems (maybe some Meson CMake cooperation?). I would be personally happy to contribute some code towards this if there is a chance that this approach would be used in Meson.

These are my thoughts on this issue, so please correct me if I got anything wrong or missed something.

[mathstuf]

On August 25, 2019 5:32:19 PM EDT, Daniel Mensinger notifications@github.com wrote:

Here are my thoughts on supporting C++20 modules in Meson:

The main problem that has to be solved with modules is the mapping of module names/identifiers to source files and the resolution of dependencies. There are multiple proposals on how such resolution mechanisms can be implemented. They range from an on-demand compiler, build system IPC mechanism and batch compilation to scanning the source files during the build step.

I'm the author of the latter.

The main problem I see with these proposals is that they all (some more than others) depend on additional functionality in the compiler. In an ideal world, the required feature would be implemented by every compiler with the same logic, same limitations, same JSON format, etc. However, this might not necessarily be the case. As a result, code that compiles perfectly fine with GCC and Clang might fail to compile with ICC. In the (granted unlikely) worst-case scenario, a module handler for every compiler has to be written and maintained.

I have already gotten verbal confirmation that the JSON format is ok with GCC (I have a patch), Clang developers, and MSVC. When I asked EDG developers, they said they have to implement whatever the other three do anyways. I don't think we'll have issues with compiler support (it's really easy anyways compared to having modules at all). Flag spelling may differ, but GCC and Clang will likely be the same.

We at Kitware also have contacts within the Fortran community and may be able to share the format with those compilers too through its standardization process.

Even if there is an interface that is supported by most compilers, a fallback solution is still required for the remaining compilers. This would also leave us with two module mechanisms to maintain. Additionally, there would also be no guarantee that code that works with the "official" mechanism would also work with the "fallback" mechanism.

My solution: Let the build system do the entire dependency management by only providing the "fallback" solution. At least for the initial support until there is a universally accepted solution.

Since depending on the compiler in any way doesn't work, both batch compilation and IPC are out. This leaves us with scanning the source code with a builtin/shared tool. I am not going to repeat what is already listed there. I want to argue why I think that this scanner shouldn't be implemented in the compiler.

See clang-scan-deps (on the phone, can get a link tomorrow) for such a tool. Emulating the compilers is hard though, so I think that is probably best at first. Such a tool would be useful for faster scanning if it proves to be too slow.

Naturally, this scanning step can not cover all possible macro/module interactions. If such an import statement is encountered where it does not know what to do (unknown macro in #if, etc.) an error is produced, and the build is aborted.

This artificially restricts what you can do with C++20 modules (module lookup is implementation-defined anyway, so this should be fine). While this restricts what developers can do initially, it would greatly increase the portabilety of the code since every compiler that supports modules should support explicit module mapping.

That being said, there is already a relatively small preprocessor implementation in python, writing one for the scanning tool shouldn't be that hard. So something like this could be supported out of the box:

#define FOO fooMod
#define BAR(x) FOO ## : ## x
export module FOO;
import BAR(bar);

This is not even the surface of the corner cases :) . Please join #modules and #sg15_tooling on the cpplang slack to discuss with other stakeholders.

And expanding the subset of module declarations that are supported later on is always possible.

The only real problem is dealing with the automatic #include conversion by the compiler. However, it should be possible to work around this by having a whitelist of compilers where the rules for automatic #include conversion can be retrieved. For all other compilers, this feature is disabled.

I'll expand on my thoughts here more tomorrow. Short answer: no conversion at all unless told so by the build system (because it needs to know).

Another advantage would be that we have full control over the scanning tool and can remove and add features as we please. With a direct compiler integration, we would have to ask all the compiler vendors to change something.

That's why we're working with ISO where all the implementors are present :) .

Of course, writing such a scanner wouldn't be trivial to build and would ideally be used by multiple build systems (maybe some Meson CMake cooperation?). I would be personally happy to contribute some code towards this if there is a chance that this approach would be used in Meson.

While I'm not opposed to collaborations, there is effort on such a tool in the Clang world already, so let's collaborate there.

Thanks,

--Ben

[mensinda]

See clang-scan-deps (on the phone, can get a link tomorrow) for such a tool.

I wasn't aware that there is already a project. Less work for us then :)

Emulating the compilers is hard though, so I think that is probably best at first. Such a tool would be useful for faster scanning if it proves to be too slow.

My idea was specifically not to emulate the compiler, rather provide a tool that works for 90%-99% of use cases and print an error for the rest. Granted, restricting the user is not very user-friendly, but it would guarantee that the code works with every setup.

This is not even the surface of the corner cases :) . Please join #modules and #sg15_tooling on the cpplang slack to discuss with other stakeholders.

Sure, my point was (primarily) to give an error and abort the build if such an edge case is discovered. Then support for these edge cases might be added later.

I have already gotten verbal confirmation that the JSON format is ok with GCC (I have a patch), Clang developers, and MSVC. When I asked EDG developers, they said they have to implement whatever the other three do anyways. I don't think we'll have issues with compiler support (it's really easy anyways compared to having modules at all). Flag spelling may differ, but GCC and Clang will likely be the same.

That's why we're working with ISO where all the implementors are present :) .

If this format becomes the standard for dependency scanning and every compiler supports it, then this would be the best-case scenario, and my main issue for relying on the compiler is resolved :)

In this case, the only remaining issue would be speed. Scanning for dependencies should be nearly instant (I haven't tested any compiler implementation yet, so I don't know about the current performance). It would also be really useful if the compiler would support scanning multiple files at once. This could reduce the process spawning overhead, especially on Windows,

[mathstuf]

Sorry, got busy and didn't circle back on this. Some links:

• clang-scan-deps review: https://reviews.llvm.org/D53354
• cfe-dev thread: https://lists.llvm.org/pipermail/cfe-dev/2019-August/063072.html

My idea was specifically not to emulate the compiler, rather provide a tool that works for 90%-99% of use cases and print an error for the rest. Granted, restricting the user is not very user-friendly, but it would guarantee that the code works with every setup.

Well, the fundamental problem seems to be something like this:

#if __has_feature(frobnitz)
import frobnitz;
#else
import fallback.frobnitz;
#endif

The preprocessor definitions are easy to get, but the feature set is not so easy. __has_attribute also is likely in the same bucket.

In this case, the only remaining issue would be speed. Scanning for dependencies should be nearly instant (I haven't tested any compiler implementation yet, so I don't know about the current performance). It would also be really useful if the compiler would support scanning multiple files at once. This could reduce the process spawning overhead, especially on Windows,

Our prior paper (which missed mailings, but is available here) showed how per-source, per-target, and whole-project scanning is possible and isomorphic in terms of build correctness (the difference is mainly in incremental build work reductions). I think clang-scan-deps is likely to support batch scanning, but maybe not right away.

[mensinda]

Well, the fundamental problem seems to be something like this:

#if __has_feature(frobnitz)
import frobnitz;
#else
import fallback.frobnitz;
#endif

Even clang-scan-deps would have problems with this. There is no way any tool could reliably detect this for all compilers and compiler flags. That's why I would propose to specifically disallow such constructs by producing a hard error in the scanning step (with some helpful message on how to work around it, if possible). I am the first one to admit that this isn't user-friendly, but these cases should be the exception and not the norm.

Also, one can work around this in meson (and CMake) by using configure_file() with data from the compiler methods to generate a config.h:

#pragma once
#define HAS_FEATURE_frobnitz 1

Then your original case can be rewritten as:

#include "config.h"
#if HAS_FEATURE_frobnitz
import frobnitz;
#else
import fallback.frobnitz;
#endif

I will repeat that I am fully aware that this adds additional burden on the user and that not all (presumably valid) use cases can be solved like this one. However, in my opinion, the benefits outweigh the costs here. Such a scanner would be limited by design but blazing fast and easy to maintain. Additionally, I would argue that defining all HAS_FEATURE_*, etc. in a separate config.h generated by the build system is preferable anyway because it makes the code easier to understand.

This scanner would ideally be fairly compact and have no external dependencies. This is because it is even lower on the software stack than build systems like meson and CMake (but not ninja). So only the C++ or python standard library would be allowed as well as a custom build.py to bootstrap the project.

That's also why I don't particularly like the idea of a clang-scan-deps, exactly because it depends on LLVM (LLVM is built with CMake, so there would also be a circular dependency 😃). Just installing the LLVM toolchain to have support for module scanning seems overkill for me. On Windows with VS and if you are only using GCC on Linux, you don't need and/or want LLVM. Especially if you want to keep a docker image small and would only need clang-scan-deps for module detection.

PS: How do you actually use clang-scan-deps? I managed to install it from llvm-git, but I can't figure out how to scan stuff.

[mathstuf]

I will repeat that I am fully aware that this adds additional burden on the user and that not all (presumably valid) use cases can be solved like this one. However, in my opinion, the benefits outweigh the costs here. Such a scanner would be limited by design but blazing fast and easy to maintain. Additionally, I would argue that defining all HASFEATURE*, etc. in a separate config.h generated by the build system is preferable anyway because it makes the code easier to understand.

The compiler is likely to also be useful as a scanner (GCC is at least). Just probably not as fast as one would like. An option to say "I don't do silly things" to allow using clang-scan-deps seems reasonable though. That also nicely removes the dep cycle you're worried about.

(LLVM is built with CMake, so there would also be a circular dependency)

Well, CMake is unlikely to use modules itself until…oh 2029 probably based on the speed of C++11 adoption of our target compilers, so that part is not a cycle :) .

PS: How do you actually use clang-scan-deps? I managed to install it from llvm-git, but I can't figure out how to scan stuff.

That, I'm not sure. You'll have to ask Michael Spencer (one of its developers).

[Bigcheese]

Our prior paper (which missed mailings, but is available here) showed how per-source, per-target, and whole-project scanning is possible and isomorphic in terms of build correctness (the difference is mainly in incremental build work reductions). I think clang-scan-deps is likely to support batch scanning, but maybe not right away.

clang-scan-deps is designed for batch processing. It doesn't even have an interface for scanning a single file (other than providing it a batch of a single file).

[Bigcheese]

PS: How do you actually use clang-scan-deps? I managed to install it from llvm-git, but I can't figure out how to scan stuff.

clang-scan-deps expects a compilation database. It needs a full command line to know how to preprocess each file.

$ clang-scan-deps --compilation-database=db.json

Upstream clang-scan-deps doesn't currently support reporting modules deps yet.

[germandiagogomez]

@jpakkane Ninja v 1.10 seems to have initial support for modules...? https://github.com/ninja-build/ninja/pull/1521

[flajann2]

So, as of today, where do we stand with meson support for C++20 modules? I am willing to ditch CMake for Meson at this point if that support is there. And it is unclear where or how the Ninja support factors in with Meson.

CMake supports Fortran, but no indication as far as I know as to when they will port that functionality over to C++20.

[jpakkane]

The module support is not implemented yet. The compiler toolchains do not support it that well yet either (at least last I looked, maybe it has changed). Once Ninja and toolchains have the necessary bits (and they are sufficiently stable) we'll add module support.

[mathstuf]

FWIW, ninja does have the necessary bits merged now (and released in 1.10). Compilers still need a spec to write against for dependency info (at least for CMake's approach), but that's mostly on me to work on for SG15. CMake would then need some CMake-side changes for usage requirements related to modules.

[germandiagogomez]

This is an implementation of http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p1184r1.pdf

Module mapper implementation relicensed and freely available:

 - http://lists.llvm.org/pipermail/cfe-dev/2020-May/065487.html

[germandiagogomez]

libcody, build system/compiler communication has been added to GCC. Might be relevant to Meson?

https://github.com/urnathan/libcody

[mathstuf]

If meson wants to run something during the build and monitor that communication, it can help. But AFAIK, ninja has no way of specifying such a tool to communicate with, so you're likely left with the compiler sending off a message to some blocking process which then starts/communicates with the necessary tool (and all the locking fun that sounds like it involves).

While that is a solution to building modules, I don't find it a particularly scalable one (you don't know what is needed until the compiler has started and now you're consuming memory, process slots, and doing IPC to figure out what work to do next). It's probably fine for small projects which are writing their makefiles by hand though, but once there's a complicated dependency graph, I only forsee contention.

[lb90]

Some news on the MSVC side: https://devblogs.microsoft.com/cppblog/introducing-source-dependency-reporting-with-msvc-in-visual-studio-2019-version-16-7/

[johan-boule]

Some news on the MSVC side: https://devblogs.microsoft.com/cppblog/introducing-source-dependency-reporting-with-msvc-in-visual-studio-2019-version-16-7/

MSVC

I did a quick test with a file main.cpp which starts with import hello; and the command line cl -std:c++latest -sourceDependencies main.json main.cpp and it bails out with error main.cpp(1): error C2230: could not find module 'hello' . So, this new option does not meet the needs of a build system, which is to obtain the source dependencies. For that, the command should not fail and outputs that main.cpp depends on a module named hello. Also, if you add a -P option to only do the preprocessing, it doesn't fail, but the main.json file obtained does not contain anything in the ImportedModules section. I would have expected the name hello there.

VS/MSBuild might be using something else that they didn't talk about because the build output window does show some form of source dependency scanning is happening and it's able to build IFC in the correct order.

Clang

I also checked again today the clang-scan-dep tool in the llvm-project git master branch, and it doesn't seem to handle modules at all.

GCC

So, amongst the three major compilers, it seems to me the one that currently has the better chance to receive support from a build system is GCC with the git devel/c++-modules branch. This is the only system that's currently documented and works as advertised.

[MikeGitb]

Has anyone filed a bugreport/feature request with msvc that the dependency information should be produced without requiring a sucessfull compilation?

[jpakkane]

I have spoken with some of them informally, so they do know what the practical requirements are. They do seem to want to make it possible to use external build systems so I'd expect this to work eventually but as of date there are no official statements one way or the other.

[johan-boule]

I tried VS 2019 16.9 Preview 1 released today. Looking at MSBuild's detailed log, I see that msvc's new -sourceDependencies option is used as -sourceDependencies:directives ... And with it, it seems to be able to extract what modules a source file is importing and exporting. So, the MS dev blog that talked a few days ago about this new option was perhaps just missing that ":directives" part in their example command line, which made it useless! It looks to me it might be possible to make use of this new option to write a build system capable of handling modules with msvc.

EDIT: On further investigation, this is far from being what we expect. It seems it's checking whether the imported header source file exists, but it doesn't seem to preprocess it. So, macros are not exported.

[mathstuf]

I haven't seen that format itself in detail, but the strategies described in our paper (presented to ISO C++ at the Kona 2019 meeting) are what is needed to properly support module compilation (C++'s module system is very analogous to Fortran's from the build system perspective).

There will be further updates, but it is expected that the format described in P1689R2 (further tweaks will be done, but the basics have been generally agreed upon) will be common between compiler vendors. I suspect that the team that implemented the /sourceDependencies:directives output format is not aware of the C++ committee's work in this area :/ . Further, getting Fortran compilers to also be able to generate this information would be ideal (hence the lack of C++-specific terminology in places).

[mattyclarkson]

Associated: https://build2.org/blog/build2-cxx20-modules-gcc.xhtml

[scivision]

CMake has a pull request enabling C++20 modules, at least for MSVC: https://gitlab.kitware.com/cmake/cmake/-/merge_requests/5926

Update: this pull request was merged and will be in CMake 3.21.0

[mathstuf]

FWIW, that's just with Visual Studio using VS' support. Full support requires compiler cooperation.

[GunpowderGuy]

What progress is being made on this issue? The only other active thread i can think of ( https://github.com/mesonbuild/meson/issues/8213 ) hasn't seen any replies

[germandiagogomez]

Is there any chance of having module compilation support in GCC 11?

[GunpowderGuy]

@germandiagogomez Build2 dev's say module support is possible with GCC 11. That compiler still has a few bugs ( which should be corrected in GCC 11.2 or 11.3 ) but they shouldn't prevent the build system from supporting modules with it

[Makogan]

I am also going to generate some noise here, I am wondering what is the current state of this feature.

[ghost]

Some folks wrote comments with useful information, but that was 7-8 months ago. I imagine the state has changed at least a little bit here. Is there a common way of handling this for different compilers at this point?

[mathstuf]

Compilers are going towards providing tools to provide dependency information via the format specified in P1689. Once the tools are available that provide that information, build.ninja files which can perform that scanning at buildtime can be written (meson will still need to do some coordination to resolve the requires/provides links across the build graph; the "collator" in this paper).

[johan-boule]

IMO, the best bet for meson to fully support modules in a reasonable time (i.e. in this decade) is to do the work of executing the dependency graph by itself, rather than delegating this to an external tool.

Correct me if I'm wrong, but meson does not aim at being a build system generator like cmake, and ninja is just an implementation detail hidden to the users, so it can realistically swap ninja for its own code.

But why the need, you may say.

Because GCC provides module support for build systems though a bidirectional communication channel between the compiler and the build system.

IIRC, GCC's mapper system, also described in P1184, has been developed nearly three years ago and merged into the main branch more than a year ago.

If your build system keeps on using ninja, you'll have a hard time finding a clunky way to redirect compiler requests back into meson and then back into ninja. I even think that's not possible without forking ninja and hack and maintain your own version. My own experience is that writing in python a graph executor is not that much work. In other words, relying on ninja does not save you much work, and currently blocks further development.

For insights, you can see in P1602 the amount of work it took to augment GNU make's capabilities. I don't see any such effort being put into ninja.

Build2 has been supporting GCC's mapper system officially for a long time. Since it doesn't depend on any external tool, it's been able to progress much faster than other build systems.

[eli-schwartz]

Correct me if I'm wrong, but meson does not aim at being a build system generator like cmake, and ninja is just an implementation detail hidden to the users, so it can realistically swap ninja for its own code.

If meson is not a build system generator, what is it? Baked ziti?

How is ninja an implementation detail hidden to users, considering that the output of meson is a build.ninja file traditionally executed by running ninja? Just like, well, cmake?

Also are these design decisions you are recommending intended to work with compilers other than GCC + some confusing IPC mechanism?

[mathstuf]

If your build system keeps on using ninja, you'll have a hard time finding a clunky way to redirect compiler requests back into meson and then back into ninja. I even think that's not possible without forking ninja and hack and maintain your own version. My own experience is that writing in python a graph executor is not that much work. In other words, relying on ninja does not save you much work, and currently blocks further development.

Ninja was patched since 2010 by Brad King to support the features required. It took until 2019 to get them merged (because the maintainers didn't see a need for it since I presume they care ~0% about Fortran and more than that for C++20 module support which is isomorphic to Fortran's module system, for better or worse). Yeah, it's not turn-key, but neither is supporting it in make (of which CMake targets POSIX make, so gmake isn't something CMake has been updated to require for things like GCC-specific support).

FWIW, the support CMake has works with upstream ninja (1.10.0) and a patched GCC to output the required information. See https://github.com/mathstuf/cxx-modules-sandbox/ for details.

I'll also note that the bi-directional communication has unbounded system resource requirements and the build tool could run into processes or open file descriptor limits trying to resolve import requirements (as there's no guarantee that a depended-upon module actually exists), so I personally see it as a toy solution and not something that will actually solve the problem presented by C++20 modules for build systems.

[eli-schwartz]

(Also, the explanation you just gave is not the first time in this very ticket that this information has been pointed out, and by you. :D)

[jpakkane]

I have been meaning to do GCC module support but the compiler on my main dev machines is not new enough so I have done other things instead.

Expanding the scanner to support GCC in addition to MSVC should be straigthforward but there are some issues such as:

• The test suite uses .ixx as the module file extension but GCC does not classify that as a C++ extension so it fails
• You need to add special flags to GCC to make it do modules
• From memory GCC wants to put module outputs to a hidden directory "somewhere", MSVC puts them next to the object files
• Some other things I don't remember offhand

I'm not a particular fan of the module mapper thing. The current scanner we have works with MSVC and also with all Fortran compilations. It spawns only one process per build target so the overhead is neglible. And further, invoking the compilations in the correct order is not particularly hard once you have scanned the sources and implementing a full fledged mapping system callback thingy seems like a lot more work. Unless there is some very big upside lurking, it does not seem to be worth it.

[mathstuf]

Without a build-time scanner, problems arise with generated sources AFAIK. There may be other downsides, but none come to mind right away. If Meson doesn't want to support generating module-using (or providing) sources at build time, that's a decision of the project that is fully available (AIUI, Meson also doesn't support per-source compilation flags, so there's room for different tradeoffs). It seems to me like you're referring to a build-time static ordering of scanning here (due to the no "system callback thingy" remark). However, the ordering can change based on contents of files, so unless Meson is going to (attempt to) regenerate build.ninja for any source file change (that might contain a module import or export), the scanning needs to be done during the build as supported by the dyndep feature of ninja.

[jpakkane]

However, the ordering can change based on contents of files, so unless Meson is going to (attempt to) regenerate build.ninja for any source file change (that might contain a module import or export), the scanning needs to be done during the build as supported by the dyndep feature of ninja.

It has supported dyndeps for almost a year now. That is how the MSVC module support is implemented.