Open permcody opened 10 months ago
While I am all for this change due to the timing results you showed: https://github.com/idaholab/moose/pull/26051#issuecomment-1864830775
I would still like to have finer control on subdirectory grouping. In the solid_mechanics
module, we have an enormous number of files under the materials
top-level directory, and so we would like to compile subdirectories under materials
as separate unity groups.
The current behavior would lead to out-of-memory errors while compiling on small/old machines.
Reason
https://github.com/idaholab/moose/pull/26051 changed the way that unity directories are lumped together. Instead of being generated per directory, unity files are generated at the topmost source directories for a project (just underneath
src
).Design
While the new design is better for nearly all applications (reduced Griffin compile time by almost 50%), it has made the exclusion of non-unity directories more brittle. The issue is there are many edge cases introduced with arbitrary directory structures and depths. For instance, consider the following tree.
With #26051, we will normally build a unity directory for "foo" and "bar", the latter of which will include files under "baz" and "buzz". However, if we want to exclude "buzz" from being built with unity. We can't just exclude
bar/baz/buzz
because the current logic doesn't see that full path as a unity directory in the first place. It only seesbar
. If we try to turn off unity at just thebar
level, that will stop unity from building at that top level, but then we don't automatically go underneath that folder to pick up individual source files without nudging the build system a bit more so we just exclude files in the downstream folders (e.g. explicitly listingbar/baz/buzz
too).The workaround right now is to explicitly list all of the directories under the top level directory that you turn off from unity which tells the build system to build those files individually. The system needs to be more robust so that if we turn off unity at any given folder at any level, all files found up to that point are still included the base unity directory "up" from that area and all files nested from that point down are explicitly separated without any further interaction from the user. Additionally, we'll need more checks for bad inputs such as turning unity on under a folder that has it turned off, or turning unity on and off on the same folder etc.
Impact
minor - most project have naturally separated their non-unity files to a separate top level folder right now so this issues doesn't really affect most projects. However, the current behavior is brittle and should be improved.