"multibuilds" - a plan to harmonize conditional compilation with compile errors, documentation, and IDEs

[andrewrk]

Here are some problems to solve:

• Functions which are only called with one particular set of build flags, such as windows-only functions, would be omitted from documentation and IDE features when targeting non-windows
• Compile errors for unused things (#335). This is currently not possible because, if a function is unused, maybe it is used when a different set of build flags are provided.
• In order for documentation to have resolved error sets, the error inference must be evaluated. The error sets might be different depending on build flags
  • this is even true for docs of generic functions
• whether or not code is unreachable can depend on comptime stuff such as build flags
• Functions which take comptime parameters, what parameters are even valid? Only actual usage can fully describe what code paths are valid.

And so here is my plan: multibuilds - The idea of building not just the native target, not just a cross compiled target - but telling the compiler about the complete set of targets / build flags that are supported by the project. This gives the compiler the ability to assume that any branches not taken, are actually dead even across all the desired set of builds.

Zig is in a unique position to explore this concept, with its strong emphasis on cross compilation support.

What this looks like from an implementation perspective is roughly:

• Add to the interface of the compiler ability to specify a set of targets/build options, rather than a single one.
• For each target/build options, the user can choose whether this is just for analysis purposes or whether it should actually produce output.
• Semantic analysis must build all the targets at once, and when there is a comptime branch which decides between targets/build options, the analysis must take both branches, keeping track of the builds separately. However it has to be able to connect them together at the end, for diagnostics purposes and for some advanced analysis that considers the full set of targets.

[matklad]

Hey, I've been working on IDEs for Rust for several year and would love to share my experience about IDEs vs conditional compilation (not that I have any insightful solutions).

What we are doing in rust-analyzer (in a small capacity at the moment) is the multibuild approach: we "instantiate" Rust crates several times with different cfg options. This in general swimmingly inside the compiler, but feels awkward at the boundary between the compilery parts and the IDE parts. Specifically, it is convenient in the IDE if you can unambiguously map source-level element (like function) to a semantic representation. With multibuild, the relationship is one-to-many, and you need some custom code to deal with it (which in the simple case would be just picking the "default" semantic repr). As an example of awkwardness, consider a function foo which has two arguments on unix and three arguments on windows. How do you render it in copmletion popup? Two items? One item with how many arguments? One multi-item with "overloads"?

Still, despite this being awkward, I don't know of a better solution for Rust style conditional compilation. However, I have a suspicion that it is possible to design conditional compilation language feature in such a way that IDE support comes naturally. Specifically, I think it should be possible to move conditional compilation from an early phase of compilation (just after parsing) to a much later phase (just before codegen). That is, I think it should be possible to make name resolution independent of the current set of cfg flags. An example of this approach is Kotlin multi-platform support:

• https://kotlinlang.org/docs/reference/platform-specific-declarations.html
• https://kotlinlang.org/docs/reference/multiplatform.html

Kotlin needs conditional compilation primarily because it targets JVM, native and JavaScript, and, to bind to platform functionality, you need a custom per-platform code. The way this feature works is roughly that you write a "header" file, which is shared between all platform, and you write an "implementation" file per platform.

Hope this is helpful!

[Validark]

As an example of awkwardness, consider a function foo which has two arguments on unix and three arguments on windows. How do you render it in completion popup? Two items? One item with how many arguments? One multi-item with "overloads"?

I think the IDE should render it based on the code context. If you're writing code in the Windows context, you get the Windows function. If you're writing code in the Unix context, you get the Unix function. If you're writing code in an ambiguous context, the IDE should tell you that the function cannot be called out of context and you have to insert some logic to check whether the system is Windows or Unix. For Zig, it could even be possible to autocomplete the code to something like:

if isWindows
    foo(arg1, arg2, arg3)
else if isUnix
    foo(arg1, arg2)

[Arya-Elfren]

I think the IDE should render it based on the code context.

This should be possible as an extension of this proposal.

The compiler goes through a function and says "for this line I need to evaluate it with these config options {win, osx, x86...}". Then the IDE asks: "If I were to insert a line here, what build options would I have to satisfy simultaneously?".

This almost looks like type narrowing, build flag narrowing?

If you have an incompatible superset it would suggest a switch statement.

---

I don't know if this is a good idea, or feasible.

[matu3ba]

Here a few possible approaches:

• One way for multibuilds would be to generate the broadest supserset of target features and do forward/backward reachability analysis from each starting/end point in zls. Each interesting symbold would then be expressed as (symbol, src:line, target_group, ?definition_src:line) and target_group be index into set of target sets. However, I suspect you would end up with a similar too many variables problem as universal libc minifying, even if finding a small encoding for src:line. Also, this requires 1. complete enough target feature logic, 2. handling target features as set, 3. sema logic on target sets outside of sema (via query infrastructure or reimplementing/exposing sema for zls somehow), 4. emitting info for querying by zls.
• Another way to do multibuilds would be to emit target info group splits in semantic analysis based on starting points in user-provided target inputs by zig build and collect the same info (src:line, target_group, ?definition_src:line) lazily by rerunning for missing knowledge.
• Yet another simpler way to do multibuilds would be to only query the target sets and let the user decide which src location infos to collect or to build.

One tradeoff with all above approaches is that the user has no (easy) way to plug own target logic.