Proposal: Conditional Compilation

[kitsonk]

Proposal: Conditional Compilation

Problem Statement

At design time, developers often find that they need to deal with certain scenarios to make their code ubiquitous and runs in every environment and under every runtime condition. At build time however, they want to emit code that is more suited for the runtime environment that they are targetting by not emitting code that is relevant to that environment.

This is directly related to #449 but it also covers some other issues in a similar problem space.

Similar Functionality

There are several other examples of apporaches to solving this problem:

• In C#, this is solved via conditional flags as well well as conditional symbols.
• In Dojo, this was solved via adopting has.js and static flags in the build tool that would allow build time "dead code removal".
• IE used to support conditional compilation using comments.
• UglifyJS accomplishes this via assertion of constants coupled with dead code removal.

  Considerations

Most of the solutions above use "magic" language features that significantly affect the AST of the code. One of the benefits of the has.js approach is that the code is transparent for runtime feature detection and build time optimisation. For example, the following would be how design time would work:

has.add('host-node', (typeof process == "object" && process.versions && process.versions.node && process.versions.v8));

if (has('host-node')) {
    /* do something node */
}
else {
    /* do something non-node */
}

If you then wanted to do a build that targeted NodeJS, then you would simply assert to the build tool (staticHasFlags) that instead of detecting that feature at runtime, host-node was in fact true. The build tool would then realise that the else branch was unreachable and remove that branch from the built code.

Because the solution sits entirely within the language syntax without any sort of "magical" directives or syntax, it does not take a lot of knowledge for a developer to leverage it.

Also by doing this, you do not have to do heavy changes to the AST as part of the complication process and it should be easy to identify branches that are "dead" and can be dropped out of the emit.

Of course this approach doesn't specifically address conditionality of other language features, like the ability to conditionally load modules or conditional classes, though there are other features being introduced in TypeScript (e.g. local types #3266) which when coupled with this would address conditionality of other language features.

Proposed Changes

In order to support conditional compile time emitting, there needs to be a language mechanic to identify blocks of code that should be emitted under certain conditions and a mechanism for determining if they are to be emitted. There also needs to be a mechanism to determine these conditions at compile time.

Defining a Conditional Identifier at Design Time

It is proposed that a new keyword is introduced to allow the introduction of a different class of identifier that is neither a variable or a constant. Introduction of a TypeScript only keyword should not be taken lightly and it is proposed that either condition or has is used to express these identifiers. When expressed at design time, the identifier will be given a value which can be evaluated at runtime, with block scope. This then can be substituted though a compile time with another value.

Of the two keywords, this proposal suggests that has is more functional in meaning, but might be less desirable because of potential for existing code breakage, but examples utlise the has keyword.

For example, in TypeScript the following would be a way of declaring a condition:

has hostNode: boolean = Boolean(typeof process == "object" && process.versions && process.versions.node && process.versions.v8);

if (hostNode) {
    console.log('You are running under node.');
}
else {
    console.log('You are not running under node.');
}

This would then emit, assuming there is no compile time substitutional value available (and targeting ES6) as:

const hostNode = Boolean(typeof process == "object" && process.versions && process.versions.node && process.versions.v8);
if (hostNode) {
    console.log('You are running under node.');
}
else {
    console.log('You are not running under node.');
}

Defining the value of a Conditional Identifier at Compile Time

In order to provide the compile time values, an augmentation of the tsconfig.json is proposed. A new attribute will be proposed that will be named in line with the keyword of either conditionValues or hasValues. Different tsconfig.json can be used for the different builds desired. Not considered in this proposal is consideration of how these values might be passed to tsc directly.

Here is an example of tsconfig.json:

{
    "version": "1.6.0",
    "compilerOptions": {
        "target": "es5",
        "module": "umd",
        "declaration": false,
        "noImplicitAny": true,
        "removeComments": true,
        "noLib": false,
        "sourceMap": true,
        "outDir": "./"
    },
    "hasValues": {
        "hostNode": true
    }
}

Compiled Code

So given the tsconfig.json above and the following TypeScript:

has hostNode: boolean = Boolean(typeof process == "object" && process.versions && process.versions.node && process.versions.v8);

if (hostNode) {
    console.log('You are running under node.');
}
else {
    console.log('You are not running under node.');
}

You would expect the following to be emitted:

console.log('You are running under node.');

As the compiler would replace the symbol of hostNode with the value provided in tsconfig.json and then substitute that value in the AST. It would then realise that the one of the branches was unreachable at compile time and then collapse the AST branch and only emit the reachable code.

[imWildCat]

@Danielku15 couldn't agree more! Currently we're running into the same issue but there's no clear path to resolve it.

The most confusing part is that: Optimizers like https://github.com/webpack-contrib/terser-webpack-plugin can provide functionalities like removing dead code. But when it comes to the combination of webpack && TypeScript && terser, it does not work at all.

I believe if we're using JS rather than TS, things can improve a bit. However, we need to move forward on this topic for better TS toolchain.

[SomaticIT]

Hello,

I would like to know if we can reopen the discussion about preprocessor directives: https://github.com/microsoft/TypeScript/issues/3538#issuecomment-197979041

Thanks

[pilot87]

I would like to see this functionality to compile project for different target web servers.

[kevmeister68]

My personal instinct after reading much of this thread is that there a desire to build the "Taj Mahal" of conditional compilation when I think for a lot of people, simple and basic conditional compilation would be a fantastic first step.

I dislike how the capabilities of C#/C++ are being conflated together. What C++ can do, relative to C#, makes them two somewhat different beasts.

C# provides a straightfoward means via #if/#endif directives to include, or not include, code.

For me, this might be debugging code in a debug build, or some internal validation code in a debug build, and stuff like that. For others it might be some conditional compilation around web server support.

Because the C# style of behaviour acts like a preprocessor, it works for all portions of a code file, and so can target import statements, class definitions, or whatever else you wish.

My general opinion is this feature is being over-thought and over-baked. If the C# example tells us anything, it's that even a basic capability (which is what C# offers relative to C++) still can be immensely useful relative to nothing at all.

[imWildCat]

Currently, I'm using Tree Shaking in webpack (https://webpack.js.org/guides/tree-shaking/) and the DefinePlugin to implement condition build for different targets: Android, iOS, web, test.

You could turn off sideEffects to make sure unused imports are not bundled: https://webpack.js.org/guides/tree-shaking/#mark-the-file-as-side-effect-free

[endel]

I just wanna express the need I'm having right now for conditional compilation based on the target "module". If I could have a slightly different output for ESM and CJS would be perfect.

#if tsconfig.compilerOptions.module === "ESNext"
const script = await import(filename);
#else
const script = require(filename);
#end 

Perhaps all we need is a way to provide custom variables at the compilation level (+ default ones such as tsconfig options) and filter out AST branches that do not meet special conditional expressions. Branches filtered out do not need to be checked/compiled. I understand this is quite complex though!

[kamilchm]

I'd like to give a shout out to the https://github.com/jarred-sumner/atbuild I use it for a few projects to generate TypeScript using TypeScript and cases like the above are easy to do.

[mwaddell]

Conditional compilation could also be used to provide conditional exporting so that items can be exported only when running in development mode for the purpose of unit testing, while not being exported in production code.

For example:

export function ComplexFunction(args: ComplexFunctionArguments) {
    ...
}

#if DEBUG
export
#endif 
function HelperFunction(args: HelperFunctionArguments) {
    ...
}

That way, unit test code (which will always be running in development mode) can directly call/mock/etc HelperFunction but attempting to import/use HelperFunction directly when running in production mode will be a compile-time error.

[Whobeu]

I came across a situation where I wanted to do a conditional compile and searching led to this this open issue.

Node.js 14.17.0 introduced a new "randomUUID" function. in the "crypto" package. As a built-in now, a module is no longer required and I prefer to use built-in's wherever possible. Prior to this new built-in, I was referencing the "v4" function of the "uuid" package on npm. The problem I am having is one module I have runs on an old Node.js 8.17.0 system as well as newer Node.js 14 and 16 systems. It would be nice to have the TypeScript code compile one way for the ES2018 target (Node.js 8) and another for ES2020 target (Node.js 14+) which I compile using to different tsconfig.json files. For now I just hand edit the one built module that could use the conditional option so that it is one of these statements:

const { v4: randomUUID } = require ("uuid");  // Node.js 8

const { randomUUID } = require("crypto");  // Node.js 14.17+

This is obviously a very minor case but it could be more complex issue for other people as I read through the comments.

[Bessonov]

@Whobeu not tested, but can something like:

let randomUUID
try {
  randomUUID = require("crypto").randomUUID
} catch {
  randomUUID = require("uuid").v4
}

work for you? Or if you use webpack, you can try fallback.

[customautosys]

Strongly support, especially needed for cross platform (e.g. Electron / Cordova)

[SomaticIT]

I agree with @customautosys. Nowadays, targetting multiple environments is a common task in large SPA/PWA.

As a reminder, I put again a link to my proposal: https://github.com/microsoft/TypeScript/issues/3538#issuecomment-197979041

[RobertWHurst]

Would it not be better for this to be implemented as a form of triple-slash directive? This would be much like we have with references and the amd module stuff. Something like:

doGeneralStuff()

/// <conditional platform="election" ...other props perhaps... >
electrionOnly()
/// </conditional>

/// <conditional platform="node" ...other props perhaps... >
nodeOnly()
/// </conditional>

doMoreGeneralStuff()

This syntax wouldn't break parsers like eslint's typescript parser and countless other projects, it's already familiar to those who have used triple-slashes in typescript and have used xml or a like. I also like it allows for the use of more than one condition specified as a prop on the conditional node. How this ties into the tsconfig and what parts are dynamic I'll leave that to the better of the conversation here.

Just an off the cuff thought I wanted to share and see what you all thought.

[RobertoMalatesta]

This proposal has been long-standing (years). Still it could be useful to have an embedded such mechanism. R

[customautosys]

Would it not be better for this to be implemented as a form of triple-slash directive? This would be much like we have with references and the amd module stuff. Something like:

doGeneralStuff()

/// <conditional platform="election" ...other props perhaps... >
electrionOnly()
/// </conditional>

/// <conditional platform="node" ...other props perhaps... >
nodeOnly()
/// </conditional>

doMoreGeneralStuff()

This syntax wouldn't break parsers like eslint's typescript parser and countless other projects, it's already familiar to those who have used triple-slashes in typescript and have used xml or a like. I also like it allows for the use of more than one condition specified as a prop on the conditional node. How this ties into the tsconfig and what parts are dynamic I'll leave that to the better of the conversation here.

Just an off the cuff thought I wanted to share and see what you all thought.

That's also OK. It doesn't really matter what the syntax is as long as there is a way to select / remove code at compile time based on a compile time constant / definition. Whether it's /// or #define / #if / #ifdef / #ifndef doesn't matter.

[RobertWHurst]

@customautosys It does matter a little. If existing tokenizers can consume the code without exploding that is a win. Things like linters don't need to be aware of conditional behavior to do their job but they can't do it at all if they can't parse the code. There are many other kinds of tools that would run into similar problems I'm sure.

[rope-hmg]

I'm really not a fan of the /// thing. Magic comments are always a bad thing.

[Pauan]

Whenever TypeScript adds a new syntax feature, existing parsers need to be updated, that's just a fact of life.

TypeScript adds new syntax fairly often, the most recent example is template string types in 4.1

[RobertWHurst]

I thought about it a bit more, and I've come around on the syntax. The triple slash stuff is currently used for js files where typescript syntax can not be used.

[customautosys]

I just re-read it and it seems the proposal does not support conditional compilation for imports. This makes it a lot less useful then. We should have a way to allow for conditional imports.

[customautosys]

There are already people doing something similar, but it's non-standardised and people are doing different things for webpack, vite etc.

https://www.npmjs.com/package/ifdef-loader https://www.npmjs.com/package/vite-preprocess

We need a standardised way to go about it so the code will not be brittle.

Magic comments seem to be the way to go. Coming from a C++ environment, I think the #if / #else syntax is something a lot of people are familiar with.

[mschwartz]

This seems like such a trivial thing to implement. The #ifdef/#if/#else/#elsif/#endif kind of syntax (or use @ instead of #) is fine.

I really want to put #ifdef MIKE_REMOVED_THIS ... #endif around a big block of code to comment it out, and / / doesn't work because there might be /* style comments within the block. The MIKE_REMOVED_THIS is a way to blame MIKE for the elimination of the code - for other contributors to see.

At the top of the file, I would put:

#define MIKE_REMOVED_THIS
#undef MIKE_REMOVED_THIS

and comment out the undef to remove the code throughout the file where the #ifdef is used.

If you have to provide #define and #undef, then so be it. It would be fine. You don't have to allow #define values to substitute within any of the TS code, just use it for #if conditionals.

It really should be a feature of the compiler/language, not some hack using a rollup/webpack layer that may not be used at all (like a server-side program).

[coolCucumber-cat]

What about something like Rust macros? It just runs a compile time and returns some code, which then replaces the macro. Which would be kinda overkill and too complex just for this issue, but macros would be nice anyway and it might actually be worth it.

[HolgerJeromin]

I think I can write exactly the same as I did for the macro idea. The current design goals of Typescript are:

Goals: "Align with current and future ECMAScript proposals."

As far as I can imagine: It is not even possible to create an ECMAScript proposal for that because there is no compilation in ES.

Non-Goals: "Exactly mimic the design of existing languages. Instead, use the behavior of JavaScript and the intentions of program authors as a guide for what makes the most sense in the language."

@RyanCavanaugh

[coolCucumber-cat]

@HolgerJeromin Same can be said for this entire issue. Conditional compilation also doesn't align with ES because compilation isn't in ES. The only reason I said this is because clearly people don't think that's an issue here.

[customautosys]

@HolgerJeromin Same can be said for this entire issue. Conditional compilation also doesn't align with ES because compilation isn't in ES. The only reason I said this is because clearly people don't think that's an issue here.

That's taking the argument to its logical extreme. Strict typing isn't in ES either, do we then say types don't align with ES even though they're the raison d'être of Typescript? ES is not compiled but I think TS was designed to be compiled / transpiled since its inception. And that means that effectively zero runtime cost compile-time features like conditional compilation can be implemented in Typescript. The fact is that this has already been implemented in several plugins for various tool stacks in npm (e.g. https://www.npmjs.com/package/vite-plugin-conditional-compiler); this feature would just standardise the reality so that it will not be a mess.

[HolgerJeromin]

Same can be said for this entire issue. That's taking the argument to its logical extreme.

No. That was exactly my point (as it was for the macros suggestion). Conditional compilation is out of (documented) scope/goal of typescript.

Strict typing isn't in ES either, do we then say types don't align with ES even though they're the raison d'être of Typescript?

There are 11 typescript goals. Not only my quoted 2.

[customautosys]

Same can be said for this entire issue. That's taking the argument to its logical extreme.

No. That was exactly my point (as it was for the macros suggestion). Conditional compilation is out of (documented) scope/goal of typescript.

Strict typing isn't in ES either, do we then say types don't align with ES even though they're the raison d'être of Typescript?

There are 11 typescript goals. Not only my quoted 2.

Wouldn't it be compliant with these 2: Provide a structuring mechanism for larger pieces of code. Impose no runtime overhead on emitted programs.

I don't think the aim of the proposal is to mimic other languages exactly? I think the main aim is to structure the inclusion of specific pieces of code at compile time which is really useful for large cross platform codebases e.g. when you need to import 1 platform specific module only for a certain platform. Sometimes dynamic imports are not suitable.

[dead-claudia]

There's another caveat, one that probably would be a death sentence to this proposal: these stated non-goals.

4. Provide an end-to-end build pipeline. Instead, make the system extensible so that external tools can use the compiler for more complex build workflows.
5. Add or rely on run-time type information in programs, or emit different code based on the results of the type system. Instead, encourage programming patterns that do not require run-time metadata.
6. Provide additional runtime functionality or libraries. Instead, use TypeScript to describe existing libraries.

[rope-hmg]

I don't see how those would spell a death sentence for this proposal.

You could argue that this is out of scope and therefore 4 would rule it out, but there are already a bunch of competing third party solutions for this problem. It would be nice if something like this could be handled at the language level. That would remove the need for weird solutions and would make it easier to change build tools. If, for example, a codebase makes heavy use of something like webpack define plugin then it's difficult to switch to a different bundler unless there is a compatible solution available there.

5 and 6 don't seem relevant. Unless I'm missing something. The core of this proposal is conditional emit. i.e. I want to be able to set some flag internally or externally (via compiler flags, env vars, or some other mechanism) that controls which bits of code are emitted. It doesn't add any reliance on runtime type information nor add any extra runtime functionality.

[btakita]

There are already people doing something similar, but it's non-standardised and people are doing different things for webpack, vite etc.

https://www.npmjs.com/package/ifdef-loader https://www.npmjs.com/package/vite-preprocess

We need a standardised way to go about it so the code will not be brittle.

Magic comments seem to be the way to go. Coming from a C++ environment, I think the #if / #else syntax is something a lot of people are familiar with.

Magic comments would also work with js files without having to go through years of ES standard deliberation & runtime implementation.

I'm facing the question of how to use conditional compilation to reduce browser payload size. There are several esbuild plugins. It would be great to know what the blessed syntax is so I can target it. I vote for magic comments because they can be done today. Magic comments can provide a prototype for additions to js & ts.

I like how the preprocess library approaches this because it can be used for various file types, including css, shell, php, etc. Also, since the code to be added by the preprocessor is in comments, there will be less build issues for when not running the preprocessor. Perhaps there is potential standard approach for preprocessing any programming language which supports comments?

[Sunjammer]

What about something like Rust macros? It just runs a compile time and returns some code, which then replaces the macro. Which would be kinda overkill and too complex just for this issue, but macros would be nice anyway and it might actually be worth it.

I swear I think about this stuff every week. I've desperately wanted something like Rust or Haxe macros in TS since day 1. I do understand some of the arguments against though, it pains me to say. The biggest one for me is imagining the intersection of JS and TS libraries which is a hellish forest on average and can't possibly be improved by a bunch of library/project specific flags and compile time function evaluations. I imagine build times suffering for reasons that could be difficult to intuit, or hunting desperately for The Flag That Is Breaking The Build And How.

The best argument for is that macros and conditional compilation is phenomenal and even hard to imagine working without if you own all or most of the code. Lots and lots of valid arguments against, in our case, and it sucks.

If TS were to implement conditional compilation or macros I'd want it as a first class language feature with no magic comment ambiguity and code completion support, it just seems silly to compromise if you are going for it because you are worried about overly complicating the AST for instance (this is pretty rote text processing and shouldn't touch the AST other than choose what gets emitted into it). I'd look at the Haxe implementation. It is simple enough yet covers nearly every practical base and can be extended through exposing of compiler macros if the team were ever to desire such madness.

[albertjin]

I just re-read it and it seems the proposal does not support conditional compilation for imports. This makes it a lot less useful then. We should have a way to allow for conditional imports.

I found a workaround for conditional import. It is something like eval('import("xxx")'). My context is to switch between development mode and production mode. In development mode, modules should be loaded with HMR (hot module replacement) enabled. In production mode, all modules are bundled in a single javascript file. If eval is not used, the bundler does not allow to create a umd output and must embed all modules in the top-level loader even the import does not actually have a chance to execute.

The following code snip is as used in a project based on vue+vite.

async function importModule(name: keyof AdminViews) {
  if (import.meta.env?.DEV_MODE) {
    return eval('import("./views/"+name+".vue")')
  }
  // in production mode, dynamically load it from a bundled javascript file
  // ...
}

eval is not of perfection and the bundler complains,

Use of eval is strongly discouraged, as it poses security risks and may cause issues with minification

For a workaround, it's better than none.

[RyanCavanaugh]

A few different aspects on this one

Conditional compilation in value-space (statements and expressions) is pretty clearly out-of-scope in the modern understanding of TS; this is basically the same as "macros" which got closed a while back.

Something like conditional code exists in NodeJS's "conditional exports" https://nodejs.org/api/packages.html#conditional-exports which let you expose different code depending on external factors.

Something akin to conditional compilation in type space -- e.g. "this interface has this member if some condition is met" or "this variable exists if (some other condition is met)" - I would still consider to be tenable. There are multiple open problems that might be well-solved by a mechanism like this, but it's an extremely blunt tool that would certainly have a lot of side effects, like creating difficulties in validating whether a .d.ts file is even valid, avoiding circularities, and keeping features like "rename" functional in code that is conditioned out. I would consider this to be a "we have tried literally everything else" sort of solution to problems like webworker vs dom environment code living in the same compilation unit.

So overall this is either out of scope / some other tool's job, or better phrased as an extremely minimal proposal for something in type space.