Make compile-time environment be required for compilation. (No delayed `.fromEnvironment`)

[lrhn]

Today our compilation tools allow, e.g, int.fromEnvironment to be delayed until a later step. Any step up to constant evaluation does not need to know, but steps after that do, and we have things that front-ends want to do which depend on constant evaluation. That, in turn, theoretically allows modular compilation to reuse the same artifact with different environments provided.

It also complicates compilation. Anywhere the language specification assumes constants to be evaluated before something else can happen, compilers must postpone that other thing until a compilation environment is provided.

With the patterns feature, constant evaluation has to happen before some type inference — so probably during the type inference phase, like almost everything else, since we can't evaluate code without knowing its own static type.

In patterns, constant patterns may contain constant variables, which may be initialized by, e.g., bool.fromEnvironment. At that point, it's not possible to say whether a map pattern (or map literal for that matter) has repeated keys. It's not possible to say whether a switch is exhaustive until we know the actual constant values used by the patterns. Because exhaustiveness affects reachability, and reachability affects type inference, we can't do correct type inference before we have the compilation environment.

I propose that we require the compilation environment to always be provided at the beginning of compilation. We can do modular compilation, but as we have decided before, we then have to ensure that the compilation environments used by all the modules are consistent (agree on any value that was actually read). Maybe by just embedding any used environment key/values into the kernel/dill file, and checking them when linking.

You cannot reuse most compilation artifacts across different compilation environments, but the way the language is currently specified, that's pretty much impossible anyway, all we do is to delay actual compilation.

The analyzer may run in a setting with no compilation environment (aka, an empty environment), but that's something code should be ready for anyway, by providing a useful defaultValue to the fromEnvironment constructors. In any case, the analyzer will be no worse off than today.

@johnniwinther

[johnniwinther]

@sigmundch @joshualitt

[sigmundch]

I'm very excited and supportive of this. Main caveat is how we mitigate and help migrate away from current uses of fromEnvironment.

Many existing uses would still work if we evaluation to happen eagerly at compile-time (when the CFE is run). That said, this requires:

• investments in our build system integration, like using parameterization of bazel aspects. This would allow fine-grain use of these constants for tree-shaking purposes (current uses include tree-shaking of field names in generated protos, custom selection and tree-shaking of widget themes in code shared across two internal apps)
• changing how the SDK is built in our build systems, so they can be treated as a parameterize module like any other if these constants are used within the SDK (which they are in the case of Flutter Web)
• study alternatives that could solve tree-shaking problems outside constant expressions. Examples include alternative link-time non-constant flags, or proposals like @munificent's @conditional.

Relevant pointers:

• https://github.com/dart-lang/sdk/issues/50535
• https://github.com/dart-lang/language/issues/304#issuecomment-494164232 (old specification for how we'd ensure environments were consistent modularly if we don't take a consistent environment route, like we would with blaze aspect parameterization)

[munificent]

The analyzer may run in a setting with no compilation environment (aka, an empty environment), but that's something code should be ready for anyway, by providing a useful defaultValue to the fromEnvironment constructors. In any case, the analyzer will be no worse off than today.

So is the idea that if I do:

String test(bool b) =>
    switch (false) {
      case const bool.fromEnvironment('windows', defaultValue: false): 'yes',
      case true: 'no'
    };

Then in analyzer, it will show up as exhaustive and free of errors. Then when I go to compile it on Windows, I get a compile error. Is that the idea?

I'm fine with that if it gets the language out of the business of dealing with weird "unknown constant values", but it also doesn't seem like a totally great user experience.

[mraleph]

/cc @mkustermann @sstrickl

[jakemac53]

• investments in our build system integration, like using parameterization of bazel aspects

Would this remove our ability to share code across apps? Or are they smart enough to share outputs if the parameters are the same? This would also be a big ask for build_runner.

Fwiw, I would probably prefer an approach that makes any environment constants global (this is what we do today in build_runner), and it could easily be supported with --define flags in bazel. The main issue with that is Tap (we can't generally change any global defines there).

[sigmundch]

investments in our build system integration, like using parameterization of bazel aspects Would this remove our ability to share code across apps? Or are they smart enough to share outputs if the parameters are the same? This would also be a big ask for build_runner.

It's smart enough. The way it works is that parameterization is not allowed to use arbitrary values, only a pregiven set of possible values per parameter. It's mostly sugar to avoid copy/pasting a bunch of aspects to have one per permutation of parameter values. So in a way, we could use it to reduce duplication we have today (e.g. we could use the same aspect declaration for different backends by making the backend parameter into an aspect parameter).

The restriction of having a known set of values means we can only use this for well known use cases with well known values, and we'd be frugal in using this for very few cases. I could see us approving some that come from a framework level (like flutter web, maybe protoc), but not include anything that is app specific. For that, we would instead ask to move to a non-constant flag instead.

Fwiw, I would probably prefer an approach that makes any environment constants global (this is what we do today in build_runner), and it could easily be supported with --define flags in bazel. The main issue with that is Tap (we can't generally change any global defines there).

Right. We can't do it globally internally via a bazel flag. That said, it would only be specified in app targets that aggregate and provide context (and the aspect extracts the parameter values from that context). So the specification of values is guarantee to be consistent for all aspects used by an app by construction.

[askeksa-google]

Does this mean that all environment values used in the platform libraries have to be specified when the platform dill is built?

In dart2wasm we are contemplating using environment values as a mechanism to control some features of the platform libraries by options given when the program is compiled (after the platform dill is built). If this is not possible, we need to use some other mechanism for this.