deferred loading: allow references to deferred types

[sigmundch]

This issue consolidates a few separate conversations we had offline.

Problem

We'd like to address a gap in the spec related to type-inference and deferred loading. Ever since Dart 2.0 we are in an inconsistent state in which we allow type inference to introduce deferred type references where we currently disallow users to write them by hand. For example:

List<prefix.Foo> x = prefix.callThatReturnsListOfFoo();
prefix.Foo Function() f = () => prefix.fooGetter;

is not allowed, but:

var x = prefix.callThatReturnsListOfFoo(); 
var f = () => prefix.fooGetter;

is allowed, even when the CFE infers x to have type List<prefix.Foo> and f as prefix.Foo Function().

Possible solutions

Last week we started conversations to address this inconsistency. We mainly considered option (1), but for completeness let me mention two other options:

(1) Allow both: remove the original restriction so both patterns would be allowed going forward. (2) Disallow both: issue an error if type inference derives a deferred type. (3) Infer dynamic: change type inference to infer a dynamic type when the type would otherwise be deferred.

The main challenge with (1) is code size. Historically we disallowed references to deferred types because Dart-1 had no type inference and this provided a syntactic approach to implement deferred loading. Now it is non trivial to support it because our subtyping rules do not model the notion of a deferred type and because implementations coupled types and classes together. The former meant that a reference to a type could not be guaranteed to be guarded by a loadLibrary call first, so it forces implementations to front-load the type. The latter meant that not only we had to front-load the type, but the code of the class as well.

The challenge with (2) and (3) is usability. Many of our users have asked that they would prefer to write well typed deferred code. In addition, (2) is tricky because because there is not an easy workaround for the user - they may need to refactor their code to remove an error.

Context from implementations

dart2js: Starting with type-inference in Dart 2, this started causing trouble in dart2js. We even added an unsound workaround to prevent code-size regressions due to this issue. In Dart 1.0, the example above with f would have been treated as a dynamic type and would have allowed us to defer Foo, but in Dart-2 a sound split would have to load the type Foo in order to reify the closure type properly. Otherwise, type checks like f is Bar Function(), where Bar is a supertype of Foo, would yield a different answer.

We are finally at a point where we can split deferring of classes and types, so we can refer to the type even before the code itself is available and as a result we will be able to handle these scenarios soundly going forward.

More details about dart2js can be found in https://github.com/dart-lang/sdk/issues/35311

ddc: currently DDC doesn't defer any code. The compiler doesn't split types from classes, so we'll likely wont be able to defer classes that are used as types.

VM: @rmacnak-google mentioned that the VM does have the coupling of classes and types as well. However the current deferred loading implementation only defers machine code, so it is not at the moment affected. They do intend to defer classes in the future (method dictionaries are big), so this change could impose additional work on the team.

/cc @leafpetersen @munificent @joshualitt @rmacnak-google

[sigmundch]

Thinking more about (1): for a long time we thought the only possible scenario to address was to front-load classes or separate types from classes. I wonder if there is another alternative here if we were to model deferred types as part of the type system.

For example, what if a deferred type is allowed to appear in a type, but only if the type is not used in a subtype check. For example, f is Bar Function() yields an error before loading preifx, but gives true/false after the prefix has been loaded?

[eernstg]

Closed issue #1209 which turned out to be a duplicate of this one.

[abirajabi]

Is there any updates on this? I'm still figuring out what's the best workaround for now. Is it to separate the types from the class or is it okay for me to just use var and let it dynamically inferred?

[sigmundch]

There have been no changes at the language level, unfortunately. That said, I think it's OK for you to use var and allow type inference to take care of inferring the type for you. The dart2js compiler splits the types from the classes, so I expect it will still defer the code of the class (it's not a bad idea to confirm this by inspecting the output, though).