dart vm can't upcasting generic type of function getter

[chunhtai]

I am not sure if this is expected or not.

The following code will crash the dart vm

void main() {
  A<Object?> a = B<Data>();
  a.done(Data());
}

typedef Fn<T> = void Function(T a);

abstract class A<T> {

  Fn<T> get done;
  // This is ok
  // void done(T a);
}

class B<T> extends A<T> {
  B();

  @override
  Fn<T> get done => (T a) => print(a);
  // This is ok
  // @override
  // void done(T a) =>  print(a);
}
class Data {

}

If you run this code, it crashes with

Unhandled exception:
type '(Data) => void' is not a subtype of type '(Object?) => void'
#0      main (package:flutter/src/widgets/test.dart:4:5)
#1      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#2      _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:184:12)

If I convert the function getter to a instance method, it will work.

[eernstg]

Check out this issue about "contravariant members".

The core issue here is that Fn<T> is contravariant in T (e.g., Fn<num> is a subtype of Fn<int>, not vice versa), and Fn<T> is the return type of the getter done in the declaration of A<T>.

This means that T occurs in a non-covariant position in the signature of a member of A<T>, which is exactly what it takes to get the behavior that you're reporting.

Dart uses dynamic type checks to enforce soundness of the heap (that is, roughly: no variable has a value that contradicts its declared type), and these "contravariant members" are particularly prone to cause failures in such a run-time type check.

(You can vote for https://github.com/dart-lang/linter/issues/4111 if you wish to help getting support for detecting when a member is contravariant in this sense, such that you can make sure you don't have any of them.)

Here is a minimal example showing such a run-time failure:

class A<X> {
  void Function(X) fun;
  A(this.fun);
}

void main() {
  A<num> a = A<int>((int i) { print(i.isEven); });
  a.fun; // Throws. We don't even have to call it!
}

However, if you insist that you want to have a contravariant member then you can still make it statically type safe. This means that we get a compile-time error at the location where the situation is created that causes the covariance problem to arise.

This approach relies on a feature which is still experimental, so you need to provide an option when running tools:

// Use option `--enable-experiment=variance`.

void main() {
  A<Object?> a = B<Data>(); // Compile-time error!
  a.done(Data());
}

typedef Fn<T> = void Function(T a);

abstract class A<inout T> {
  Fn<T> get done;
}

class B<inout T> extends A<T> {
  B();

  @override
  Fn<T> get done => (T a) => print(a);
}

class Data {}

The compile-time error in main ensures that we won't have a reference to an object of type B<Data> whose static type is A<Object?>. The reason for this is that B<Data> is simply not a subtype of A<Object?> any more (because of the modifier inout on the type variables).

You can change the declaration to A<Data> a = B<Data>(); which is accepted with no errors, and then you won't have the run-time error.

However, as you can see, you also have to give up on the ability to forget that the actual type argument is Data, which is exactly the point: In order to make the invocation of a.done(...) type safe, you must remember that this function needs an argument of type Data, and if you're allowed to think that any Object? will do then there is no way we can avoid performing the type check at run time and potentially have the run-time failure.

You mention that the following variant is more forgiving:

void main() {
  A<Object?> a = B<Data>();
  a.done(Data()); // No compile-time error, succeeds at run time.
  a.done(false); // No compile-time error, throws at run time.
}

typedef Fn<T> = void Function(T a);

abstract class A<T> {
  void done(T a);
}

class B<T> extends A<T> {
  B();

  @override
  void done(T a) =>  print(a);
}

class Data {}

The reason why this variant will run successfully (until we reach a.done(false), at least!) is that it does not obtain an object which is mistyped at any point in time (that is, we maintain soundness at all times).

In particular, we can check dynamically that the argument passed to a.done(Data()) has the required type (it has type Data, as required by the actual value of a), and then we can proceed without throwing.

At a.done(false) we perform the same dynamic type check, but it fails and the invocation throws.

In contrast, the expression a.done in the original version of this example evaluated to obtain a function object of type void Function(Data), but the static type of a.done was void Function(Object?). That's a soundness violation because void Function(Data) is not a subtype of void Function(Object?). When we have a soundness violation we will have a run-time type error, period.

So you never take the next step and try to call that function object. So it doesn't help that you might pass an argument like Data() that would have satisfied the function object, because we don't even try to call the function object in the situation where the function object itself is "bad".

Note that you can combine the two approaches if it is important for you to use separate function objects rather than instance methods:

void main() {
  A<Object?> a = B<Data>();
  a.done(Data());
}

typedef Fn<T> = void Function(T a);

abstract class A<T> {
  Fn<T> get _done;
  void done(T a) => _done(a);
}

class B<T> extends A<T> {
  B();

  @override
  Fn<T> get _done => (T a) => print(a);
}

class Data {}

The invocation of _done(a) will take place because the static type of _done inside the body of the class is void Function(T), and there's nothing unsafe about evaluating _done in this context. (In other words, no member is "contravariant" when used from inside the class itself). The dynamic type check will now take place at the invocation of the instance method done, and the invocation of the function object relies on the fact that the type has already been checked, and it just works. The price you'll have to pay in order to get "the best of both words" is that it is more costly at run time to call two functions than it is to call just one; YMMV.

I'll close this issue because it is all working as specified.

[eernstg]

The examples in this issue will all give rise to type checks at run time. Even the examples that do not give rise to a run-time error will perform a check, and it is very easy to come up with a modified version of the example where there are no compile-time errors, but we still get a type error at run time:

void main() {
  A<Object?> a = B<Data>();
  a.done(Data2()); // Throws.
}

typedef Fn<T> = void Function(T a);

abstract class A<T> {
  Fn<T> get _done;
  void done(T a) => _done(a);
}

class B<T> extends A<T> {
  B();

  @override
  Fn<T> get _done => (T a) => print(a);
}

class Data {}
class Data2 {}

It is actually possible to use a mechanism known as an "existential open" operation to establish all the needed information at compile time (which means that there will not be any type errors at run time, even if you try out one of the cases like the one that fails because it uses a Data2 rather than a Data object).

There will be some type casts (because we don't have an actual language mechanism to support the existential open operation during type checking), but they will never fail because they rely on relationships that are guaranteed to hold for the existential open operation.

It's more complex, so you may wish to do it because you really want to play safe, or you may wish to continue using code that has some dynamic type checks because it is simpler (and, perhaps, you don't see any run-time type errors in practice).

Here's the version that uses an existential open operation (which basically means that we're able to discover which type argument T the given A<T> actually has). It uses the package typer:

import 'package:typer/typer.dart';

void main() {
  A<Object?> a = B<Data>();

  // Every time we use `done` on a receiver of type `A` (or any
  // subtype of `A`) we must "open" it first.

  // This object knows the type argument of `a`, which enables an
  // approach that never gives rise to any dynamic type checks which
  // are able to fail.
  var typer = a.typerOfT; 

  // The `callWith` invocation provides access to the type argument
  // of the given instance of `A<...>` under the name `T`, no matter
  // which value it has.
  typer.callWith(<T>() {
    // At this point it is guaranteed that `a` is an `A<T>`. Note
    // that it is not an `A<S>` where `S` is some unknown subtype of
    // `T`, it is actually an "A<exactly T>".
    var openedA = a as A<T>; // Tell the type system that it is so.

    if (Typer<Data>() <= typer) {
      // At this point is is guaranteed that `Data` is a subtype
      // of `T`. Tell the type system that `Data() is T`.
      var data = Data() as T;

      // This is now completely statically type checked: We are
      // passing a `T` to the `done` of an `A<T>`.
      openedA.done(data);
    }
  });
}

typedef Fn<T> = void Function(T a);

abstract class A<T> {
  Fn<T> get done;
  Typer<T> get typerOfT => Typer(); // Enable "existential open" on this type.
}

class B<T> extends A<T> {
  B();

  @override
  Fn<T> get done => (T a) => print(a);
}

class Data {}

If we create a Data2 class to create the same mismatch as in the first example we will just do nothing (and developers can write else {...} if they wish to handle the failure to use a Data()).

[chunhtai]

I can explain my use case, and maybe there is a better way we can do this without fooling the compiler.

This is for Flutter's https://github.com/flutter/flutter/blob/125543505d2608afccbbd1486bd3380d7c388893/packages/flutter/lib/src/widgets/routes.dart#L1119

The class is quite complex so i will simplify to this

I have a ModalRoute class with a generic type T which also holds the a list of PopEntry with same type T.

typedef Fn<T> = void Function(T a);

class ModalRoute<T> {
   List<PopEntry<T>> popEntries = <PopEntry<T>>[];

   void add(PopEntry<T> entry) => popEntries.add(entry);

  void pop<T>(T result) {
     for(entry in popEntries) {
        entry?.onPopInvoked(result);
     }
  }
}

class PopEntry<T> {
  FN<T> onPopInvoked; 
}

This is all good if the type involve are all the same. e.g.

ModalRoute<int> route = ModalRoute<int>();
route.add(PopEntry<int>());
route.pop(1);

but what I want to support is to loosen the type on PopEntry

ModalRoute<int> route = ModalRoute<int>();
route.add(PopEntry<Object?>()); // This will throw compile error
route.pop(1);

The only way to get pass this is to hold the list as Object?

class ModalRoute<T> {
   List<PopEntry<Object?>> popEntries = <PopEntry<Object?>>[];

  void add(PopEntry<Object?> entry) => popEntries.add(entry);
}

but then it will crashes the

route.pop(1);

in runtime.

Thus, I use the workaround to change PopEntry

class PopEntry<T> {
  void onPopInvoked(bool didPop, T? result) {...}
}

[eernstg]

That's an interesting case, @chuntai!

This is a long response, but the situation is rather complex, even with that small excerpt of the code that you have shown. On the other hand, I think it's worth diving into the details because they bring out relationships that are useful to think about and deal with in a careful manner.

First, note that the very core of the example introduces potential run-time type errors with a program that has no compile-time errors or warnings:

typedef Fn<T> = void Function(T a);

class PopEntry<T> {
  Fn<T>? onPopInvoked;
}

void main() {
  PopEntry<num> entry = PopEntry<int>()..onPopInvoked = (_) {};
  entry.onPopInvoked; // Throws.
}

The point is that there is a "contravariant member" in the class PopEntry, and that's a veritable run-time failure factory. So we don't even have to call onPopInvoked in order to get that run-time failure, it is enough to look it up and ignore it.

If we want to make contravariant typing relationships safe then we need to take several steps. First, PopEntry needs to be invariant in T. This implies that PopEntry<int> is not a subtype of PopEntry<num>, they are just unrelated types.

(We must use invariance rather than contravariance because onPopInvoked is mutable, and the associated setter has an occurrence of T which is in a covariant position. If it were final then we would only have the getter, and T would only occur in a contravariant position, and then we could use contravariance.)

So here is an invariant version of PopEntry, using the language feature 'variance' (which is an experimental feature, enabled using --enable-experiment=variance):

typedef Fn<T> = void Function(T a);

class PopEntry<inout T> {
  Fn<T>? onPopInvoked;
  PopEntry([this.onPopInvoked]);
}

In order to avoid relying on the experiment, we can emulate invariance as follows:

typedef Inv<X> = X Function(X);
typedef PopEntry<T> = _PopEntry<T, Inv<T>>;

class _PopEntry<T, Invariance extends Inv<T>> {
  Fn<T>? onPopInvoked;
  _PopEntry([this.onPopInvoked]);
}

We need to have a supertype, PopEntryBase, in order to be able to abstract over different type arguments (because types like PopEntry<int> and PopEntry<num> are now unrelated types whose least upper bound would be Object, except that it is now PopEntryBase). We also need to get access to a Typer of the actual type argument, such that we can do the "existential open" when needed:

sealed class PopEntryBase {
  Typer<Object?> get typerOfT;
}

typedef Inv<X> = X Function(X);
typedef PopEntry<T> = _PopEntry<T, Inv<T>>;

class _PopEntry<T, Invariance extends Inv<T>> implements PopEntryBase {
  Fn<T>? onPopInvoked;

  _PopEntry([this.onPopInvoked]);

  @override
  Typer<T> get typerOfT => Typer();
}

We will then handle the pop entries as a List<PopEntryBase> which is capable of holding an element of type PopEntry<S> for any S. I renamed popEntries to _popEntries because we want to impose a typing discipline on the elements in this list, so we don't want to reveal it to the world. (You could return a copy of the list if some client must definitely have it.) I made _popEntries final because there is no reason (in this reduced example) that it needs to be mutable.

Next, add will have to perform a type check in order to ensure that _popEntries only contains elements of type PopEntry<U> such that U is a supertype of T (that is, we want to have a guarantee that an actual argument of type T can be passed to the function).

We cannot express this constraint in the signature of the function (we could do it using an upper bound on a type parameter, but Dart doesn't have such lower bounds, at least not yet ;-). So we will perform this check at run time.

Finally, there is an extension method which will perform a special case of the same check at compile time, typedAdd. You may use this method in order to ensure that all the type checks are performed at compile time, but it will only work when you are adding a PopEntry<T> to a receiver whose static type is ModalRoute<T>, the two type arguments must be the same. (Note that the receiver can actually have type ModalRoute<V> where V is some subtype of T, that's fine, but the statically known types must match up).

The resulting program is here:

import 'package:typer/typer.dart';

typedef Fn<T> = void Function(T a);

class ModalRoute<T> {
  final List<PopEntryBase> _popEntries = [];

  /// Add an [entry].
  ///
  /// This method accepts an [entry] which can be a [PopEntry<S>]
  /// for any `S`. It is then checked at run time that it is
  /// for an `S` which is a supertype of [T], which means that
  /// it is guaranteed that an actual argument of type [T] can be
  /// passed to the function of that [entry].
  ///
  /// We cannot express this constraint in the parameter type without
  /// lower bounds (`void add<S super T>(PopEntry<S> entry)`), and
  /// Dart does not have those (yet).
  void add(PopEntryBase entry) {
    // Ensure that `entry` is a `PopEntry<S>` such that
    // `T <: S`. This subtype relation has "the wrong direction",
    // that is, it can not be tested using any expression of the
    // form `e is U`.
    var typer = entry.typerOfT; // `typer` reifies said type `S`.
    if (Typer<T>() <= typer) {
      typer.callWith(<S>() {
        // This typing is guaranteed, tell the type system about it.
        entry as PopEntry<S>;
        _popEntries.add(entry);
      });
    } else {
      // Error handling: Can not add `entry` to `_popEntries`,
      // we wouldn't be able to call its function with an
      // argument of type `T`, as done in `pop` below.
    }
  }

  void pop(T result) {
    for (var entry in _popEntries) {
      var typer = entry.typerOfT;
      typer.callWith(<ActualT>() {
        // These typings are guaranteed, based on `typer`
        // and based on the checks in `add`.
        entry as PopEntry<ActualT>;
        result as ActualT;
        // This invocation is statically type safe.
        entry.onPopInvoked?.call(result);
      });
    }
  }
}

extension ModalRouteExtension<T> on ModalRoute<T> {
  /// Add an [entry] in a statically checked manner.
  ///
  /// This method accepts an [entry] which is a [PopEntry<T>].
  /// Note that [PopEntry] is invariant in [T], and this implies
  /// that an actual argument of type [T] can safely be passed
  /// to the function of that [entry].
  ///
  /// Note that `this` can have the run-time type [ModalRoute<S>]
  /// for any `S` which is a subtype of [T], but [entry] will have
  /// run time type [PopEntry<T>] (not just [PopEntry<S>] where `S`
  /// is some unknown subtype of [T]).
  void typedAdd(PopEntry<T> entry) => add(entry);
}

sealed class PopEntryBase {
  Typer<Object?> get typerOfT;
}

typedef Inv<X> = X Function(X);
typedef PopEntry<T> = _PopEntry<T, Inv<T>>;

class _PopEntry<T, Invariance extends Inv<T>> implements PopEntryBase {
  Fn<T>? onPopInvoked;

  _PopEntry([this.onPopInvoked]);

  @override
  Typer<T> get typerOfT => Typer();
}

void main() {
  var route = ModalRoute<int>();
  route.add(PopEntry<Object?>((o) { print('`Object?` entry: $o'); }));
  route.typedAdd(PopEntry<int>((i) { print('`int` entry: $i'); }));
  route.pop(1);
  route.pop(2);
}

Of course, it is still possible to encounter a run-time type error simply because ModalRoute<T> is dynamically-checked covariant in T (just like List, by the way). For example:

void main() {
  ...
  (route as ModalRoute<Object>).pop('Hello!'); // Throws.
}

This would fail at run time (just like (<int>[] as List<num>).add(1.5)). If you wish to avoid this kind of run-time type error then you could use the same techniques on ModalRoute that we've used on PopEntry, except that the situation is simpler because this is all about covariance. So it's probably reasonable to assume that the dynamically checked covariance (a la List) can be handled using common Dart programming habits, and then we can reserve the Typer based techniques and emulated invariance for the cases that are more tricky.

[chunhtai]

@eernstg wow thanks for the detailed answer, really appreciate. There is a lot of new things I learned about dart for your answer. I have some question on the terminology. While I can guess some of them, but I am not sure if I got them correctly.

What are contravariant and invariant in the context of dart? These two words are new to me.

I assume covariant means transform method input type to subclass type from the uses of key word covariant is that correct?

[eernstg]

Thanks for the kind words, @chuntai!

Covariant is just like 'increasing' for functions ("if a ≤ b then f(a) ≤ f(b)"), with respect to types (that is, "if T1 is a subtype of T2 then F<T1> is a subtype of F<T2>" — for example, List is covariant in its type argument, as are all other Dart classes ... at least for now).

Contravariant is just like 'decreasing' for functions ("if a ≤ b then f(a) ≥ f(b)"), with respect to types (that is, "if T1 is a subtype of T2 then F<T1> is a supertype of F<T2>". Dart doesn't support this for classes, yet, but #524 would enable it, using in on the type variable which should be contravariant. On the other hand, parameter types of function types are contravariant: void Function(num) is a subtype of void Function(int). (That's also the reason why it's much better to use a function type directly if you need to model contravariance, rather than making the function type a member of a class.)

Finally, invariant means that different inputs yield incomparable outputs. This can't immediately be modeled by a simple mathematical function (assuming that it's mapping real numbers to real numbers, or something like that). But we can state what it is all about as with the other variances: "Even if we know that T1 is a subtype of T2 then we have no known relationships between F<T1> and F<T2>".

The keyword covariant is actually somewhat different. It is used with formal parameters of instance methods. It states that "this parameter has a type which is covariant with the type of the enclosing class". For example:

class A {
  void foo(covariant Object o) {}
}

class B extends A {
  void foo(String s) {}
}

So this parameter type is covariant because B is a subtype of A, and String is a subtype of Object (they "vary together"), and this is significant (somewhat dangerous, in fact) because it may come as a surprise:

void main() {
  A a = B();
  a.foo(true); // OK, according to the declaration of `foo` in `A`.
  // But it actually throws, because it's a `B`, and it wants a `String`.
}

Because of this danger, the types can't be specified as shown if we do not have the modifier covariant on that parameter (in A or in B).

In any case, the work covariant is still associated with a typing relationship which is similar to an increasing function, which means that it isn't entirely crazy to use that word in one way for type parameters, and in a different way for formal (value) parameters.