Interface default methods

[lrhn]

See more detailed proposal here: https://github.com/dart-lang/language/blob/master/working/0884/interface_default_methods_proposal.md

Adding new members to an interface is a breaking change in Dart, for several reasons. One reason is that a subclass might already implement the interface, and it won't have an implementation of the new member, and that's a compile-time error. (Other reasons include subclasses already having a member with that name and an incompatible interface).

An interface default method is a concrete method added to an interface which is inherited (or rather: mixed in) by all non-abstract classes implementing the interface. It's implementation inheritance along implements relations.

As such, it needs to be controlled in order to avoid the issues that caused the user to use implements over extends.

Not all methods will be inherited, so an interface default method needs to be marked as such, perhaps with the default keyword:

  default double get distanceFromOrigo => sqrt(x * x + y * y);

The method is not just inherited along the normal superclass path. Instead it is mixed in on any class which implements the interface where the default method was introduced, and which does not already declare or inherit a non-synthetic concrete member with the same name.

This uses the existing member mix-in functionality which is used to mix-in mixin members into mixin application classes, only just for the individual default methods. (We may need to define what mixing in a single member means, but it's completely consistent with what a mixin application does to all the mixin members.)

A synthetic member is either a noSuchMethod forwarder or another default method. If a class has a non-trivial noSuchMethod and an applicable default method, the default method is used. (We might need to reconsider that if it breaks mocks). A default method must be usable as a mixin member declaration with no super-class requirement, so the default method cannot do super-invocations on anything except Object members.

It is a compile-time error if this mixed-in implementation does not satisfy the interface of the class. (Can happen if also implementing another interface with a more specific signature and no default method).

(Effectively: If a class has an interface member with no non-synthetic implementation, if precisely one interface default method applies, mix that in. Otherwise, if the class has a non-trivial noSuchMethod, add a nSM-forwarder. Otherwise do nothing, which might cause an error.)

If multiple interfaces can introduce default members with the same name, none of them are mixed in, unless precisely one of the interfaces is a subtype of all the remaining ones. In that case, that interface's default member wins. (Do we need to lower the precedence of platform library default members?)

Compared to extension members, interface default methods are normal, virtual, interface members. All you get is an implicit mixin of number of methods if you need it. You can always declare your own implementation of the member, and use normal virtual dispatch to get the best available implementation.

[eernstg]

This idea is certainly promising!

We could consider a different point of view on this mechanism: If a class C is allowed to declare that one or more mixins are considered to be sources of default implementations of methods in the interface of C, then the semantics could be that those mixins are mixed in whenever a class D that implements C has such methods in its interface, but no implementation:

mixin M<X> {
  void foo(int i) {}
}

abstract class C default M<int> {
  void foo(int i);
}

class D implements C {} // Implies `with M<int>`

We could also say that we only mix in the individual methods that are missing, but let's explore the idea where we mix in the whole mixin:

It is possible to express the single-method default mechanism using a mixin that declares just one method (and we could make the in-class default method be syntactic sugar for that). So there is no loss in expressive power in doing this.

If a mixin declares several methods (corresponding to having a set of cooperating default method implementations) then we could make it an error if a proper subset of them are unimplemented: You get all or none of these. We could also allow that, and we could let this mean that all the methods are mixed in (thus overriding some existing implementations), or it could mean that only the missing implementations are provided, or we could allow developers to control this choice explicitly.

In any case, it seems likely to me that there would be situations where it is useful to be able to say that "here is an implementation of a set of methods, take it or leave it", in addition to the situation where the default implementations are simply independent, and you can have any subset.

[leafpetersen]

@rakudrama If this supported library private default methods, would this solve your longstanding request for a way to put a hidden method on all implementations of Iterable?

[rakudrama]

@leafpetersen I'm not sure it does.

The essence of what I want to do is implement a double-dispatch between two classes where the code lives in one file. Things are complicated by type parameters. Consider making JSArray.addAll faster when adding another JSArray:

class JSArray<E extends Object?> ... {

  void addAll(Iterable<E> items) {
    // The E above is like `E' extends E`
    if (this is JSExtendableArray) {
      items._addAllToJSArray<E>(this);
    } else {
      // throw not growable.
    }
  }

  void _addAllToJSArray<T>(JSArray<T super E> receiver) {
    JS('', 'Array.prototype.push.apply(#, #)', receiver, this);
  }
}

dynamic extension on Iterable<Q> {
  void _addAllToJSArray<T>(JSArray<T super Q> receiver) {
    for (var e in this) receiver.add(e);
    // ^ this for-in is polymorphic in the iterable, but no worse than the current
    // addAll code.

    // 'add' should not need a parametric covariance check, so can be
    // lowered to 'push'.
  }
}
// The dynamic extension could be cloned into all 100 Iterable classes,
// but dart2js would not want that code bloat.
// Cloning could be manually by matching interesting subtypes:
dynamic extension on List<Q> {
  void _addAllToJSArray<T>(JSArray<T super Q> receiver) {
    for (var e in this) receiver.add(e);
    // ^ we expect some kind of common ListIterator here.
  }
}

[leafpetersen]

It seems like you could do this with default methods as described below. You need to say something about how multiple interface default methods with the same name from different interfaces get resolved, which isn't specified above, but let's ignore that for now. I think what you need to do is something like the following:

class JSArray<E extends Object?> ... {

  void addAll(Iterable<E> items) {
    // The E above is like `E' extends E`
    if (this is JSExtendableArray) {
      items._addAllToJSArray<E>(this);
    } else {
      // throw not growable.
    }
  }

  void _addAllToJSArray<T>(JSArray<T super E> receiver) {
    JS('', 'Array.prototype.push.apply(#, #)', receiver, this);
  }
}

abstract class Iterable<Q> {
  // Other Iterable members declared

  default void _addAllToJSArray<T>(JSArray<T super Q> receiver) {
    for (var e in this) receiver.add(e);
    // ^ this for-in is polymorphic in the iterable, but no worse than the current
    // addAll code.

    // 'add' should not need a parametric covariance check, so can be
    // lowered to 'push'.
  }
}

// The dynamic extension could be cloned into all 100 Iterable classes,
// but dart2js would not want that code bloat.
// Cloning could be manually by matching interesting subtypes:
abstract class List<Q> {
  // Other List members declared
  default void _addAllToJSArray<T>(JSArray<T super Q> receiver) {
    for (var e in this) receiver.add(e);
    // ^ we expect some kind of common ListIterator here.
  }
}

Now every class that implements List or Iterable gets a hidden _addAllToJSArray method as written above. Again, you need to say something about how conflicts get resolved, but then you needed to say something about that in your dynamic extension method thingie as well, no?

[lrhn]

The usual way an interface default method is inserted is that any concrete class which implements the interface, but does not have a concrete implementation of the interface member, will get the default member implementation mixed in.

Dart does not distinguish between classes and interfaces (that much), so we'll probably say that only concrete classes gets the default implementation. That avoids IterableBase getting a default implementation for Iterabel and therefore not allowing ListBase to get the more specialized version for lists. We could also do what we do for nSM-forwarders, and say that a default implementation can be overridden by another default implementation in a subclass, if a more specialized one is available.

(We'll have to decide whether a class with a noSuchMethod will get a nSM-forwarder or the default member implementation, but probably the latter).

[leafpetersen]

@lrhn I'm not sure if your comment was in response to me, but the concrete question that I had above was the following. Given:

abstract class A {
  default String foo() => "A";
}

abstract class B {
  default String foo() => "B";
}

abstract class B2 extends B {
  default String foo() => "Extended B"
}

abstract class B3 implements B {
  default String foo() => "Implemented B"
}

class C implements A, B {}

class D implements B, A {}

class E implements B, B2 {}

class F implements B2, B {}

class G implements B, B3 {}

class H implements B3, B {}

Which concrete version of foo is selected for each of the C, ..., H classes. Does order matter? Does the extend relationship between B and B2 matter? Does the implements relationship between B and B3 matter?

It seems unpleasant if the accidental order of the "implements" clause matters, but maybe hard to avoid.

It seems useful to allow a specialization of an interface to specialize a default method for that interface, but for that to work you probably want the specialized method to take precedence? Seems potentially tricky to specify, but maybe not too bad.

[lrhn]

Not claiming that I have all the details figured out yet, but I do like a puzzle, so:

abstract class A {
  default String foo() => "A";
}

abstract class B {
  default String foo() => "B";
}

abstract class B2 extends B {
  default String foo() => "Extended B"
}

abstract class B3 implements B {
  default String foo() => "Implemented B"
}

// No reason to prefer A over B or vice versa, so no method introduced.
// Class stays incomplete and a compile-time error.
class C implements A, B {}   

// Ditto.
class D implements B, A {}

// Here we can say that B2 is more specific than B.
// B2 is a subtype of B, so this declaration is equivalent to just `implements B2`,
// and in that case we'll want to use the B2 default method.
class E implements B, B2 {}  // gets B2.foo

// Ditto.
class F implements B2, B {}  // gets B2.foo

// Same argument as E and F: B3 is more specific than B.
class G implements B, B3 {}  // gets B3.foo

// Ditto.
class H implements B3, B {}  // gets B3.foo

So, order of implements does not matter. Extends vs Implements does not matter. You don't actually have to look at the declaration of the class, just the total set of super-interfaces that it implements. For each member with no real concrete implementation, see which interfaces want to supply a default implementation. If any one of those is a subtype of all the rest, it wins (trivially if there is only one), otherwise no default method is mixed in, which might make the class invalid.

That also means that if there is a conflict, there is no syntax to override the conflict resolution manually. Even if you know that the Foo class default foo method is better than the Bar class default foo method, and the signatures are compatible, implementing both Foo and Bar will make you get neither:

class A implements Foo, Bar {}  // no foo!

You can introduce a superclass implementing Foo:

class _A implements Foo {}  // gets Foo.foo.
class A extends _A implements Bar {} // Does not get a new member, but inherits Foo.foo from _A.

[munificent]

What does this feature add over simply telling people "Stop using implements and use with everywhere instead."? My hunch is that we already have a solution to this problem, we just aren't in the habit of using it because of decades of Java/C# legacy. Interfaces bad. Mixins good. with is even shorter than implements. :)

[eernstg]

Much shorter! :-)

But one crucial difference is that there must be a with in the recipient class declaration D in order to obtain a set of member implementations, so we can't use mixins to implicitly add an implementation of a new method m to existing subtypes of a given class C—but if we add that new method m as an interface default method to C then D will implicitly get that implementation of m if it is concrete and doesn't have an implementation of m. With mixins we'd have to edit the declaration of D, and we'd have to manually check whether or not m should be added.

[lrhn]

I probably wouldn't use interface default methods in a new language. As you say, just making implements act like with would allow people to write their interfaces and non-interface classes accordingly, and all would likely end up working.

We didn't start there, and now have a lot of implements going around. Interface default methods is a way to add default method implementations to an existing interface based language in a way that doesn't break existing sub-classes based on implements (or "not too much", in Dart's case, "at all" if you had overloading).

The "mix in only if class doesn't already have an implementation" is distinguishes it from a normal mixin. We could have that as a feature in mixins too - some methods are only mixed in if they are needed, others are always mixed in. That might be useful.

If we just changed implements to mean with everywhere ... a lot of code would probably still run, but some would have unexpected mixed-in members overriding the members you actually want. Our classes are not currently written to be all-interface or all-implementation. We have classes that are both. If you want to make a class iterable, you can extend, implement or mix in Iterable. If you used implements, it's probably because you don't want the default implementation.

[munificent]

But one crucial difference is that there must be a with in the recipient class declaration D in order to obtain a set of member implementations, so we can't use mixins to implicitly add an implementation of a new method m to existing subtypes of a given class

Sure, I understand the actual behavioral difference. But if we evolve the ecosystem to encourage people to get in the habit of using with instead of implements, then I think we can get to a point where we don't feel much need to add another feature. I worry a lot that we're starting to get close to our complexity limit for the language. We've piled a lot of features in over the past couple of years, and our and our users' brains aren't getting any bigger. :)

[eernstg]

Ah, sorry—I didn't understand how many changes to the semantics of mixins you were considering, in order to get a similar effect as implements plus interface default methods. But of course everybody's brain is getting better every day they use Dart! ;-)

[lrhn]

Another use-case came up.

We are considering (no promises) to move some utility extension methods on Iterable from package:collection to dart:core. This includes firstOrNull. It's an easy extension method to write. However, it feels asymmetric without lastOrNull. The problem is that lastOrNull is not easy to implement efficiently as an extension method. You have to either

• Manually iterate from start to end of the iterable, which is inefficient for iterables which can do an efficient last.
• Use .isEmpty + .last, which risks starting two iterations, and do two computations of the first element. If the underlying iterable is a .where computation, there might be more elements computed before deciding .isEmpty.
• Use .last and catch the (State)Error, but that can hide an error happening during iteration.
• Use is _EfficientLenghtIterable to decide optimize some cases (but not all).

There simply is no good way, using only the Iterable interface, to get the last element efficiently if it's there, and not do extra computation, or dangerous error catching, if there are no elements. That's why we're adding lastOrNull to begin with, because it's not there.

All of those solutions are so sub-par, I will prefer not to add lastOrNull as an extension method, because there is no way to get efficient implementations on iterables which can be efficient.

Adding instance members to Iterable today is a breaking change of an insurmountable scale, so that's also out. No amount of saying that people should use with IterableMixin instead will change that a very large group of people didn't, and we can't make them. It's also not how we've recommended people use Iterable. Or mixins. That ship has long sailed, and we need to work with the ecosystem we actually have.

If we had interface default methods, I would add lastOrNull as one, and then make sure that all the platform iterables which can do a better-than-default implementation will do so. Having a way to add methods to an interface, without breaking existing implementations, but also with a way for other classes to override the default, is something we do not have, and it is something we need.

[TimWhiting]

Why not call toList on the Iterable, then isEmpty and last? It should only have to iterate once to create the list, as long as last is implemented efficiently for the built-in list.

[lrhn]

That would definitely iterate the entire iterable, and copy all the elements, even if the iterable could find the last element efficiently. Doing that on a (one billion element) List (which has efficient isEmpty and last) would be horrible overkill. But the extension method cannot know whether an iterable can be efficient. Sure, it can recognize List, Set, Map and Queue, maybe even the internal _EfficientLengthIterable, but it won't recognize a user-created collection. And doing type checks and having multiple paths costs too. The only one who knows is the iterable itself, which is why a virtual method, which the iterable can override with an efficient implementation, is so much better.

[Levi-Lesches]

Having a way to add methods to an interface, without breaking existing implementations, but also with a way for other classes to override the default, is something we do not have, and it is something we need.

Isn't this what extends does? Can someone elaborate on why one would implement a class instead of extending it? My teams and I always extend rather than implement for this reason, and I never saw a reason to need a compiler error if the superclass chose to implement a concrete method. If they needed me to override it, they would make it abstract.

[chen56]

now ,dart 3,

// plan 1:
// base class can do it , you can not impl a base class,can  safe add method.
base class InterfaceAndDefaultImpl {
   String foo() => "X";
}
base class A extends InterfaceAndDefaultImpl{
}

// plan 2:
// or  base mixin can do it , you can not impl a base mixin,can  safe add method.
base mixin InterfaceAndDefaultImplMixin {
  String foo() => "X";
}
base class B with InterfaceAndDefaultImplMixin{
}

[lrhn]

Making the type base will ensure that everybody has to extend it, or mix it in, of ours a mixin. It doesn't solve the primary use-case, though, which is it to make it a non-breaking change to add a new member to an interface. Making an interface into a base class is itself a breaking change, so if you haven't prepared for it, you can still not add a new member.

[chen56]

@lrhn , non-breaking

base is to ensure that mixin interface users should not use implements, but use with, because the semantics of implements is full implementation, not default implementation.base mixin Interface

example:

base mixin Interface {
  // 1. one month ago you defined interface method a
  void a();

  // 2.now, a month later, you have another interface method b
  //   b need a default impl
  void b() {
    print("b: defalut impl");
  }
}

base class OneImpl with Interface {
  // 1. one month ago, you impl a
  @override
  void a() {
    print("a: your impl");
  }

  // 2. one month late, your get b, and no problem
  //    b method have a default impl
}

main(){
  // 1. one month ago, it is ok
  OneImpl().a();

  // 2. a month later，  still no problem
  OneImpl().a();
  OneImpl().b();// and have a new default impl method, you can override it
}

[lrhn]

Absolutely. If your type is already declared as base, the problem is solved.

If it isn't, it's breaking to make the hitherto implementable type be base. With interface default methods, you don't need to make the type base first, before adding a new member.