Support method/function overloads

[nex3]

This has been discussed periodically both in the issue tracker and in person, but I don't think there's a tracking issue yet. (EDIT: Here is the original issue from 2011 - https://github.com/dart-lang/sdk/issues/49).

Now that we're moving to a sound type system, we have the ability to overload methods—that is, to choose which of a set of methods is called based on which arguments are passed and what their types are. This is particularly useful when the type signature of a method varies based on which arguments are or are not supplied.

[jodinathan]

any news on this? maybe with dart 2.0? =]

[lrhn]

Not in Dart 2. This is a significant change to the object model of Dart. Currently, a Dart object has at most one accessible member with any given name. Because of that, you can do a tear-off of a method. If you could overload methods, tear-offs would no longer work. You would have to say which function you tore off, or create some combined function which accepts a number of different and incompatible parameter signatures. It would make dynamic invocations harder to handle. Should they determine that method to call dynamically? That might cause a significant code overhead on ahead-of-time compiled programs.

I don't see this happening by itself. If we make a large-scale change to the object model for other reasons, then it might be possible to accommodate overloading too, but quite possibly at the cost of not allowing dynamic invocations.

[jodinathan]

but with a sound dart we don't have dynamic invocations, do we?

[eernstg]

We can certainly still have dynamic invocations: If you use the type dynamic explicitly and invoke an expression of that type then you will get a dynamic invocation, and it is an important part of the Dart semantics that we have enough information available at run time to actually make that happen safely (that is, we will have a dynamic error if the invocation passes the wrong number of arguments, or one or more of the arguments has a wrong type, etc).

Apart from that, even with the most complete static typing you can come up with, it would still be ambiguous which method you want to tear off if you do x.foo and foo has several implementations. So it's more about first class usage (passing functions around rather than just calling them) than it is about static typing.

[matanlurey]

@lrhn:

If you could overload methods, tear-offs would no longer work.

You already cannot tear off what users write instead of overloads, which is multiple methods:

class Foo {
  void bar() {}
  void barString(String s) {}
  void barNumber(num n) {}
}

... so given that overloads would be sugar for that, I don't see it any worse.

@eernstg:

and it is an important part of the Dart semantics that we have enough information available at run time to actually make that happen safely

Is it being dynamically invokable a requirement? I don't think it is.

I'd heavily like to see a push for overloads in the not-so-distance future. My 2-cents:

(@yjbanov and @srawlins get credit for parts of this discussion, we chatted in person)

Proposal

Don't allow dynamic invocation of overloaded methods

... or limit how they work:

class Foo {
  void bar() => print('bar()');
  void bar(String name) => print('bar($name)');
  void bar(int number) => print('bar($number)');
}

void main() {
  dynamic foo = new Foo();

  // OK
  foo.bar();

  // Runtime error: Ambiguous dispatch. 2 or more implementations of `bar` exist.
  foo.bar('Hello');
}

If you wanted to be real fancy (@munificent's idea, I think), you could have this generate a method that does dynamic dispatch under the scenes. I'm not convinced this holds its weight (and makes overloading, which should be a great static optimization a potential de-opt), but it's an idea.

I realize this adds a feature that is mostly unusable with dynamic dispatch, but Dart2 already has this issue with stuff like reified generics.

Consider this very common bug:

var x = ['Hello'];
dynamic y = x;

// Error: Iterable<dynamic> is not an Iterable<String>
Iterable<String> z = y.map((i) => i);

Limit tear-offs if the context is unknown

Rather, if the context is ambiguous, then make it a static error.

void main() {
  var foo = new Foo();

  // Static error: Ambiguous overload.
  var bar = foo.bar;

  // OK
  var bar = (String name) => foo.bar(name);

  // Also OK, maybe?
  void Function(String) bar = foo.bar;
}

... another option is have var bar = foo.bar basically generate a forwarding closure (a similar de-opt to the dynamic dispatch issue). Again, not my favorite, but I guess no more bad than code already being written.

Side notes

Let's consider how users are working around this today:

1. Using Object or dynamic with is checks and optional arguments:

   class Foo {
   void bar([dynamic nameOrNumber]) {
     if (nameOrNumber == null) {
       print('bar()');
       return;
     }
     if (nameOrNumber is String) {
       // ...
       return;
     }
     if (nameOrNumber is num) {
      // ...
      return;
     }
   }
   }

   • This works with dynamic dispatch
   • This works with tear-offs
   • This isn't very efficient, and it's very hard/impossible to create complex overloads
   • You lose virtually all static typing

2. Creating separate methods or constructors:

   class Foo {
   void bar() {}
   void barString(String s) {}
   void barNumber(num n) {}
   }

   • This doesn't work with dynamic dispatch
   • This doesn't work with tear-offs
   • This is the most efficient, but cumbersome and creates a heavy API surface
   • Best static typing

I think the idea for overloads is no worse than 2, and you can still write 1 if you want.

EDIT: As @srawlins pointed out to be, another huge advantage of overloads over the "dynamic"-ish method with is checks is the ability to have conditional type arguments - that is, type arguments that only exist in a particular context:

class Foo {
  void bar();
  T bar<T>(T i) => ...
  List<T> bar<T>(List<T> i) => ...
  Map<K, V> bar<K, V>(Map<K, V> m) => ...
}

It's not possible to express this pattern in dynamic dispatch (or with a single bar at all).

[matanlurey]

By the way, this would have solved the Future.catchError issue:

class Future<T> {
  Future<T> catchError(Object error) {}
  Future<T> catchError(Object error, StackTrace trace) {}
}

... as a bonus :)

[eernstg]

@matanlurey,

Is it being dynamically invokable a requirement? I don't think it is.

That was actually the point I was making: It is important that there is a well-defined semantics of method invocation, and if just one static overload is allowed to exist then every dynamic invocation will need to potentially handle static overloads, and that presumably amounts to multiple dispatch (like CLOS, Dylan, MultiJava, Cecil, Diesel, etc.etc.), and I'm not convinced that it is a good trade-off (in terms of the complexity of the language and its implementations) to add that to Dart.

In particular, the very notion of making the choice among several method implementations of a method based on the statically known type is a completely different mechanism than standard OO method dispatch, and there is no end to the number of students that I've seen over time who just couldn't keep those two apart. (And even for very smart people who would never have a problem with that, it's likely to take up some brain cells during ordinary daily work on Dart projects, and I'm again not convinced that it's impossible to find better things for those brain cells to work on ;-).

[matanlurey]

@eernstg:

and if just one static overload is allowed to exist then every dynamic invocation will need to potentially handle static overloads

Why? If we just don't allow dynamic invocation to invoke static overloads, nothing is needed.

In particular, the very notion of making the choice among several method implementations of a method based on the statically known type is a completely different mechanism than standard OO method dispatch, and there is no end to the number of students that I've seen over time who just couldn't keep those two apart

I just want what is already implemented in Java/Kotlin, C#, or other modern languages. Do they do something we aren't able to do, or is this just about preserving dynamic invocation? As I mentioned, the alternative is users write something like this:

class Foo {
  void bar() {}
  void barString(String s) {}
  void barNumber(num n) {}
}

Not only do we punish users (they have to name and remember 3 names), dynamic invocation cannot help you here (mirrors could of, but that is no longer relevant).

[nex3]

It's worth mentioning that if we decide to support overloads without dynamic invocations, this means that adding an overload to an existing method will be a breaking change--one that probably won't be obvious to API designers.

[matanlurey]

Depending how we do it, we theoretically could support a dynamic fallback overload:

class Future<T> {
  // This one is used for any dynamic invocations only.
  Future<T> catchError(dynamic callback);
  Future<T> catchError(void Function(Object));
  Future<T> catchError(void Function(Object, StackTrace));
}

It's not clear to me this is particularly worth it, though. Other hotly requested features like extension methods would also suffer from being static only, and changing a method from invocation to extension would be a breaking change.

[nex3]

I expect it won't be too surprising to users that changing an existing method is a breaking change. Adding a new method being a breaking change, on the other hand, is likely to be very surprising, especially since it's safe in other languages that support overloading.

[matanlurey]

Right, because they never supported dynamic invocation (or only do, like TypeScript).

One project @srawlins was working on back in the day was a tool that could tell you if you accidentally (or on purpose) introduced breaking changes in a commit. I imagine a tool could help, or we could even add a lint "avoid_overloads" for packages that want to be dynamically-invoke-able.

[nex3]

Users aren't going to know to run a tool to tell them that overloads are breaking changes any more than they're going to know that overloads are breaking changes. And even if they did, the fact that adding an overload requires incrementing a package's major version would make the feature much less useful for anyone with downstream users.

I don't think a lint would do anything, because upstream API authors don't control whether their downstream users dynamically invoke their APIs. In fact, since we don't have robust and universal support for --no-implicit-dynamic, the downstream users probably also don't know when they're dynamically inovking APIs.

[matanlurey]

OK, I think we can note that this feature would be breaking for dynamic invocation and leave it at that.

The language team hasn't given any indication this particular feature is on the short-list for any upcoming release, and I'm assuming when and if they start on it we can revisit the world of Dart (and what support we have for preventing dynamic invocation entirely).

I would like to hope this issue continues to be about implementing the feature, not whether or not it will be a breaking change (for all we know this will happen in Dart 38, and dynamic invocation has been disabled since Dart 9).

EDIT: For anyone reading this, I am not saying that will happen.

[munificent]

I also think overloading would be a fantastically useful feature, but it's complexity is not to be under-estimated. If the language folks seem to cower in fear every time it comes up, that's not without reason. This tweet sums it up pretty well:

C# language design is

10% exploring cool ideas

75% overload resolution

15% being sad at past decisions we made

[yjbanov]

This tweet sums it up pretty well

It's nice of the author to leave a hint though: "I would do it more like F#. It is there in a very basic, simple form" :smile:

[matanlurey]

@munificent Definitely understand it shouldn't be underestimated. Do we have a requirement that all new features support dynamic invocation? If so, didn't we already break that with type inference?

[eernstg]

I'm with @munificent on the need to recognize the complexity ('C#: 75% is overload resolution' ;-), but I'm not worried about the complexity of specifying or even implementing such a feature, I'm worried about the complexity that every Dart developer is involuntarily subjected to when reading and writing code. In particular, I'm worried about the inhomogeneous semantics where some decisions are based on the properties of entities at run time, and other decisions are based on properties of entities produced during static analysis—one is fine, the other is fine, but both at the same time is costly in terms of lost opportunities for developers to think about more useful things.

One way we could make the two meet would be based on a dynamic mechanism that compilers are allowed to compile down to a static choice whenever that's guaranteed to be correct. For instance, using the example from @matanlurey as a starting point:

abstract class Future<T> {
  ...
  Future<T> catchError(Function)
  case (void Function(Object) onError)
  case (void Function(Object, StackTrace) onError)
  default (Function onError);
  ...
  // Could be a bit nicer with union types.
  Future<T> catchError2(void Function(Object) | void Function(Object, StackTrace))
  case (void Function(Object) onError)
  case (void Function(Object, StackTrace) onError);
}

class FutureImpl<T> implements Future<T> {
  Future<T> catchError
  case (void Function(Object) onError) {
    // Implementation for function accepting just one argument.
  }
  case (void Function(Object, StackTrace) onError) {
    // Implementation for function accepting two arguments.
  }
  default (Function onError) => throw "Something";
  ...
  // The variant with union types would just omit the default case.
}

There would be a single method catchError (such that the tear-off operation is well-defined and preserves the full semantics), and the semantics of the declared cases is simply like a chain of if-statements:

  Future<T> catchError(Function onError) {
    if (onError is void Function(Object)) {
      // Implementation for function accepting just one argument.
    } else if (onError void Function(Object, StackTrace) onError) {
      // Implementation for function accepting two arguments.
    } else {
      default (Function onError) => throw "Something";
    }
  }

However, the declared cases are also part of the interface in the sense that implementations of catchError must handle at least the exact same cases, such that it is possible to generate code at call sites where it is statically known that the argument list satisfies a specific case. In that situation we would have code that directly calls one of those cases (such that there is no run-time penalty corresponding to the execution of a chain of if-statements, we have already chosen the correct branch at compile-time).

For instance, we always know everything about the type of a function literal at the call site. Special types like int and String are constrained by the language such that we can't have one instance which is both at the same time, and with sealed classes we can have more cases with that property.

This means that we will have multiple dispatch in a way where the priority is explicitly chosen by the ordering of the cases (so we avoid the infinite source of complexity which is "ambiguous message send"), and the mechanism will double as a static overloading mechanism in the cases where we have enough information statically to make the choice.

I'm not saying that this would be ridiculously simple, but I am saying that I'd prefer working hard on an approach where we avoid the static/dynamic split. And by that split I don't mean code which is full of expressions of type dynamic, I mean code which may be strictly typed (--no-implicit-cast and whatnot), because that would still allow the same operations applied to the very same objects to behave differently, just because the type checker doesn't know the same amount of things at two different call sites.

... Java/Kotlin, C#, or other modern languages. Do they do something we aren't able to do, or is this just about preserving dynamic invocation?

Neither (we can surely make a big mess of things as well ;-), but, to me, it is very much about avoiding a massive amount of subtleties for the developers, also for code which is statically typed to any level of strictness that we can express.

[matanlurey]

Most Dart developers don't want dynamic invocation (in Dart2, it is actively bad in many places with reified types), so it seems to me trying to preserve that feature for new language features isn't worth the time or effort.

[eernstg]

@matanlurey, if that's concerned with this comment, it sounds like maybe you did not notice that I'm not talking about dynamic invocations, or certainly not only about them:

I don't mean code which is full of expressions of type dynamic, I mean .. strictly typed .. [code that still causes a] massive amount of subtleties for the developers

[munificent]

Do we have a requirement that all new features support dynamic invocation?

I think dynamic invocation is red herring. C#'s dynamic supports full C# overload resolution. The complexity isn't around dynamic invocation. It is more inherent to how overload resolution interacts with type inference, overriding, generics, generic methods, optional parameters, and implicit conversions.

(We don't have implicit conversions in Dart yet, but we will once you can pass 0 to a method that expects a double.)

I just slapped this together, but here's a sketch that might give you a flavor of how it can get weird:

class Base {
  bar(int i) {
    print("Base.bar");
  }
}

class Foo<T extends num> extends Base {
  bar(T arg) {
    print("Foo<$T>.bar");
  }
}

test<T extends num>() {
  Foo<T>(null);
}

main() {
  test<int>();
  test<double>();
}

[matanlurey]

I might be ignorant, but isn't there a similar set of complexity for extension methods? Meaning that if we need to eventually figure out how to dispatch extension methods, at least some of the same logic holds for dispatching overload methods?

(It looks like, casually, most OO languages that support extensions support overloading)

[munificent]

Potentially, yes, but I think they tend to be simpler. With extension methods, you still only have a single "parameter" you need to dispatch on. You don't have to worry about challenges around tear-offs. Things might get strange if we support generic extension classes. I don't know. But I would be surprised if extension methods weren't easier than overloading.

[matanlurey]

Thanks for this! A few more questions, but don't feel like they are important to answer immediately :)

Potentially, yes, but I think they tend to be simpler.

Are there some limitations we could add to overloads to make them easier to implement and grok? I might be incredibly naive ( /cc @srawlins ) but I imagine 95%+ of the benefit could be gained with a few simplifications:

• Either no tear-off support, or force users to write void Function(String) bar = foo.bar
• Don't support overloading on bottom or top types
• Don't support overloading on generic types

For example, today I was writing a sample program for a r/dailyprogramming question.

I wanted to be able to write:

abstract class DiceRoll {
  int get amount;
  int get sides;
}

abstract class DiceRoller {
  /// Roll a dice defined by the expression "NdN".
  List<int> roll(String expression);

  /// Roll [amount] of dice with [sides].
  List<int> roll(int amount, int sides);

  /// ...
  List<int> roll(DiceRoll roll);
}

But I'd either have to write:

abstract class DiceRoller {
  /// Roll a dice defined by the expression "NdN".
  List<int> rollParse(String expression);

  /// Roll [amount] of dice with [sides].
  List<int> roll(int amount, int sides);

  /// ...
  List<int> rollFor(DiceRoll roll);
}

Or do something extra silly like:

abstract class DiceRoller {
  List<int> roll(dynamic expressionOrAmountOrRoll, [int sides]) {
    if (expressionOrAmountOrRoll is int) {
      if (sides == null) {
        throw 'Expected "sides"';
      }
      return _rollActual(expressionOrAmountOrRoll, sides);
    }
    if (sides != null) {
      throw 'Invalid combination';l
    }
    if (expressionOrAmountOrRoll is String) {
      return _rollAndParse(expressionOrAmountOrRoll);
    }
    if (expressionOrAmountOrRoll is DiceRoll) {
      return _rollActual(expressionOrAmountOrRoll.amount, expressionOrAmountOrRoll.sides);
    }
    throw 'Invalid type: $expressionOrAmountOrRoll';
  }
}

The former is hard for the users to use (and find APIs for) and the latter sucks to write, test, and basically forgoes any sort of static type checking.

You don't have to worry about challenges around tear-offs.

Does that mean tear-offs wouldn't be supported for extension methods, or that it's easier?

I imagine folks would find it weird if you could do:

void main() {
  // This will, or will not work, depending on if `map` is an extension method or not?
  wantsAClosure(['hello', 'world'].map);
}

void wantsAClosure(Iterable<String> Function(String) callback) {}

Things might get strange if we support generic extension classes

Do you mean (psuedo-syntax):

  /// ['hello, 'world'].joinCustom()
  extension String joinCustom(this Iterable<String> parts) {
    // ...
  }

Or:

  extension Map<K, V> groupBy<K, V>(this Iterable<V>, K Function(V) groupBy) {
    // ...
  }

[munificent]

Either no tear-off support, or force users to write void Function(String) bar = foo.bar

I believe the latter is what C# does. It helps, though it causes some weird confusing behavior. It's always strange for users when you can't take a subexpression and hoist it out to a local variable.

Don't support overloading on bottom or top types

I don't think that's the cause of much of the pain.

Don't support overloading on generic types

That might help, but it's probably too painful of a limitation in practice. One of the key uses of overloading is being able to extend the core libraries without breaking them, and many of the classes where that would be most helpful, like Iterable and Future, are generic.

abstract class DiceRoller {
  /// Roll a dice defined by the expression "NdN".
  List<int> roll(String expression);

  /// Roll [amount] of dice with [sides].
  List<int> roll(int amount, int sides);

  /// ...
  List<int> roll(DiceRoll roll);
}

Yeah, I've run into this exact scenario.

Just allowing overloading by arity (number of parameters) would help many of these simple cases and doesn't require a lot of static typing shenanigans. Dynamically-typed Erlang supports it. Though it would interact in complex ways with optional parameters in Dart.

You don't have to worry about challenges around tear-offs.

Does that mean tear-offs wouldn't be supported for extension methods, or that it's easier?

That it's easier. Once you've don't the extension method lookup statically, you know exactly what method is being torn off, so you can just do it.

With overloading, there are interesting questions around whether the lookup should be done statically, dynamically, or some combination of both.

Things might get strange if we support generic extension classes

Do you mean (psuedo-syntax):

/// ['hello, 'world'].joinCustom()
extension String joinCustom(this Iterable<String> parts) {
  // ...
}

I mean:

extension class Iterable<int> {
   int sum() => fold(0, (sum, element) => sum + element);
}

test<T>(Iterable<T> elements) {
  elements.sum(); // <--???
}

I'm sure @leafpetersen and @lrhn can figure out how to handle all of this, but I'm not sure if I could. :)

[matanlurey]

Thanks! I am sure I will understand this issue eventually :)

One of the key uses of overloading is being able to extend the core libraries without breaking them, and many of the classes where that would be most helpful, like Iterable and Future, are generic.

Did you mean one of the key uses of extension methods, or overloading?

[lrhn]

Overloading and extension methods are orthogonal. Both allow "adding a method" to a class without breaking an existing method with the same name. If you have both, there is a good chance that the extension method won't completely shadow the original method. Extension methods are not virtual, which is annoying. You can add them from the side, which is useful. We don't have a way to add a virtual method from the side, and I'm not sure it's possible.

The languages with overloading mentioned so far do not have optional parameters the same way Dart does. They do have optional positional parameters, so that might not be an issue. We still have to handle cases like:

  int foo(int x, [int y, int z]) => ...
  int foo(int x, {int y, int z}) => ...
  ...
     theFoo.foo(42);

Likely it's just an unresolved overload error at compile-time. Again, a dynamic invocation might not apply here, but if it does, then it's not clear that there is a solution. Maybe we can solve it by (theFoo.foo as int Function(int, int, int))(42). I'd like as to actually induce a preference on the expression.

As for

extension class Iterable<int> {
   int sum() => fold(0, (sum, element) => sum + element);
}
test<T>(Iterable<T> elements) {
  elements.sum(); // <--???
}

my way of figuring that one out would just be to say "extension method does not apply". The static type of elements is Iterable<T>, which is not the same as, or a subtype of, Iterable<int>, so the static extension method cannot be used. Since elements does not have a sum method, your program won't compile. Now, if it had been:

test<T extends int>(Iterable<T> elements) {
  elements.sum(); // <--???
}

then the extension method would likely have applied.

More controversial is:

extension List<T> {
  R join<R>(R base, R Function(R, T) combine) => ...;
}

Should that function "override" the join function on List? Shadow join completely, or only act as an alternative overload? What if I named it fold instead? That kind of conflict is troublesome, but probably not a real issue (it'll just allow people to shoot themselves in the foot, and linting can tell you to stop being silly).

Anyway, this is about overloading, not extension methods.

[jodinathan]

Would it make it easier if it is available only for positioned parameters? From my experience, overloadable methods are normally very well defined and with few parameters.

Atenciosamente,

Jonathan Rezende

Em 9 de jul de 2018, à(s) 06:53, Lasse R.H. Nielsen notifications@github.com escreveu:

Overloading and extension methods are orthogonal. Both allow "adding a method" to a class without breaking an existing method with the same name. If you have both, there is a good chance that the extension method won't completely shadow the original method. Extension methods are not virtual, which is annoying. You can add them from the side, which is useful. We don't have a way to add a virtual method from the side, and I'm not sure it's possible.

The languages with overloading mentioned so far do not have optional parameters the same way Dart does. They do have optional positional parameters, so that might not be an issue. We still have to handle cases like:

int foo(int x, [int y, int z]) => ... int foo(int x, {int y, int z}) => ... ... theFoo.foo(42); Likely it's just an unresolved overload error at compile-time. Again, a dynamic invocation might not apply here, but if it does, then it's not clear that there is a solution. Maybe we can solve it by (theFoo.foo as int Function(int, int, int))(42). I'd like as to actually induce a preference on the expression.

As for

extension class Iterable { int sum() => fold(0, (sum, element) => sum + element); } test(Iterable elements) { elements.sum(); // <--??? } my way of figuring that one out would just be to say "extension method does not apply". The static type of elements is Iterable, which is not the same as, or a subtype of, Iterable, so the static extension method cannot be used. Since elements does not have a sum method, your program won't compile. Now, if it had been:

test(Iterable elements) { elements.sum(); // <--??? } then the extension method would likely have applied.

More controversial is:

extension List { R join(R base, R Function(R, T) combine) => ...; } Should that function "override" the join function on List? Shadow join completely, or only act as an alternative overload? What if I named it fold instead? That kind of conflict is troublesome, but probably not a real issue (it'll just allow people to shoot themselves in the foot, and linting can tell you to stop being silly).

Anyway, this is about overloading, not extension methods.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dart-lang/sdk/issues/26488#issuecomment-403424824, or mute the thread https://github.com/notifications/unsubscribe-auth/AF6Y3jQTXj788r5T9g9c8q-24N1wE3Qaks5uEygbgaJpZM4IhzBM.

[matanlurey]

I also tend to agree that trying to combine overloads and optional parameters (either named or positional) is probably not worth its weight. A lot of the places that optional parameters are used today are to emulate overloads, and users would likely use overloads instead if available.

[nex3]

Not allowing optional named parameters with overloads will make it difficult to backwards-compatibly extend APIs that were originally defined as overloads. This would incentivize API designers to add lots of overloads with additional positional parameters, which is generally less readable and more fragile than named parameters.

My preference for APIs like

  int foo(int x, [int y, int z]) => ...
  int foo(int x, {int y, int z}) => ...

would be to disallow the definition at compile time. This ensures that overload calls are never ambiguous, and that API designers are aware when they try to design an API that would be ambiguous and can avoid it (e.g. by making int y mandatory in the first definition above).

[matanlurey]

Not allowing optional named parameters with overloads will make it difficult to backwards-compatibly extend APIs that were originally defined as overloads. This would incentivize API designers to add lots of overloads with additional positional parameters, which is generally less readable and more fragile than named parameters.

I mean, if that allows overloads 5 years sooner than trying to capture all use cases, I'd still prefer it - the reality looks like even if we got the language team to agree in principal, needing to capture all uses cases (dynamic invocation, optional parameters, etc) would likely be such an effort that it would not be prioritized.

It would be trivial to move APIs that have heavy use of optional positional parameters to use overloads. Here is @munificent's example case:

// Before
int range(int minOrMax, [int max]);

// After
int range(int max);
int range(int min, int max);

For the named case, its not easier, but the option here is to just not change APIs that used named parameters to use overloads.

[rrousselGit]

Somehow I see a lot in this conversation the need of overload to add a new callback parameter like in :

  Future<T> catchError(void Function(Object));
  Future<T> catchError(void Function(Object, StackTrace));

I think another solution to this problem would be to make void Function(Object) assignable to void Function(Object, StackTrace)

Typescripts allows it. It's pretty useful for many methods such as Array.map((T, index) => S) where we usually don't need index.

It won't solve all overloads problems, but will definitely help.

[RdeWilde]

Would really like this. Most major languages support it.

[lrhn]

Making void Function(Object) assignable to void Function(Object, StackTrace) is a tantalizing idea.

It would mean that you can call any function with more arguments than it expects, the extra arguments are just "lost". It's really the dual of optional parameters. You can also see it like every function value actually has all optional parameters (infinite positional parameters, all names) that it just doesn't use, and declaring parameters makes the type more restrictive in that the argument doesn't accept everything any more, and it might be required.

It would make it harder to detect some errors, but we can still make it an error to pass more arguments than the static type expects (there is no idea what that argument means), so it's only invalid dynamic calls, and you're really asking for it there.

It would also allow a super-class to add more arguments. Subclasses which don't expect these arguments are still valid. That might actually be a problem, because there will be no hint that the subclass didn't omit the parameter deliberately, and it's likely to not be able to act correctly on calls that pass that parameter and expects it to have a meaning. Still, adding parameters to superclasses is huge, since it's currently a completely breaking change.

As for overloading, that makes it much harder for Dart to maintain its dynamic behavior. A dynamic call uses only the name to select the method. I guess it could also use the actual argument types to select the more precise one, but at the expense of having to do this dispatch at run-time. It's not clear which method to extract on a tear-off (we might be able to wing that based on context type, and fail if it's not enough, e.g., on a dynamic tear-off). Overriding not a clear match for the Dart object model, it will need to be designed in a way that is both backwards compatible and forwards usable. I have no idea whether that is actually simple or not :)

[matanlurey]

@lrhn:

is a tantalizing idea.

I disagree. We are just making the language silently accept more things that might not be intended. There would be a lint created for this and enforced in Flutter and Google Day 1 this feature is added to the language.

As for overloading, that makes it much harder for Dart to maintain its dynamic behavior

We don't want dynamic behavior (https://github.com/dart-lang/sdk/issues/26488#issuecomment-402532000). The alternative to adding overloads in the language is to have users write the overloads themselves (https://github.com/dart-lang/sdk/issues/26488#issuecomment-401424881) - either with specialized methods or named constructors. Neither of those support dynamic invocation either, so that argument doesn't hold weight for me.

[natebosch]

It would mean that you can call any function with more arguments than it expects, the extra arguments are just "lost". It's really the dual of optional parameters.

For cross-linking purposes this was also discussed in dart-lang/sdk#10811

[rrousselGit]

I disagree. We are just making the language silently accept more things that might not be intended.

How so? I don't quite see how can this lead to problems. We are talking about callbacks, not any function call. Typescript has been doing it for years and there's no tslint rules preventing it. I haven't seen anybody complaining about it either. It's actually a big helps for a functional programming approach.

I find this healthier than having thousands of closures with _ as parameters everywhere. Example:

PageRouteBuilder(
  pageBuilder: (_, __, ___) => Foo()
)

[matanlurey]

Dart has different calling conventions than JavaScript, so it's not a 1:1 fit.

[lrhn]

Even if we don't allow calling with more arguments than the static type allows[1], it would still allow a unary function to be assigned to a binary function type, which may or may not be a mistake. If we silently allow it, then it might hide some error. That's always a trade-off: The more things you allow, the more permissive the language is, the fewer errors you can detect because the language assigns a meaning to the program anyway. On the other hand, the more restrictive you are, the more you require the programmer to jump through hoops to tell the language why something is OK. Different people have different preferences for this trade-off, and, as Matan shows, Google code tend to be on the very restrictive side, preferring to reject potentially wrong programs at the cost of some false positives and extra developer work, where Dart has traditionally been very permissive in allowing the programmer to decide what is correct rather than the language.

You can fit a restrictive language for a particular use-case, so you don't see overhead when sticking to that particular style, but it always comes up anyway.

Dart 2 is more restrictive than Dart 1 - for example a List<dynamic> is no longer a List<int>, and a class not implementing its interface is an error rather than a warning. Dart 2 is still more permissive than some other language, allowing you to do dynamic invocations at run-time. Some want that gone too, others do not.

What I do believe strongly is that Dart will need a way to evolve an API without immediately breaking all clients of the API. We currently do not have that, so any change to a public API is a breaking change. Packages might get away with breaking changes because of versioning (at the risk of "version hell"), but the platform libraries cannot.

Allowing "over applying" arguments to a function is a change that tips the trade-off more towards permissibility. It's perfectly safe from a language and type perspective, but it might hide some domain-semantic errors, where you are passing the wrong function and it's allowed anyway.

An explicit coercion might be a way to get the ability without the permissibility. If I write stream.catchError(print!) and that automatically coerces print to (Object v1, StackTrace v2) => print(v1), then I get to cheaply (rather than freely) pass a lower-arity function where a higher-arity function is allowed, but I have to opt in to it at the actual coercion point. It won't work for a declaration-site which wants to be permissible. We could have that too, so catchError was typed as

Future<T> catchError(FutureOr<T> Function(Object, StackTrace)! handler, [...])

which means that it auto-coerces its first argument if necessary. That's just a local "annotation" on a static function type, not an intrinsic part of the function type itself. you can cast in and out of the auto-coercion for the same function.

So, yes, a tantalizing idea. Maybe not practically applicable in its raw form (you won't know without considering it and trying to see if the issues it introduces can be managed), but it has some positive perspectives that are worth remembering.

[rrousselGit]

I think the ability to reduce breaking changes and improve readability is important. Whether it is achieved by method overload or less restricting callback assignations doesn't matter.

The current behavior causes problems. For example, in Rxdart, we have combineLatest2, combineLatest3, ..., combineLatest7. This hurts readability. combineLatest2 could be interpreted as taking the 2 latest values. This hurts discoverability too, as the IDE will suggest 7 times the same function in auto-complete. Library maintainers also need to document the same function many times.

Also, one of the dart goals is to be easy to pick up for JS/java/c# developers. All of which have their own way to handle multiple function prototypes under the same name.

[munificent]

A dynamic call uses only the name to select the method. I guess it could also use the actual argument types to select the more precise one, but at the expense of having to do this dispatch at run-time. It's not clear which method to extract on a tear-off (we might be able to wing that based on context type, and fail if it's not enough, e.g., on a dynamic tear-off).

You can look at Dart's current optional parameters as defining "overloads" — there are multiple valid signatures you can use to invoke the function. There is "runtime dispatch" going on in that the function body must substitute in default values for parameters that are not passed.

In Dart today, when you tear-off a method that takes optional parameters you get all of the "overloads" bundled together — the function type retains optionality.

If we were to add real overloads, I think it would be great to preserve all of those properties. When you tear off a method, you get a callable object that retains all of the overloads for the given name. This means function types get pretty complex (!) — something like a intersection type for all of the signatures.

But it preserves dynamic calls, aligns with the existing behavior and is, I think, a very powerful, useful feature. Effectively multimethods, which would really set us apart from other statically-typed languages that can only do single dispatch.

What I do believe strongly is that Dart will need a way to evolve an API without immediately breaking all clients of the API. We currently do not have that, so any change to a public API is a breaking change.

+100

Packages might get away with breaking changes because of versioning (at the risk of "version hell"), but the platform libraries cannot.

I see this as an argument for making the platform libraries smaller and relying on external packages for more functionality. Doing that in a usable way probably means something like extension methods so that you don't sacrifice the nice object.method(...) syntax.

Of course, combining extension methods and runtime-dispatched overloads isn't easy. :)

[lrhn]

@munificent

We can definitely treat optionally-parameterized functions as implementing multiple overloads in one declaration. Tearing off all overloads in one go is a big mouthful, but we could aim lower and allow you to tear off a sub-set of the methods which can be described by a single function type (including optional parameters). The tear-offs might not cover the entire function type, and hitting the holes will cause errors, but it's still typable without having to introduce general union types.

I see this as an argument for making the platform libraries smaller and relying on external packages for more functionality.

If we can make the platform libraries smaller, then we have already solved the problem of making at least some breaking changes to the platform libraries. :)

[munificent]

Serious question: do we know real examples of APIs which could benefit from method overloading?

Sure, look at any Java or C# API. Overloading is used heavily and often in patterns that don't map to Dart's support for trailing optional parameters.

A much closer example is +. int + int should return int, while int + double returns double. Right now, the language specification special-cases that operator and a couple of others. Basically there are a handful of overloaded methods. Sort of a "overloading for me but not for thee" situation.

But, since each of those has to be special-cased in the spec, some aren't covered. It would be nice if int.clamp(int, int) returned an int, but it doesn't. It returns num.

[rrousselGit]

A more obvious example that is currently impossible with dart optional parameters: Different return types

Other languages allow defining a compose or pipe function, which fuses multiples functions into one. Very useful for functional programming.

The problem is that is has a prototype similar to the following:

R pipe<T, T2, R>(Func1<T, T2> cb, Func1<T2, R> cb2);
R pipe<T, T2, T3, R>(Func1<T, T2> cb, Func1<T2, R> cb3, Func1<T3, R> cb3);

In dart we'd have to have different name for the same function. Which leads to a pipe2, pipe3, ...

Rxdart library faces this exact problem with combineLatest operator. Where it defined 7 times the same function with different names

[RdeWilde]

Also custom operators can act differently based on the parameter type. For example all sorts of numerics (float, int, bigint, etc).

[rrousselGit]

If dart ever has non-nullable types (and looks like it will), then this feature has an entirely new usage.

There are some situations where with custom parameters, the prototype may switch between nullable and non-nullable. Something that would not be possible with our current optional parameters.

[munificent]

This is a good point. Do you have any concrete APIs you can point us at where this comes into play?

[rrousselGit]

StreamBuilder is a potential use-case due to its initialData.

In a non-nullable world, the StreamBuilder API could evolve to make it null safe:

StreamBuilder<T>({
  @required Stream<T> stream,
  T initialData,
  @required Widget builder(BuildContext context, T value, ConnectionState state),
  Widget errorBuilder(BuildContext context, dynamic error, ConnectionState state), 
});

We have to scenarios here:

• initialData is omitted => T value of builder is nullable
• initialData is present => T value is not nullable (unless T itself is nullable)

[rrousselGit]

It basically fits with any optional parameters that depend on other parameters.

Consider the following function prototype:

foo({ bar, baz });

Sometimes we may do things like:

assert(bar != null || baz != null)

which could translate into:

foo({ @required bar, baz });
foo({  bar, @required baz });

or

assert((bar != null && baz == null) || (baz != null && bar == null));

which translates into:

foo({ @required bar });
foo({ @required baz });

---

A more concrete example is thePositionned widget where we cannot specify simultaneously all arguments.

Another example is DecoratedBox, where we cannot specify both color and decoration together.

[nex3]

Iterable.firstWhere() is a core library example. Ideally it would have three signatures:

E firstWhere(bool test(E element));
E firstWhere(bool test(E element), {E orElse()});
E? firstWhere(bool test(E element), {E? orElse()});