dart-lang / language

Design of the Dart language
Other
2.65k stars 203 forks source link

Support operator (parameter) overloading #2456

Open Andrflor opened 2 years ago

Andrflor commented 2 years ago

They have been a lot of discussions around function overloading tracked on #1122 But what about only operator parameter overloading?

Some people on the web talk about "operator overloading" in dart. But such a thing does not exists, they confuse "operator overriding" and "operator overloading"

Contrary to function overloading, operator overloading would not add any particular complexity since overloads would share the same return type, have no tear off and always exactly one parameter.

For the use cases I'm thinking about Transparent Reactive Programming. But there are many more.... Let me cite here this excellent post by @Terrain2 in #1122

Operators

Personally i don't actually think method overloading would add that much to the language (though it would be nice), but i would really want to overload multiple signatures of the same operator, if you look at the implementation of + for int, you'll find it inherits from num, thus num operator +(num other) is the signature - however, int + int evaluates to an int, even though that's not what the operator says it does, almost as if int has 2 signatures for +: int operator +(int other) and num operator +(num other). This makes sense, because if you add 2 integers, you won't get a fractional value, thus it's safe to say it's an integer. This isn't possible to implement in pure dart, this is simply a part of the type system and int being one of the core types, so therefore it gets special treatment, just like num is really a union of int|double, FutureOr is literally just a union type, and bool can't be extended or instantiated, it's just the type shared by 2 objects, all 3 of these examples make perfect sense, but are also impossible to implement in pure dart in the same way.

I'd imagine that for operators, there might not be as much overhead for multiple implementations based off the signature, since they're always restricted to a single required parameter, and overload resolution could essentially be simplified to dynamic operator +(dynamic other) => implementations[other.runtimeType](other); where implementations is a map, of course then i guess it's based on the runtime type and not the static type, so if an implementation used num it couldn't take an int, so obviously something a little more sophisticated, but certainly a hell of a lot faster than needing to look up multiple parameters with signatures, aswell as optional parameters which more or less conflict with overloading, cause if you have range(int min, [int max = 100]) and range(int max), which one is called with range(3)?

I'd imagine that statically, it should look for the closest type in the supertypes of the operand's type (i.e. first checking if anything takes int, then int?, then num, then num?, then Object, then Object?, and finally dynamic, and if the operand is Never then no implementation needs to be chosen at all), and dynamically (for faster resolution), it could look for in some given order the first implementation that supports the runtime type of the object (perhaps simply the order it's written in the class with subtypes being before their supertypes, and something more like operator +(other) => implementations.firstWhere((impl) => other is impl.operandType, orElse: () => throw TypeError());)

With multiple operator overloads per operator, equality would be less of a pain to implement

class Foo {
  bool operator ==(Foo other) => true;
}

void main() {
  assert(Foo() == Foo());
  assert(Foo() != Object()); // Inherited from Object, and these are not the exact same instance
}

and i'd also love to be able to add extension operators that are already implemented in the class, for other types, but that isn't possible right now

extension BigIntArithmetic on int {
  BigInt operator +(BigInt other) => BigInt.from(this) + other;
}
extension IntArithmetic on BigInt {
  BigInt operator +(int other) => this + BigInt.from(other);
}

void main() {
  print(BigInt.one + 1); // 2
  var x = BigInt.one;
  x++; // Incrementing on BigInts! heck yeah!
  print(x); // 2
}

Maybe a restriction for this would be that you can't overload for a subtype of what's already overloaded in extension methods? but a really great use case would be defining custom equality rules for a class that has never heard of your library via extension methods, and that would break this use case, albeit only for the == operator specifically - maybe an alternative could be allowing extension methods to add another implementation for an operator if, and only if, the only operator in the superclass that supports the type is of Object, and the given type will always return false in that implementation, a common pattern in equality operators is to check for each type before doing anything, otherwise returning false, so in a chain of such type checks, if the extension's operand type skips all of them and will be false, the overload is accepted for an extension

class Foo {
  final num value;
  Foo(this.value);
  bool operator ==(Object other) {
    if (other is num) return value == other;
    return false;
  }
}

extension FooExtensions on Foo {
  // Allowed: the catch-all implementation can only ever return true if it has a num, which a BigInt isn't
  bool operator ==(BigInt other) {
    return BigInt.from(value) == other;
  }

  // Error: int is a num, and a num could potentially return true in the catch-all, so this is *not* allowed
  bool operator ==(int other) {
    return value == other;
  }
}

class SubFoo {
  // On the other hand, this is completely fine, because it's overridden in a subclass, and as such doesn't change the behaviour of Foo based on what extensions you import...
  bool operator ==(int other) => value == -other;
}

A big benefit i see is any numerical type could support the increment and decrement operators aswell as being able to add its own type with the + operator, and maybe just in case this is the only reason it needs to support int, there could be some annotation that adds a warning to any code using the + or - operators with an int operand, instead of with ++ or --

Originally posted by @Terrain2 in https://github.com/dart-lang/language/issues/1122#issuecomment-735254199

lrhn commented 2 years ago

Dart doesn't have overloading. One of the arguments against overloading is that it makes tear-offs harder. That doesn't apply to operators, because we don't have operator tear-offs anyway. (Well, yet!)

Another argument is that overloading doesn't fit well into Dart's overriding model. Take:

class C { 
  final num value;
  C(this.value);
  C operator +(int x) => C(value + x);
}

That class, by itself, works. Now extend it as:

class D extends C {
  D(super.value);
  D operator +(num x) => D(value + x);
}

This is a currently valid class. The D.+ method overrides the C.+ operator and widens the parameter type.

With overloading, would that be an override of C.+ or a separate overloaded + method?

If I do D d = D(42); C c = d + 1;, would it call C.+ or D.+. The C.+ method has a more specific type (int is a subtype of num, so most languages with overloading would choose C.+). That would be breaking compared to the current Dart behavior.

We'd probably need some new syntax to allow us to separate an override from an overload (absences of @override being a hint). We'd also need a way to choose a specific overload when both works, because if I do D d2 = d + 1;, that's not currently valid because C.+ returns a C. I need to be able to do, effectively, D d2 = d D::+ 1, selecting the + introduced by D. (I guess D d2 = d + (1 as num); would work. Not good usability, though.)

Dart differs from languges with overloading in that we can override with wider parameter types. And that we can add more optional parameters in a subclass (but that doesn't apply to operators). While overloading is easier for operators than it is for methods in general, it's still not easy.

Andrflor commented 2 years ago

This is a currently valid class. The D.+ method overrides the C.+ operator and widens the parameter type.

It wouldn't be a valid operator overload. Like i said in the first post operator overload is only intended for the parameter and not the return type.

This means that

D operator +(num x) => D(value + x);

Is NOT a valid overload of

C operator +(int x) => C(value + x);

The valid overload would be

C operator +(num x) => C(value + x);

A @overload annotation may be better to signify overload When you consider only parameter overloading then there is no conflict with overriding An overload can't have the same parameter as an override or the base operator it must be either a Super Type or an Unrelated Type.

I edited the post to precise that I'm talking about "parameter only" overloading.

lrhn commented 2 years ago

The issue is still that D operator+(num x) => ... from D is currently a valid override of C operator+(int) => ... from C. Dart allows signatures to differ in overridden members.

Whether the return type of D.+ is C or D has no impact on the question of whether D.+ overrides or overloads C.+.

If you say that D operator+(num x)=>... is not a "valid overload", does that mean that it wouldn't overload C.+ (but would still override it), or that it wouldn't be a valid declaration at all (bearing in mind that it's currently a valid declaration, so that would be a very breaking change)?

In order to have overloading, you need to have separate methods with the same name, which is what Dart currently doesn't have. However, given a class like

class Uber {
  void act(int x) { ... }
  void act(double x) { ... }
}

with two distinct, overloaded methods named act (return type not important), what would

class Unter extends Uber {
  void act(num x) { ... }
}

mean? The Unter.act method iscurrently a valid override of both void act(int) and void act(double). Does it override one or both, or does it introduce a third overloaded method named act? And how would you make it actually override one or both of the methods in Uber.?

The Dart model, with covariant overriding on the function type, is a significant distinction from the object models of other languages which have overloading, and where only methods with the same signature can override. (Same parameter signature, in the case of Java, since Java 5 allowed a covariant return type variance.)

Andrflor commented 2 years ago

That's my bad because in fact

C operator +(num x) => C(value + x);

Is not a valid overload either since it's another override. Overload should never be able to override. So a valid overload would be

C operator +(String x) => C(value + x.length);

I wasn't all that clear... Let me sum up the overload constraints: 1- Only for operators. 2 -Only a parameter overload (same return type). 3 - Can't override.

Those constraints should allow to define overloads operators in extensions. Those also mean that

class Unter extends Uber {
  void act(num x) { ... }
}

Can't be an overload for two reason.

  1. It's not an operator.
  2. It's an override.

Why it's working? There is no tear-off on operators There is always exactly one parameter Always the same return type An overload can never be an override.

What happen if I override and overload at the same time? Simple, overloads constraints are based on the overridden operator.

lrhn commented 2 years ago

So, what you are suggesting is parameter-type based overloading that only applies to extension operators:

We actually considered letting argument/parameter types affect whether extension methods applies to an invocation. We decided against it for a couple of reasons.

Primarily, it requires the compiler to do type inference once per possible applicable extension member, since the type of an argument expression can depend on the parameter type of the method being called. We can't just do type inference once and then check all the possible extension methods afterwards. We decided to keep it down to one type inference pass. That's why the extension method is chosen entirely based on the receiver's static type, because we know that before we do type inference on the argument expressions.

Secondarily, we didn't want to introduce overloading that way. In a language without overloading, allowing overloading only with non-virtual extension methods was seen as encouraging people to write their APIs using extension methods instead of instance methods. That would be bad, giving people the trade-off between virtual method or overloading, but never both. That argument still stands.

Andrflor commented 2 years ago

So, what you are suggesting is parameter-type based overloading that only applies to extension operators:

No, this is not what i suggest.

Parameter-type based overloading that only applies to operators

This is what I suggest I never mentioned any extension specificity or constraint

Let me sum up the overload constraints: 1- Only for operators. 2 -Only a parameter overload (same return type). 3 - Can't override.

lrhn commented 2 years ago

Ok, I'll try again to define what such a feature would actually have to look like.

(The "same return type" seems unnecessary and restrictive. Or maybe I don't understand which functions must have the same return type.)

Since operators cannot be torn off, the only case where it matters which of the operator methods you call, is when you actually call, and therefore you will have arguments to use for resolution.

There are some underdefined parts of that algorithm:

Andrflor commented 2 years ago

By different i was thinking of "Non equal and non overriding"

(The "same return type" seems unnecessary and restrictive. Or maybe I don't understand which functions must have the same return type.)

You're right that's unnecessary and restrictive.

  1. I'm not sure what will be a good way to infer the parameters..

  2. The more specific as done with extensions would be amazing.

  3. I completely forgot that =[] had two arguments :face_with_spiral_eyes: The type parameter is problematic indeed. What about defining a priority? Maybe use an annotation to define the priority on an operator that is called with a type parameter? Using @override to make it chosen when ambiguous. (Or introduce a new annotation?) Without annotation have always the other method chosen when disambiguating. Avoidance of any disambiguation at call point would be very cool.

  4. I would say that it overrides all valid overrides (Simpler to understand). Or we could remove inheritance from overloaded methods. (Simple to understand, maybe simpler to implement?, but less powerful). Or like C# virtual method as you said. (This may complexify the language so I would avoid it).

ykmnkmi commented 1 year ago

If Dart has binary extensions like this, this would be nice for vector math:

extension binary on Vector with (num value) {
  Vector operator + {
    var vector = clone();
    return vector.addValue(value);
  }

  Vector operator - {
    var vector = clone();
    return vector.subValue(value);
  }
}

extension binary on Vector with (Vector other) {
  Vector operator + {
    return Vector.add(this, other);
  }
}
mateusfccp commented 1 year ago

That doesn't apply to operators, because we don't have operator tear-offs anyway. (Well, yet!)

@lrhn Is there any issue tracking this? I didn't find anything except for #691, which included not only operator tear-offs but other kinds of tear-offs, but it was closed by the author.

dcharkes commented 10 months ago

Like i said in the first post operator overload is only intended for the parameter and not the return type.

(The "same return type" seems unnecessary and restrictive. Or maybe I don't understand which functions must have the same return type.)

Agreed. I'd want to write:

class Distance {
  Distance operator * (int value) => ...

  Area operator * (Distance value) => ...
}
dcharkes commented 9 months ago

There are some underdefined parts of that algorithm:

  • How are the types of the arguments inferred? Traditionally, Dart looks up the method first, then uses the method's parameter types as context types for the argument expressions. Using the types of the argument expressions to look up the method creates a cyclic dependency. We can either perform argument expression type inference once per possible method (more expensive) or perform argument expression inference without a context type (less precise).
  • If more than one method matches the argument types, which one is chosen (if any). The simplest choice is to say that it's a compile-time error if the invocation is ambiguous. We can try to define when one is "more specific" than another, which we already do for extension methods, but if we don't, it means that we need to provide some way to disambiguate. Possibly you can just do that by using as X on each argument until it precisely matches one of the declarations. If that's possible ...
  • Declarations must have different parameter types. We can define "different" as "not mutual subtypes", and require at least one parameter to be different (the operator is []= has two arguments). That still does not guarantee that the two operators are distinguishable when called, if one of the types is a type parameter. Consider a class class C<T> { operator +(int x) => ...; operator+(T x) => ...; }. Those two + operators have different types, but when doing C<int>() + 4, it's still ambiguous. Which again suggests that what we really need is a way to disambiguate at the call point, and as is not enough in this example.

@chloestefantsova working on an inference implementation that can say "x" resolves to "x1" or "x2" and "y" is inferred to be of type "y1" or "y2" which then leads to resolution later. This should address issue 1 and possibly issue 2 as well, deferring decision making until later. If then it turns out to be really ambiguous, we can issue an error. @chloestefantsova Would your new algorithm address the above concerns?

One possible issue with this approach as discussed with @johnniwinther is that it could lead to an exponential amount of un-chosen possibilities when large expressions with many operators have many overloads. @chloestefantsova I don't remember exactly how your solution works. Is this an issue?