proposal: spec: operator functions

[deanveloper]

The only issue that I could find about operator overloading currently #19770, although it's currently closed and doesn't have many details.

Goal

Operator overloading should be used to create datatypes that represent things that already exist in Go. They should not represent anything else, and should ideally have no other function.

New operators should not be allowed to defined, we should only be able to define currently existing operators. If you look at languages that let you define your own operators (Scala, looking at you!) the code becomes extremely messy and hard to read. It almost requires an IDE because operator overloading is very heavily used.

Using an operator for anything other than it's purpose is a bad idea. We should not be making data structures with fancy looking + operators to add things to the structure. Overloading the + operator in Go should only be used for defining addition/concatenation operators for non-builtin types. Also, as per Go spec, binary operators should only be used for operating on two values of the same type.

Operators should only operate on and return a single type. This keeps things consistent with how operators currently work in Go. We shouldn't allow any type1 + string -> type1 stuff.

Operators should only be defined in the same package as the type they are defined on. Same rule as methods. You can't define methods for structs outside your package, and you shouldn't be able to do this with operators either.

And last but not least, operators should never mutate their operands. This should be a contract that should be listed in the Go spec. This makes operator functions predictable, which is how they should be.

Unary operators should not need to be overloaded.

+x                          is 0 + x
-x    negation              is 0 - x
^x    bitwise complement    is m ^ x  with m = "all bits set to 1" for unsigned x
                                      and  m = -1 for signed x

This part of the spec should always remain true, and should also remain true for anything using these operators. Perhaps ^x may need to have it's own, as there's no good way to define "all bits set to 1" for an arbitrary type, although defining a .Invert() function is no less readable IMO.

Unary operations on structs would then therefore be Type{} + t or Type{} - t, and pointers would be nil + t and nil - t. These may have to be special cases in the implementation of operator functions on pointers to types.

Assignment operators should also never be overloaded.

An assignment operation x op= y where op is a binary arithmetic operator is equivalent to x = x op (y) but evaluates x only once. The op= construct is a single token.

This should remain the same just as unary operators.

If we do not permit overloading the ^x unary operator, this means that we only need to define binary operations.

Issues/Projects aided by operator overloading

19787 - Decimal floating point (IEEE 754-2008)

26699 - Same proposal, more detail

19623 - Changing int to be arbitrary precision

9455 - Adding int128 and uint128

this code - Seriously it's gross
really anything that uses math/big that isn't micro-optimized If I went searching for longer, there'd probably be a few more that pop up

Syntax

What's a proposal without proposed syntaxes?

// A modular integer.
type Int struct {
    Val int
    Mod int
}

// ==============
// each of the functions would have the following function body:
//
//      if a == Int{} { // handle unary +
//      a.Mod = b.Mod
//  }
//
//  checkMod(a, b)
//  nextInt = Int{Val: a.Val + b.Val, Mod: a.Mod}
//  nextInt.Reduce()
//
//  return nextInt
//
// ==============

// In all of these, it would result in a compile error if the types of the arguments
// do not match each other and the return type.

// My new favorite. This makes for a simple grammar. It allows
// people who prefer function calls can instead use the `Add` function.
operator (Int + Int) Add
func Add(a, b Int) Int { ... }

// My old favorite. Abandoning the `func` construct clarifies
// that these should not be used like a standard function, and is much
// more clear that all arguments and the return type must be equal.
op(Int) (a + b) { ... }
operator(Int) (a + b) { ... }      // <- I like this better

// My old second favorite, although this looks a lot like a standard method definition.
// Maybe a good thing?
func (a + b Int) Int { ... }

// It can be fixed with adding an "op" to signify it's an operator function, although
// I do not like it because it just reads wrong. Also, looks like we're defining a package-level
// function named `op`... which is not what we are doing.
func op (a + b Int) Int { ... }

// Although at the same time, I don't like having words
// before the `func`... I feel that all function declarations should begin with `func`
op func (a + b Int) Int { ... }

// Another idea could just be to define a method named "Plus", although this
// would cause confusion between functions like `big.Int.Plus` vs `big.Int.Add`.
// We probably need to preserve `big.Int.Add` for microoptimization purposes.
func (a Int) Plus(b Int) Int { ... }

Considering other languages' implementations.

C++

// there's several ways to declare, but we'll use this one
Type operator+(Type a, Type b)

I think C++ isn't a bad language, but there are a lot of new programmers who use it and think it's "super cool" to implement operator functions for everything they make, including stuff like overloading the = operator (which I have seen before).

I also have a couple friends from college who really enjoyed defining operator functions for everything... no bueno.

It gives too much power to the programmer to do everything that they want to do. Doing this creates messy code.

Swift

static func +(a: Type, b: Type) -> Type

Note that custom operators may be defined, and you can define stuff like the operator's precedence. I have not looked much into how these operators end up being used, though.

C

public static Type operator+ (Type a, Type b)

Operator functions in C# end up being massively overused in my experience. People define all of the operators for all of their data structures. Might just be a consequence of using a language with many features, though.

Kotlin

operator fun plus(b: Type): Type // use "this" for left hand side

https://kotlinlang.org/docs/reference/operator-overloading.html, actually a really nice read.

Operator functions get used everywhere, and even the standard library is littered with them. Using them leads to unreadable code. For instance, what does mutableList + elem mean? Does it mutate mutableList? Does it return a new MutableList instance? No one knows without looking at the documentation.

Also, defining it as a method instead of a separate function just begs it to mutate this. We do not want to encourage this.

Open Questions

Which operators should be allowed to be overridden?

So, the mathematical operators +, -, *, /, % are pretty clear that if this gets implemented, we'd want to overload these operators.

List of remaining operators that should be considered for overloading:

• << and >> (I do not think this is a good idea)
• |, &, and &^ (also do not think this is a good idea)
• <, >, <=, >=, ==, != (maybe a good idea?)
  • If we include <, we should include == to prevent the confusing case of x <= y && y >= x but x != y.

Overloading equality may be a good thing. big.Int suffers because the only way to test equality is with a.Cmp(b) == 0 which is not readable at all.

I have left out || and && because they should be reserved exclusively for bool or types based on bool (has anyone ever even based a type on bool?) and see no reason to override them.

Should we even allow operator overloading on pointer types?

Allowing operator overloading on a pointer type means the possibility of mutating, which we do not want. On the other hand, allowing pointer types means less copying, especially for large structures such as matrices. This question would be resolved if the read only types proposal is accepted.

Disallowing pointer types

• Does not allow mutation
• No need for nil checks in operator implementation

Allowing pointer types

• Allows operators to be consistent with the rest of the type's methods.
  • ie *big.Int is *big.Int everywhere else, it would be good for consistiency
• Since it's consistent, it makes it easier to pass into other functions.
  • ie Can't pass big.Int into a function that takes *big.Int

Perhaps it should be a compile-time error to mutate a pointer in an operator function. If read-only types were added then we could require the parameters to be read-only.

Should it reference/dereference as needed?

Methods currently do this with their receivers. For instance:

// Because the receiver for `NewInt` is `*big.Int`,
// the second line is equivalent to `(&num).NewInt(....)`

var num big.Int
num.NewInt(5000000000000000000)

So should the same logic apply to operator functions?

---

I'm aware that this will probably not be added to Go 2, but I figured it would be a good thing to make an issue for, since the current issue for Operator Functions is quite small and, well, it's closed.

---

Edits:

• Added #9455 to list of proposals which could be solved using this instead
• Fixed a clarity issue
• Changed examples to use a modular integer rather than a *big.Int since the math/big package is designed to be used in an efficient way, and added that read-only types would benefit this proposal

[deanveloper]

Correct. The intended use case (for this proposal at least) is not for large matrices or anything like that. While I would like to improve this proposal to allow it, I can’t think of a good way of doing it without making it not feel like Go.

Possibly something along the lines of:

package big // import “math/big”

// just for example, the more obvious statement here would be `operator (*Int + *Int) Add`
operator (*Int + *Int) addOp

// i is the memory address of the assigned variable
// a and b are the operands
func (i *Int) addOp(a, b *Int) {
    i.Add(a, b)
}

// usage:
// var sum big.Int
// sum = &sum + big.NewInt(100)
// var add = big.NewInt(6)
// sum = &sum + add

This way we can reuse pointers and avoid copying. I think the cognitive overhead might be a bit high though. Also comes at the risk of allowing a and b in addOp to be mutated (intentionally or unintentionally)

This also works great for mutable structures, but quite terribly for immutable structures.

[cosmos72]

I wrote a proposal some time ago - it should still be somewhere in this thread - about operator overload with receiver used as (mutable) result. The idea was something like

package big

operator +(*Int, *Int) = (*Int).Add

// Add computes x + y and stores result in z. also returns z.
func (z *Int) Add(x *Int, y *Int) *Int {
    // ...
    return z
}

and then teach the compiler that it should translate

var a, b ,c *big.Int
fmt.Println(a .+ b)
fmt.Println(a .+ b .+ c)

into

var a, b ,c *big.Int
fmt.Println(  new(big.Int).Add(a, b) )
fmt.Println(  new(big.Int).Add(a, b).Add(c) )

i.e. to inject compiler-generated temporaries to store the results, and reuse them where possible.

Definitely more complicated to implement than your first proposal, but more efficient at runtime. And hopefully more readable than your second proposal.

[UPDATE] It also allows programmers to choose between

• "quick and easy" syntax with "overloaded" operators, if readability is most important and performance comes second (but it will still be better than what's possible with immutable operands):

  var a, b, c *big.Int
  a .+ b .+ c

• existing syntax with method calls, if performance is critical and readability comes second:

  var a, b, c *big.Int
  var result big.Int
  result.Add(a, b).Add(c)

[tc-hib]

This may be at least partly remedied by drawing attention to the parallels between normal methods and the operators.

I'm not sure the parallel with method calls is such a good thing: x .+ y .* z would be equivalent to x.Add(y.Mul(z)) while x .+ y .+ z would be equivalent to x.Add(y).Add(z)

I disagree with the original proposal, in that we should not assume what users need with regards to operators available for overloading; outside of the Go-sphere, symbols have any number of different uses and interpretations. An example I like to use is a Lua library called LPeg, which uses operator overloading to create something entirely new by reusing existing syntax.

I think this is exactly what we should try to avoid. Except for + on strings, arithmetic operators should not lose their meaning, IMO.

There's probably a widespread need for "real world" types such as decimal types ("big decimal", fixed point, floating point, or just bigger ints). This is what we (humans) read and write. We have them in UIs, SQL databases, literals in our own code, and, most importantly, our paper notes. These types would naturally use the same operators as int and float, with same precedence rules and same meaning. (talking about + - * / % and all comparisons)

Vectors and matrices are out of scope IMHO.

I also think we don't need mutable receivers. I would be more than happy if I had a decimal128 with operators but had to use method calls on a big decimal type, just like big.Int Vs. int64. And the cost of copying a decimal128 would rarely matter.

[deanveloper]

While I like the idea, it gets a little more complicated... first of all with operator +(...) = (*Int).Add

The Add method both returns a value and mutates its receiver (which it also returns). This causes a couple complications when we aren’t talking about math/big. For instance, if the returned value is different than the receiver, then we run into a predicament with translating a = b .+ c. Instead, I propose the “signature” of operator would look like operator (T <op> T) = func (*T, T, T).

To get this to work with pointer arguments, you would use a double-pointer as the first argument. This is because the operator function needs to be able to assign to the lvalue, which is a pointer. This would allow flexibility between usages for value-types and pointer-types being used as operands.

operator (*Int + *Int) = func (lvalue **Int, a, b *Int) {
    (*lvalue).Add(a, b)
}

The signature is a bit more confusing for pointer types... although I think that operators for pointer types should be pretty niche. Also it’s not super confusing when essentially all it means is “we allow ourselves to change where an lvalue points to”.

Usage on the “calling” side of the operators remains the same and much less confusing:

var a, b, c *big.Int

a = b + c

fmt.Println(a + b + c)

translates to

var a, b, c *big.Int

a.+(b, c)

var _temp = new(big.Int)
_temp.+(a, b)
_temp.+(_temp, c)
fmt.Println(_temp)

Typed this on a phone, apologies for any typos

[cosmos72]

I see a lot of "we don't need this (implicit: for the use I have in mind)" and very few general proposals that try to cover all reasonable cases where other languages use operators or operator overloading, namely:

• arithmetic on arbitrary precision numbers
• arithmetic on bound-checked or saturating numbers
• arithmetic on elements of finite fields
• arithmetic on vectors and matrices
• arithmetic on quaternions, octonions...

and surely there are more reasonable cases I do not know about.

Some of them require large and variable-sized representations, which means memory allocation and usually involves pointers. Some of them will/would be used in situations where performance matters, possibly a lot. Some other will be used in situations where ease of writing the code (and source code readability) is instead the most important consideration.

Yes, it's possible to write a proposal that intentionally addresses a subset of the use cases. But that would close the door for a very long time to the use cases that are left out - so in my opinion the author would better have (and describe) a very good reason to leave out such use cases.

That's why I came up with a proposal that internally uses explicit receivers, and modifies and returns them: the syntax and readability of overloaded operators is not changed, but internally it allows for a more efficient implementation, especially if the things being added/multiplied/etc. are large and may need to allocate memory.

Surely, it's more complicated to implement - both inside the compiler and for programmers that want to add operator overloading to their types - but it's not more complicated to use: one still writes a + b + c or a .+ b .+ c or whatever the chosen syntax will be.

I wrote my opinion, and I will now stop flooding this thread :)

[mikelward]

Operators should only operate on and return a single type

It would be nice to be able to write e.g.

time.Now() + time.Hour

As sugar for

time.Now().Add(time.Hour)

[deanveloper]

While I somewhat agree that it looks nice, by the communative property of addition, we would also infer that time.Hour + time.Now() would be valid. That would either translate to something like time.Hour.Add(time.Now()) which doesn't make sense, or time.Now().Add(time.Hour), which is confusing since it's out of order. Or we could throw a compiler error, but that'd mean that addition is no longer communative, which may also be confusing.

Also, while it might look good for times, what about the following:

var i pkg.Uint
var j pkg.Int

k := (i + j*i) / j

If they were all the same type, this would make a lot of sense. However, because they are different types, it isn't clear what is going on here. What is the type of k? the type of (i + j*i)? What if j is negative, but the numerator resolves as pkg.Uint? Since the type of each expression is not clear, how do we address overflow/underflow bugs?

Not to mention, it's not going to be easy to separate the values:

k := j*i
k = i + k // possible compiler error
k = k / j // another possible compiler error

Moral of the story, which we have learned from several languages in the past: we should not allow operators over different types. It gets extremely confusing.

[evrimoztamur]

@deanveloper I don't necessarily agree with that. In no language that supports operator overloading is there a guarantee for commutative operators. Even a basic operator like * can have different meanings depending on whether it's multiplying numbers, vectors, or matrices; not all of which are commutative.

What should on the other hand be clear is that time.Hour is a Duration and time.Now() is a Time instance. The standard library already makes it clear that it is ordered such, and not vice versa.

All in all, so as long as people using Go know that operators can be overloaded, you should let the compiler/language server let you know that that's not the way you add Time and Duration together.

[DeedleFake]

@deanveloper:

To illustrate @evrimoztamur's point, there's already precedent for non-commutative operators in Go: String concatenation. "a" + "b" is very much not the same thing as "b" + "a". I see no particular reason that user-created operator meanings would have to follow that rule, either.

That being said, I agree that operators over differing types would be a bad idea. I'm not particularly in favor of operator overloading in general, but if it was added I would want it added in a limited capacity. User-defined operator overloads should get no special privileges over the built-in ones in my opinion, and outside of interfaces, which are very different from all other types in Go, I can't think of any regular binary operator that takes multiple types simultaneously.

[radonnachie]

Am I understanding the gist of the resistance to this point correctly:

The community is against the change because of the complexity and confusion of implicit type checking and the unbounded depth that could reach in users hands. The users expect that they'd be able to write multiple overloads for the various combinations that the infix notation could be called upon, or write a generic subset that would require some kind of hierarchical type inheritance and smarts in order for the expected types to be inferred etc.

Couldn't we do away with the majority of the complexity, and inhibit any confusion, by keeping the language's limit to a single function definition (no actual overloading), and just implementing the syntactic replacement (a + b to a.Add(b)). Thusly the one definition (specified by the infix arguments' types) would have a clear return type and definition. The intermediate values would be explicated, the function definition singular, and the sugar gained. Complex chains of the infix sugar would possibly need inline conversions or field-referencing, but that's just upholding the specificity criterion of clear code.

This might only cut back on the expectations of the users, but this limited/restricted option seems like it would be inline with the community's goals for the language. Happy to learn from your responses!

[maj-o]

It's not the same. 1*2+3=6 1.mul(2.add(3)).=5 Main reason for this is to have a decimal type. Perfectly arbitrary precision decimals, search.

[zephyrtronium]

@RocketRoss

This might only cut back on the expectations of the users, but this limited/restricted option seems like it would be inline with the community's goals for the language. Happy to learn from your responses!

Generally, part of the requirement for language changes is that they must address the needs of everyone who has the problem. (This is why all of the error handling proposals to date have been rejected, for example.) Limiting user operators to a single argument type does not meet the needs of people who want them for linear algebra, as a broad use case, because it does not allow one to distinguish between matrix and scalar multiplication. So, I don't think this approach can be considered acceptable.

[Efruit]

Generally, part of the requirement for language changes is that they must the needs of everyone who has the problem.

Doesn't that make this problem intractable? Some participants (in both error handling and operator overloading) have mutually exclusive needs.

1. The folks doing linear algebra may want to define signatures for all manners of different matrix shapes and operand types.
2. Those looking to write vector operations may want to define totally new operators i.e. for dot or cross products.
3. Those who want code to be predictable may want only one operator signature per type, all following predictable integer semantics.
4. Those who don't want the + operator to be able to panic may not want any of this at all.
5. And I want to define DSLs using operator overloading in unusual (some would say "wrong") ways.

Each and every one of these is a valid viewpoint that's been presented in this thread. 1, 2, and 5 conflict with 3 and 4. 2 even conflicts with the language design, as infix operators are limited, and changing that would likely require major language specification, lexer, and parser changes.

[radonnachie]

@maj-o 12+3 is always 5. But taking your point as `21+3`, the operator substitution would follow the operator precedence, avoiding any increase in confusion: playground

2*1+3 = 5
2.Mul(1).Add(3) = 5
2.Mul(1.Add(3)) = 8 // this would not be the result of operator substitution following precedence
2+1*3 = 5
2.Add(1.Mul(3)) = 5
2.Add(1).Mul(3) = 9 // this would not be the result of operator substitution following precedence

Regarding the intractability.. I'm wondering if the following reduced implementation wouldn't be just a minor lexer change, and less so a language alteration, hopefully reducing the number of benefactors' boxes that need to be checked for justification:

Let's start with the most basic extension to Go-Lang towards achieving something in this direction:

• 1️⃣ (Mathematical?) operators (a subset at least?) between user-defined types (particularly structs?) are replaced with a corresponding named function (+ -> .Add etc. and even +=->&.Add).
• 2️⃣ The replacement follows the operator precedence.
• 3️⃣ It would be expected that the named function return a value (of whatever type the author deems reasonable), so that chains of operators are applicable (except in inappropriate substitutions such as +=).

We would not be considering:

• ❗ 1️⃣ adding carte-blanche for new operator definitions (tough on the dot vs cross product, but I imagine some inline conversion to a wrapper struct, with the alternative .Mul definition, could be a more suitable work around than the current named functions, see the attached playground)
• ❗ 2️⃣ implementing function overloading (cannot define T.Add(U) and T.Add(V)... I imagine some more expansive sugar substitution would handle T.AddU(U), T.AddV(V) etc...). Thus all operator substitution candidates that are created will likely have homogenous operands, and be scoped to the author's source code.

I don't see any room for ambiguity arising so far, and I do see a simple improvement to the language being achieved: time.Now() + time.Hour and similarly for user-defined types that are purposed for math or numerical analysis...

What am I missing in the considerations? I don't mind a hard no if I can understand why not. I'm viewing this as a lexer capability that is on par with the (&v). substitution to invoke pointer-receiver definitions. Am I wrong? Or is this interpretation sufficiently different to be a different issue, given I'm not requesting any operator nor function overloading?

[cosmos72]

One thing missing from your proposal is consideration of memory allocations:

if a and b are types that contain heap-allocated data, for example because they are vectors, matrices or arbitrary precision numbers (see math/big),

then translating a+b into a.Add(b) means the returned value will have to be created from scratch, including allocation of its data in the heap.

That's why types in math/big define arithmetic operations differently: z = a + b is written as

z.Add(a, b)

which stores the result into an existing z: if its internal heap-allocated data is large enough, it will be reused, avoiding any allocation.

There's also a general principle for it: receive buffers from the caller and use them (write your result there), do not allocate them yourself and return them.

[radonnachie]

Nice point... Let me know if my interpretation is off:

Pointers are the perfect mechanism for specifying 'buffers' to the callee. Indeed my understanding is that is their sole effective purpose.

a+b written anywhere needs to occupy some new memory, right? if a+b > x etc, so the z.Add(a, b) seems quite gross to me, unless it's more specifically to minimise allocations during chained additions, e.g. z.Add(a, b).Add(c).. I mean is that allowed? Surely not because that would be overloaded... I don't think I'm understanding the z.Add(a, b) design decision. I also think that z := a + b is far more in line with Go than var z Big; z.Add(a, b)...? (not trying to derail to conversation on Big's decisions, just hoping to learn from reactions)

❓ 1️⃣ In critical positions, one could exercise heap pedanticism with z := Big(0); z += a; z += b; z += c;. There asides maybe this is the point where some more smarts need to come to play (i.e. I concede): it would be nice to have a + b + c not produce 2 allocations, but instead to have the second addition be more of a += operation on the first allocation. I'm not sure of the contemporary approach to this, but surely other languages have either mitigated it or deemed it acceptable.

❓ 2️⃣ Taking a page out of math/bigs implementation.. Perhaps writing the named functions cleverly would allow a solution to come up: func (z Int) Add(x Int) *Int could be chained explicitly to func (z *Int) Add(x Int) *Int, where the former returns a new allocation and the latter operates on the pointer received... it means a + b + c would create one new allocation, with the + c invoking the pointer receiving Add... But that is overloading.... As a concession (only differentiate between pointer and value receiver functions) it'd be quite powerful... It'd also open up function reuse for functional-programing paradigms.. Maybe that's an addendum to this proposal, because I see the heap-allocation control as a separate request...

I'm feeling like branching to a new proposal-issue...

[ianlancetaylor]

z.Add(a, b) may be gross, but we got there because experience showed us that memory allocation is the dominant cost of using types like big.Int. People writing efficient code with math/big have to be able to control the memory allocation of every operation in order to avoid unnecessary allocations.

This is definitely a concern with operator overloading, if we want to be able to use operators with types like big.Int. And since big.Int is more or less a poster child for operator overloading, I think that some solution is required. That solution could perhaps mean writing down rules for how the compiler is required to combine assignment with operators. I'm not sure.

[radonnachie]

@ianlancetaylor thanks for patiently reiterating the importance of allocation control. I see that z.Add(z, a) works as z += a, so I can understand the decision. Would you mind confirming that ❓ 1️⃣ z := big.NewInt(0); z += a; z += b; z += c; //etc would be quite capable of controlling the allocation? This wouldn't require any function overloading implementations. The second point, ❓ 2️⃣ , requires only pointer and value receiver function overloading, again I think a separate point of discussion and would be implemented orthogonally.

If there is enough credence in the combination of these two ideas, or too much backlog in this issue as it stands, then I'll happily start 2 new proposal-issues that reference this. Just looking for some confirmation before doing so.

[cosmos72]

@ianlancetaylor can contradict me if he wants,

but I think the three-argument methods to perform arithmetic on arbitrary types (z.Add(a,b) etc.) are really the only reasonable way to go. Self-assignment operators z += a; z *= b; z /= c can keep allocations quite under control (question: how do you rewrite z = a / z ?), but fail short of the (in my opinion) first main goal of operator overloading:

1. source readability and expressiveness: write simple, naturally-looking high-level source using operators, and let the compiler figure out the details

In other words, if operator overloading is ever implemented in Go, it would really better allow writing things like

z = a + b * c / d

The second goal (still in my opinion) of operator overloading is:

2. Go toolchain should compile such high-level source to reasonably efficient machine code - i.e. something that minimizes heap-allocation of temporaries, not code that heap-allocates one temporary for each operation.

Some time ago I wrote in this thread an analysis about how to reach both these goals at once. It's somewhat complicated but feasible - it starts from three-argument methods z.Add(a, b) and analyzes what the compiler must do to minimize the number of temporaries.

I'm afraid anything simpler than that will fall short in at least one of these two (somewhat competing) goals. i.e. source readability/expressiveness and runtime efficiency.

[win-t]

Hi all, because we now have generic, can we utilize it?

for example, we can define + and * operator as built-in function

func OpAdd[T interface{ OpAdd(T) T }](a, b T) T { return a.OpAdd(b) }
func OpMul[T interface{ OpMul(T) T }](a, b T) T { return a.OpMul(b) }

and also add OpAdd and OpMul method to basic types like

func (a int) OpAdd(b int) int { /* compiler intrinsic of a + b to avoid recursive call */ }
func (a int) OpMul(b int) int { /* compiler intrinsic of a * b to avoid recursive call */ }

and then we can convert the expression

a * b + c * d

into

OpAdd(OpMul(a, b), OpMul(c, d))

with that in place, we can easily implement an operator for a custom type like this one https://go.dev/play/p/h_SVp8-Gjl8

In summary:

• adding built-in function Op*** into language spec
• adding built-in method Op*** into language spec for basic type
• we need to change the compiler to convert the normal binary expressions into function calls into the built-in Op*** function (honoring the operator precedence)
• we need to change the compiler to generate all the built-in Op*** methods to the basic types

[cosmos72]

This, or something more or less similar, can be implemented without language changes now that we have generics.

The language changes are "only" needed to be able to write

a * b + c * d

instead of the more cumbersome (but explicit)

OpAdd(OpMul(a, b), OpMul(c ,d))

(although I'd still recommend three-argument operations, with the receiver being used to store the result, to avoid unnecessary memory allocations for large objects).

Clearly, this issue is a proposal for operator functions, so the OpAdd, OpMul and friends alone would not address it.

[DeedleFake]

It has been fairly well established that the Go developers do not want general operator overloading in the language, and I'm inclined to agree with the reasoning given, namely that it can disguise an expensive operation as though it was just addition or something, and there's no way to know what it's really doing at the call-site anymore. With the exception of string + string, all operators in Go currently are O(1), which is actually very nice. While operator overloading could certainly be convenient in some situations, such as math/big, I don't think that it's worth the complication that it introduces.

Maybe infix functions would be a better approach, so that you could do a plus b. I don't think that it really gains much over a.Plus(b), though.

[win-t]

Maybe infix functions would be a better approach, so that you could do a plus b. I don't think that it really gains much over a.Plus(b), though.

what do you suggest to handle precedence @DeedleFake ?

[DeedleFake]

It was just a random thought, not an actual suggestion, but probably just simple left-to-right would be the least surprising.

[win-t]

(although I'd still recommend three-argument operations, with the receiver being used to store the result, to avoid unnecessary memory allocations for large objects).

@cosmos72 I think for big.Int, we would only have Op***Assign family (with pointer receiver), but not OpAdd or OpMul (with non-pointer receiver)

EDIT: after some thinking, I agree we should provide three-argument operator like @cosmos72 said, so, provided big.Int with OpAdd and OpAddStore method, the compiler can use something like SSA analysis to know if a variable is only used once or not, if so, the compiler use OpAddStore for + operator, otherwise it uses OpAdd. but yes, as @DeedleFake said, you can't distinguish whether some operator is O(1) or not at call-site

[ncruces]

If being O(1) is an issue, we should probably remove operators from floats, as I'm pretty sure most people are not familiar with the fact that operations on some floats can be more than two orders of magnitude slower that operations on other floats.

Operator overloading, if added, will be abused. Not adding it and also not making types that'd benefit greatly from it (decimal, vec3, mat4, quaternion, big.Int…) native, so that they can use operators without overloading, just makes Go less useful in fields that need those types.

There's no avoiding the trade off.

[win-t]

many say that operator overloading will be abused to the point that the code is hard to understand. but from what I learned from Haskell (I'm not saying I'm an expert on it), as long they form semigroup (in other words, the types must always be the same), I don't see how it will be abused. so *T will only be addable to another *T but not T, and vice versa T is only addable to another T but not *T operator overloading in other languages like C++, where you can define multiple of it and the types are not the same, of course, will make the code harder to understand can someone give an example of how it is abused?

[ianlancetaylor]

In Go today a * b always executes in roughly constant time, never allocates memory, and never panics. If we permit operator overloading none of those will be the case.

[soypat]

My understanding is that it is common to prohibit the use of operator overloading on large projects where code complexity is a concern, for example in game development (allowed for simple implementations) and mars rover environments (strict prohibition).

My experience: I have worked on several numerical libraries that deal with quat, vector and matrix operations, the whole lot of it. There has been only one case where I regretted the fact there was no operator overloading in Go, it was when I had to work with the big.Int type. This was early on in my Go career and today I think the reason I felt this was method API design of the big package. Having functions for operations instead of methods does so much more for readability.

So I'd rather not have operator overloading. It seems to be a language feature avoided by those who practice software engineering and I don't have a need for it to develop numerical libraries. There are other more pressing issues regarding Go in a numerical setting than the lack of syntactic sugar.

[duaneking]

All I want is generic support in operator overloading, with full support for all operations, with generics support.

Let me overload any operator that is supported; Type constraints as I have seen them in go don't define these well enough because we are missing critical binary and logical operations, and are not able to do things like using constraints.Unsigned to write generic code that leverages operator overloading for binary operations.

Let me do that!

Let me program to interfaces so that I can write more dry and solid code by design.

[y1yang0]

operator overloading introduces a huge complexity, either in compiler implementation and code maintaibility, which is also the root of the expansion of C++ complexity IMHO.

[duaneking]

I would love to be able to write code that can simply be leveraged with type sets that I've previously defined. Let me use the composition of objects to define what they are.

You can't tell me that you want to enable composition over inheritance as a core design constraint, and then tell me that I can't compose objects without begging the question if that violates the design by making custom composition harder. Not to mention, it seems kind of contradictory.. and unfair. The fact is, while operator overloading for things like (+) and (-) might seem odd, for distinct sets of data and various models in some very distinct domains, it makes a lot of sense and enables separation of concerns and DRYing that leads to cleaner code.

For example, I should be able to add a Money type to another Money type with (a + b). I'm possibly going to have to make sure the two currencies are the same, or I can enable functionality around exchange rates through. But the fact remains, no matter what I choose to do, it's kind of important that I have full control... if only for compliance and auditing.

I fully understand that people shouldn't be messing with operators in ways that could create bad code; But the community can't even agree on what that is half the time. I'm sure we can all point to tech that just seems wrong, but trying to design against bad code will only get you so far. And in many ways, it'll force people to create bad code to get around the artificial constraint that you imposed with good intentions that got in their way while they were just trying to do their jobs.

All I'm asking is that, if golang is truly about composition, that we truly go into and enable composition. Let me compose my objects. Not just to define the methods that are a part of an interface, let me define the operators that must be enabled for the interfaces used in the process/strategy I have defined, because they are effectively just function calls that the language is not letting me override right now.

The current solution only enables half of the problem; Operator overloading and operator functions would enable solutions for the other half and truly allow people to Go,

[StephanVerbeeck]

That operator overloading will never happen in GO was one of the very first design strategies. If you as GO user feel that you are missing something then you can not resist the temptation to produce unreadable code (and should switch to one of the plenty languages in which such is possible and customary). Sorry I don't want to be rude but operator overloading was one of the worst mistakes in computer history. Operator overloading HIDES function calls (these are not visible in the source code) causing code that can only be read by those who created the program. It is like hiding easter-egg functionality in a program but at a large scale. I'm a senior programmer with 40 years of experience in about any programming language and system that exists/existed including multiple self created scripting languages and I'm also the person who ported Python to the VAX/VMS platform. Just take my word for it, operator overloading in GO may never happen and that is one of the foundations of GO. This ongoing conversation ONLY exists to prevent future users from creating duplicate proposals here.

[duaneking]

I apologize, but as respectfully as possible, I do not believe that opinion alone is worthy of accepting as fact. I'm bringing data and examples and asking questions; people are replying with insults and accusations.

It's fine for people to have opinions, and I welcome opinions with data to back them up, but we should be willing to admit when we don't have data to back them up.

I'm actually a big fan of DRY and SOLID and personally enforce these in my projects; So any incorrect assertion that this would generate bad code 100% of the time is flat out wrong. I gave a great example above: and I explicitly defined the concerned boundaries. I respectfully understand that if some disagree with me, that's one thing, But you shouldn't be making random arguments without data either, because with all respect, I'm bringing examples and data to you... and it feels like some of you are replying with opinion and insults.

Where is the data?

[aamironline]

Only a small percentage of the math/big api can be replaced with the small set of binary operators defined in the language.

Looks like you have not tired the complex expressions!

[duaneking]

Given the power of the fmt.Stringer interface in golang, its important to consider that effectively this is another perfect example of why you would want type conversion that you can inject your own code into as a form of operator overloading; in this case it's simply of a string composed of a Strategy taking the prior typed instance as input and returning a string.

What's important here is that we can inject that strategy ourselves. The value is that we have control over the code in the resulting callback; and the fact that it's used by so many things grants it even more utility.

[Ralf-Heete]

I propose a comprehensive package of enhancements for Go, which includes operator overloading, lambda functions, as well as getter and setter methods similar to C#.

In order to control potential side effects - a prevalent concern with these features - my suggestion is to implement operator overloading, and getter/setter methods through lambda functions in Go. These lambda functions should only allow conditional expressions, effectively preventing side effects. This approach could provide a layer of control not currently available in Python due to possible side effects with lambda functions.

Furthermore, within these lambda functions in Go, I would suggest disallowing function calls. We should only permit further lambda expressions and constructors. This approach would ensure the purity of the lambda function and maintain a level of simplicity and straightforwardness in the code, which is in line with Go's philosophy.

This comprehensive package could open up new patterns and capabilities in Go, while effectively managing the risk of side effects. Each component must be carefully considered and implemented to ensure consistency with the existing language philosophy and objectives.

I understand that any change to the language needs to be meticulously planned and executed considering its potential impact on existing code and the learning curve for the community. However, these are my initial thoughts and I'm eager to hear feedback and insights from the community on this proposal.

Example:

Python code:

python

     // Implement Add Operator with a lambda
      class MyClass:
               def __init__(self, value):
                    self.value = value

               __add__ = lambda self, other: (
                            (lambda: NotImplemented),
                            (lambda: MyClass(self.value + other.value))
                         )[isinstance(other, MyClass)]()

Equivalent Go code (hypothetical, as Go does not currently support operator overloading or lambda-style functions): go

 type MyType struct {
         value int
 }

 lambda (self MyType) __add__ (arg MyType) MyType {
       return MyType{self.value + arg.value} 
 }

Getter Proposal:

 type MyStructure struct {
           private int
 }

 // Define a getter for the private field
 getter (self MyStructure) Public int {
      return self.private
 }

 func foo(bar MyStructure) int { 
        return bar.Public
 }

in this proposal, Public is a getter function that acts like a public field, providing access to the private field private. You would access it directly, as if it were a public field (bar.Public), rather than using the usual method call syntax (bar.GetPrivate()).

Setter Proposal:

type MyStructure struct {
    private int
}

// Defining a setter for the private field 
setter (self *MyStructure) Public(value int) {
    if self.private == 1 {
        self.private = value
    }
}

func foo(bar *MyStructure, value int) { bar.Public = value }

In this case, Public is a setter method that you're using like a field, to set the value of the underlying private field.

[Ralf-Heete]

That operator overloading will never happen in GO was one of the very first design strategies. If you as GO user feel that you are missing something then you can not resist the temptation to produce unreadable code (and should switch to one of the plenty languages in which such is possible and customary). Sorry I don't want to be rude but operator overloading was one of the worst mistakes in computer history. Operator overloading HIDES function calls (these are not visible in the source code) causing code that can only be read by those who created the program. It is like hiding easter-egg functionality in a program but at a large scale. I'm a senior programmer with 40 years of experience in about any programming language and system that exists/existed including multiple self created scripting languages and I'm also the person who ported Python to the VAX/VMS platform. Just take my word for it, operator overloading in GO may never happen and that is one of the foundations of GO. This ongoing conversation ONLY exists to prevent future users from creating duplicate proposals here.

Stephan Veenbeck presents some commonly expressed concerns about operator overloading: it can make the code difficult to read and hide its functions, leading to confusion and errors. These are valid concerns that should be considered in the discussion of potential enhancements for Go.

A potential approach to addressing these concerns might look like this:

Clarity and Readability: Operator overloading needs to be implemented in a way that maintains the readability and comprehensibility of the code. Careful design and clear guidelines could help prevent misuse and restrict the use of overloaded operators to logical and easily understandable ways.

Use Control: Operator overloading should be restricted to specific and appropriate use cases. A possible solution could be to allow it for certain types or in certain contexts to ensure that it does not unnecessarily complicate the code or hide what's actually happening.

Existing Practices in Other Languages: While operator overloading has caused problems in some languages, it is effectively utilized in others. It's worth examining the practices and experiences from these languages to identify possible problems and find solutions.

Community Feedback: It's important to consider the feedback and concerns of the community. A dialogue with users could help address concerns and lead to a solution that can leverage the benefits of operator overloading while also preserving the readability and comprehensibility of the code.

It's crucial to note that introducing such a change needs to be carefully thought out and planned to avoid unwanted side effects. Ultimately, any changes in the language should aim to enhance developer productivity and satisfaction without compromising the principles of simplicity and clarity that have made Go what it is.

[bkahlerventer]

There are a simpler solution to the operator implementation. Just allow more characters in the definition of the name of a function similar to what scala does it and allow for infix notation for single parameter functions and do not treat operators in any special way other than the way they are defined.

currently: type MyValue struct { value int }

// Defining a + for the private field func (r MyValue) add(v *MyValue) MyValue { return MyValue{r.value+v.value} }

use currently as:

val1.add(val2)

new:

type MyValue struct { value int }

// Defining a + for the private field func (r MyValue) +(v *MyValue) MyValue { return MyValue{r.value+v.value} }

// Defining a ++ for the private field func (r MyValue) ++() MyValue { return MyValue{r.value+1} }

use new as: val3 := val1.+(val2) // normal val3 := val1 + val2 // infix notation

this way anyone can add "new operators" such as:

~>() >>>() <--->() etc.

it will get interesting for the * and & operators for pointer references. Also := and = for assignment.

[chenyanchen]

There is two cases in my develop experiences.

1. Add (operate) two same type value.

This case is simplify for developer:

type Point struct {
    X, Y int
}

func (p Point) Add(other Point) Point {
    return Point{p.X + other.X, p.Y + other.Y}
}

func main() {
    p1 := Point{1, 2}
    p2 := Point{3, 4}
    fmt.Println(p1 + p2) // Output: {4 6}
}

2. Compare two same type value.

type Block struct {
    PrevHash []byte
    Hash     []byte
}

func (b Block) Compare(other Block) bool {
    return bytes.Compare(b.PrevHash, other.PrevHash) == 0 &&
        bytes.Compare(b.Hash, other.Hash) == 0
}

func main() {
    b1 := Block{[]byte("p0"), []byte("h1")}
    b2 := Block{[]byte("p1"), []byte("h2")}

    m := map[Block]struct{}{
        b1: struct{}{},
        b2: struct{}{},
    }
}

[rufreakde]

At least we could implement a custom optionalChaining operator even though this was dismissed with such an option. Would love to see custom operators They can be limited but having the ability would come a long way.

[Cyberhan123]

I'm a little worried about whether this will turn golang into another python. For example, if we customize a symbol like @ for matrix multiplication, this is equivalent to inventing another operation, and its behavior is confusing. What I mean is that only by consulting the documentation can you know the difference between it and *.

[win-t]

I think it is okay if the operator symbol is not overloaded with existing ones.

if you encounter a weird symbol like *. or +., it just means that you need to look at the docs, it is just the same when you encounter an unknown method name, you look up the docs.

an infix operator is just another form of method invocation, but it is more readable.

((a +. b) *. c) +. (d *. e)

vs

a.Add(b).Mul(c).Add(d.Mul(e))

you can know that if it is not a standard operator symbol, then it must be a method

NOTE: btw, it is just a matter of preference, I prefer the former, but someone maybe prefer the later

[exapsy]

I don't personally think this serves any good purpose that solves a specific problem. It's just syntactic sugar without making something more readable necessarily, neither serving any compilation or type-safety purpose.

From my understanding, which might be wrong, I think this proposal about operator functions comes from other languages that may have this functionality on top of them. But, why? What problem does it solve?

Okay, fair enough this example from @win-t is fair

((a +. b) *. c) +. (d *. e) vs a.Add(b).Mul(c).Add(d.Mul(e))

Although, I see a fair bit of prons and cons in this one where the cons are more than the pros

Pros

• Easier to write and read
• Easier to comprehend what this code is doing

Cons

• (in this specific example) The +. ... the dot (.) is simply just distracting. Now it makes it kinda even unreadable. For the sake of what? Having operator overloading? It can be fair, if operator overloading indeed serves a very good purpose, but let's go through that.
• When are you gonna actually ... do such big operation? Without encapsulating it inside a function or comments anyways? Like the example by itself, feels very-very rare case. It's an extreme example from my experience. I never ever had to see anybody write such big operation. Even in graphics programming, for which you usually do not use Go for, you're usually encapsulating such big operations into functions such that you don't need overloading, or the operation is already supported by the language. Or if you really need such big operation, probably Go is not what you need. Like, personally I wouldn't use Go for graphics programming and matrices. I would use some other language because of support for Arenas, very free use of allocators and such. Like, using the right tools for the right jobs.
• Give me a real life example, where you would actually use that? How frequent would that example be in real life? Why not just place such a big complex operation inside an object's function which also explains what it actually does? Is it so frequent that it becomes a struggle in everyday programming? Or is it a once in a year I may need that? And after all, it's just syntactic sugar. Not syntactic sugar like Assembly to C, or for i, n := range objects, but like a self-explanatory code being translated to literal keywords of the language itself. This breaks simplicity by nature. You have to know what those operators on those objects are actually doing. When with a function, its very easy to read the documentation or read the name of the function.
• Proposal of a feature that people may misuse for the sake of using it because "it's cool". It happens often. Hate me, but it happens often that people use features just because they exist. That what was the whole debate over generics anyways, wasn't it? For which I advocate for. But I agree that people may misuse generics. Difference is that generics have many real life examples and already the standard library starts using them because they are indeed useful.
• Proposal proposes a way to Add(), Mul() etc. an object, when the side-effects may not be apparent what they exactly do, and the user might expect something else from what is actually happening under the hood. People won't hover over +. to see what it does on these objects. But people will hover on what AddMatrix() does to ensure what it does. This offers safety and less bugs.
• Abstractions. Such specific abstractions are just purely syntactic sugar for the sake of making an abstraction. Yes sure, because some people may feel comfortable from other languages. But what is their purpose? What do they solve? When? How often?
• This ((a +. b) *. c) +. (d *. e) makes sense when you have Integers or any numerical types. This a.Add(b).Mul(c).Add(d.Mul(e)) makes sense when you have structs where you don't actually know what an operator actually may actually do and what side-effects it has.
• +, - * all these serve a very specific purpose in a language. And by overloading those operators, you're breaking that purpose. Now you're start doubting what an operator actually does. You're breaking a social contract that "hey I'm sure what these operators are doing". No, now you're never sure. You have to look if everything's an integer, or a float, or an object etc. This breaks simplicity and introduces bugs.
• In a.Add(b).Mul(c).Add(d.Mul(e)) you may just explain on top of what this chained functions are doing. It's not like ((a +. b) *. c) +. (d *. e) is easy to understand either on what exactly it is doing and why. So, you have the same problem here, in neither case, you know why you're doing that operation in the first place. So .... you're gonna have to place comments on top to explain to future programmers what this is doing anyways.
• Now I have to distinguish primary type operators with object type operators? This is just breaking simplicity of the language. Now I have to be aware of where there is a primary type operator vs an object type?

For the fairness of all, I agree, ((a +. b) *. c) +. (d *. e) is more readable.

Absolutely right it is.

But for numerical types.

When you're not dealing with numerical types, how are you gonna know what is that overloaded function is doing under the hood, how are you gonna differentiate easily between primary type and object type operators (no a simple +. [dot] is not doing it :P it's even more confusing because the difference is so small). What are the usecases of this and how often are they. What problem does it solve, when do we actually need huge computational operations between objects and why not just do them inside the objects themselves if they're so huge?

tl;dr; I see 100:1 problems/what it solves on this proposal and bringing much confusion. I'm not hating the concept, I just don't see what value does it serve and how often that value is shown in actual real-life code.

[win-t]

Hi, just want to clarify

• (in this specific example) The +. ... the dot (.) is simply just distracting. Now it makes it kinda even unreadable. For the sake of what? Having operator overloading? It can be fair, if operator overloading indeed serves a very good purpose, but let's go through that.

• +, - * all these serve a very specific purpose in a language. And by overloading those operators, you're breaking that purpose

the dot (.), is just an example, it doesn't need to be a dot, we can use any symbol that not distracting, but it's enough to communicate "it is not a normal operator, look up the docs", another alternative might be @+ @*. I want to make it clear, you cannot overload existing operators, no overload for + - * /, +=, *=, ...

one may argue, that we will have multiple definitions of the @+ operator in the codebase, as everyone can define one for their type, it is not as uniform as the normal + operator that will have the same meaning anywhere in codebase. But the same situation also arises with normal methods, everyone can define Add() for their type, and for both situations, you are expected to look up the docs to understand its purpose, does it have a side-effect, etc ...

But for numerical types.

I can argue, that the correct usage of the custom operators might be not just "numerical" types, but any types that resemble mathematical sense (group, semigroup, monoid, ...).

In the end, it is just a matter of using the proper "name" for what purpose: Add(), AddMatrix(), AddModularArith() Foo(), Bar(), @+, etc ... one must choose which one is the most make sense for the type they defining. And sometimes @+ or +. is just the right name, because for some particular types, it behaves just like normal + but with slightly different behavior. I've done some financial calculations using math/big.Rat, and it is a bit pain to work with. It's hard to compare the code with the formulas assembled by the business/product team. The Rat API might be defined with care so that you can reason about performance and memory usage. but most of the time, we don't care about performance, we care about correctness and readability.

Rob also has a comment about how the Int type probably should be done. We should strive for correctness first.

Personally, the operator function is not a feature that I miss the most in golang, I can live without it. I'm neutral about this issue, just want to add some comments.

But if we want to add to the language, I want it to be defined more like magma (in the mathematical sense). the operand and the result must have same type

[Kytech]

It seems like most of the pushback towards this feature sounds a lot like the typical arguments for and against operator overloading. However, in general, it seems that operator overloading is most beneficial for numeric data types, things like BigInt, fixed-precision numeric values (ex. currency or other uses), Vectors, Matrices), while other types don't really use overloading, except for comparison and equality.

I think a good middle ground could be only allowing overloading of operators like comparison and equality, then having Go add support for numeric types like vectors, BigInt, fixed-point, etc. that would have first-class support for standard mathematical operators (though we may want some way to differentiate matrix multiplication vs element-wise, perhaps with an @ operator or some alternative), essentially locking those operators down to being available only on types that the language permits. This seems to give the simplicity, usability, and readability benefits that can come from operator overloading to the most common use case, while avoiding pitfalls of bad operator overloading.

Having equality and comparison be overrideable on any struct is valuable since it would allow a single, unambiguous method for equality and comparisons/ordering of all types. The language could even enforce that the operands must be the same type for an equals operator to return true. The fact that there are two different ways to do this (function for structs, operators for primitives) presently makes it less simple than it could be.

Arbitrary operator definition would most certainly complicate the compiler, and lead to possible confusion since it could easily result in operators that do not make much sense on their own. I don't see that happening considering that Go wants to keep the compiler as simple as they can.

I do, however, think operator overloading is a net positive and would like to see support for overloading existing operators. It seems to me that bad operator overloading implementations generally do not get much traction, with libraries doing this poorly generally being avoided. I very rarely find myself needing to overload anything more than an equality operator on most things in languages with the feature, and I think most devs have the common sense to not needlessly throw it around. I find I end up using libraries that do it well often on the flip side, but those are largely when dealing with numeric or vector types in other languages, and they're a huge benefit to code readability. It seems to me that most code authors have learned the lesson to use operator overloading carefully, which seems to have us to the point where comparison operators are the only ones most override unless you are a numeric type, which led me to the suggestion I made earlier.