proposal: Go 2: Partially Applied Functions

[teivah]

Let's consider a higher-order function foo taking as an argument another function:

func foo(f func(int) int) int {
  // Do something
}

Let's now imagine we need to call foo by passing a function that would depend on an external context (something not passed as arguments) like a Context interface:

type Context interface {
    isBlue() bool
}

Today's Option 1: Custom Structure and Method

We wrap this Context in a custom structure and we pass a method to foo.

For example:

type MyStruct struct {
    context Context
}

func (m MyStruct) bar(i int) int {
    if m.context.isBlue() {
        return i + 1
    } else {
        return i - 1
    }
}

func Test(t *testing.T) {
    m := MyStruct{}

    foo(m.bar) // Call foo with a method
}

Today's Option 2: Pass a Closure

We create a closure and we call foo this way:

func Test(t *testing.T) {
    context := newContext()

    bar := func(i int) int {
        if context.isBlue() {
            return i + 1
        } else {
            return i - 1
        }
    }

    foo(bar) // Call foo with a closure
}

Proposition: Partially Applied Function

In FP languages, there is a more elegant solution allowing to facilitate the unit tests: partially applied functions.

func bar(context Context, i int) int {
    if context.isBlue() {
        return i + 1
    } else {
        return i - 1
    }
}

func Test(t *testing.T) {
    context := newContext()

    f := bar(context, _) // Apply partially bar
    // The blank operator means the argument is not provided
    // At this stage f is a func(int) int

    foo(f) // Call foo with f
}

First, it allows to keep passing a pure function which does not depend on any external context. Then, writing an unit test with a partial function is way easier as it depends only on the input arguments.

Furthermore, the fact that Go does not allow function overloading could really ease the implementation/adoption of partially applied functions in my opinion.

It would be awesome to have such features in Go 2 :)

[ianlancetaylor]

This is often call currying.

[teivah]

This is often call currying.

@ianlancetaylor Not really. Currying is translating the evaluation of a function that takes multiple arguments into evaluating a sequence of functions, each with a single argument.

Something like this:

func f(a int)(b int) int {
  return a + b
}

func client() int {
  return f(1)(2)
}

In most of the FP languages, it is not mandatory to use currying for applying partial function. It can be done using a blank operator (as proposed in the issue).

Yet, it is often a mixed concept because it may seem more natural to partially apply a currying function to prevent having to use a blank operator. Something like this:

func f(a int)(b int) int {
  return a + b
}

func client() int {
  pf := f(1) // pf is a func(int) int
  pf(2)
}

[deanveloper]

Just for clarity, would this be (similar) to Function.bind in JavaScript?

And if it is, I think the syntax is a bit misleading. To me it looks like we are calling f, even though the code in f never gets executed. Perhaps there should be an alternative syntax for this?

[teivah]

@deanveloper I'm not familiar with Javascript, to be honest, but I don't really think it is similar.

Here, this is just a mechanism to create a function from another function. If we implement a foo function with two arguments A and B, we can partially apply foo with a given value for A. As a result of this call, we get another function taking only one argument B. Then, each time we call this function, the value for A is automatically applied without having to pass it explicitly (we just have to pass a value for B).

The alternative syntax is what we discussed above with currying function. But this is for me another topic (which may or may not be interesting for Go v2, I don't know).

[beoran]

You can already do this, with a bit of boilerplate: https://play.golang.org/p/Ba36DN1S3jD

package main

import "fmt"

type Context struct {
  blue bool
}

func (c Context) isBlue() bool {
  return c.blue
}

func bar(context Context, i int) int {
    if context.isBlue() {
        return i + 1
    } else {
        return i - 1
    }
}

func apply_bar(apply func(Context, int) int, context Context) func(int) int {
    return func(i int) int {
        return apply(context, i)
    }
}

func foo(f func(int) int) {
   fmt.Printf("Calling with 1: %d\n", f(1))
}

func main() {
    context := Context{false}
    f := apply_bar(bar, context)    
    foo(f) // Call foo with f
    context.blue=true // see what happens if context changes
    foo(f) // Call foo with f, stays the same since context was copied by value.

    context2 := Context{true}
    f2:= apply_bar(bar, context2)   
    foo(f2) // Call foo with f2
}

I'd say, use go generate or write specific library to solve this problem.

[teivah]

@beoran You're right it works but we still have to write something specific for that. It's still far more elegant, in my opinion, to manage partial functions. Don't you think?

Do you mind if I quote your solution as an alternative with the current Go version?

[lrewega]

Maybe I'm missing something but it seems this proposal is just syntactic sugar for a closure:

func bar(context Context, i int) int {
    if context.isBlue() {
        return i + 1
    } else {
        return i - 1
    }
}

func Test(t *testing.T) {
    context := newContext()

    f := func(i int) int { // Apply partially bar
        return bar(context, i)
    }

    foo(f) // Call foo with f
}

You allude to closures but the example is maybe a bit contrived: both partial application and a closure are one-liners.

From the book The Go Programming Language:

Simplicity requires more work at the beginning of a project to reduce an idea to its essence and more discipline over the lifetime of a project to distinguish good changes from bad or pernicious ones. With sufficient effort, a good change can be accommodated without compromising what Fred Brooks called the 'conceptual integrity' of the design but a bad change cannot, and a pernicious change trades simplicity for its shallow cousin, convenience. Only through simplicity of design can a system remain stable, secure, and coherent as it grows.

This appears to me to be a pernicious change (in terms of legibility) for the sake of convenience.

[beoran]

@teivah No problem to quote my example. However, I am not currently in favor of this proposal. Like @lrewega says, this is syntactic sugar, for a rather uncommon use case. I think it's not worth making go more complex for such a limited use convenience feature.

[teivah]

@lrewega @beoran Thanks for your remarks.

Yet, what about the following example?

i := 5
f := func() int {
    return i
}
i = 6

fmt.Printf("%v\n", f())

Here, it displays 6, not 5. Partial functions ensures that the value we passed during the partial application has been fixed and cannot be mutated. This is why for me, this is not only syntactic sugar.

[4ad]

I wish Go had partial functions, but partial functions are mostly useful because of currying. Using _ as a blank parameter is not currying, it's unexpected and confusing. If this is considered further, better syntax is needed.

[deanveloper]

Here, it displays 6, not 5. Partial functions ensures that the value we passed during the partial application has been fixed and cannot be mutated. This is why for me, this is not only syntactic sugar.

We shouldn't implement immutability into only one feature of the language. That's just confusing and isn't consistent with the rest of Go.

[teivah]

@deanveloper I'm not sure to get your remark. For example, a string is immutable. It does not mean that it is confusing and inconsistent with the rest of Go.

Here, I was not proposing a deep change with immutable data structures or whatever. Just a way to fix a function argument ;)

[deanveloper]

That was my bad, I didn't mean immutability in that sense. I mainly just meant that an out-of-scope variable always holds the value that it holds and never gets "fixed" to anything in Go, and I think it would be a bit better (and more intuitive to read) if you save the value to a variable, use an anonymous function, and call that instead. For instance with your example:

i := 5

iInF := i
f := func() int {
    return iInF
}

i = 6

fmt.Printf("%v\n", f())

Would print 5

[teivah]

@deanveloper Sure but how do you write a unit test? You need to wrap it in another function. Maybe something like this: https://github.com/golang/go/issues/29171#issuecomment-446116025

Obviously, there is no a hard limitation. It's just an elegant solution (in my opinion) to solve a common problem without having to write custom code.

[reusee]

Another option with go2 generics:

func PartialEvaluateIx_I_(
    type A1, A2, R1
) (
    fn func(A1, A2) (R1),
    a2 A2,
) (
    ret func(A1) (R1),
) {
    return func(a1 A1) (R1) {
        return fn(a1, a2)
    }
}

func PartialEvaluateI__x_I_(
    type A1, A2, A3, A4, R1
) (
    fn func(A1, A2, A3, A4) (R1),
    a1 A1,
    a2 A2,
    a4 A4,
) (
    ret func(A3) (R1),
) {
    return func(a3 A3) (R1) {
        return fn(a1, a2, a3, a4)
    }
}

func PartialEvaluateIx_x_I__(
    type A1, A2, A3, A4, R1, R2
) (
    fn func(A1, A2, A3, A4) (R1, R2),
    a2 A2,
    a4 A4,
) (
    ret func(A1, A3) (R1, R2),
) {
    return func(a1 A1, a3 A3) (R1, R2) {
        return fn(a1, a2, a3, a4)
    }
}

[beoran]

I updated my example boved and linked correctly to go playground to show that if you pass the context by value, so it gets copied, you get the effect of partial function application without interference from captured variables. True, there is some boilerplate to write (as always in Go), but as @reusee shows, generics should solve that problem once we get them. So I'd rather push for generics that for this rather limited use feature.

[teivah]

@beoran Why not both? :)

[beoran]

@teivah Go has been from the "get go" a programming language that only includes the most widely useful features.This to avoid the language from becoming too complex and too hard to understand.

If we now look at https://blog.golang.org/go2-here-we-come for the Proposal Selection Criteria, namely:

1. address an important issue for many people,
2. have minimal impact on everybody else, and
3. come with a clear and well-understood solution.

Then I think the feature proposed in this issue does not meet 1. It is not an important issue for many people. Of course I am only a sample of one, but I miss enums in Go, I miss unions, and I sometimes miss generics. But I never missed partial functions, especially because Go already has closures that allow you to emulate them relatively easily.

That's why I respectfully ask that this proposal be declined.

[teivah]

@beoran Ok I do understand. No problem.

[faiface]

@teivah I'm sorry, but you're not using the terms correctly. The term partial function refers to a function that doesn't return for all possible arguments, but crashes or doesn't terminate for some. The opposite of a partial function is a total function, a function that returns a value whatever argument you provide. Even the link you provided explains this, so it seems like you haven't even read it.

The term you're looking for is partial application or currying, which is taking a function and filling some of its arguments and thereby getting a function that takes the remaining arguments.

[teivah]

@faiface Right, maybe I should have rather used the notion of partially applied function. Yet, I disagree currying is not the same thing in my opinion (see my comment there: https://github.com/golang/go/issues/29171#issuecomment-445962391). Anyway, I have been told that this was not mandatory and it could be solved using closures (which for me is not the most elegant solution as partially applied function can ensure arguments are not going to be mutated in between the partial application and the actual function call).

[faiface]

@teivah Yeah, no problem, just wanted to clarify what means what :). Other than that, I think partial applications could be useful in Go, although not very often I believe.

[ianlancetaylor]

This proposes a new feature to the language that can be done using other mechanisms, and that few people are asking for. Also, the syntax does not seem intuitive to me. If I can write f(v, _) then why can't I write f(_, v) or g(a1, _, a2, _, a3)? But that seems too confusing.

If we want to make changes in this area it's perhaps best to consider #21498.

[JCCPort]

This proposes a new feature to the language that can be done using other mechanisms, and that few people are asking for. Also, the syntax does not seem intuitive to me. If I can write f(v, _) then why can't I write f(_, v) or g(a1, _, a2, _, a3)? But that seems too confusing.

If we want to make changes in this area it's perhaps best to consider #21498.

This feature would be super useful for some data analysis tasks. For example (basic curve fitting):

I have some function f(x, a, b, c) mathematically expressed as f(x; a, b, c) meaning I am returning values of f for different values of the variable x where I am considering a, b, c to be parameters of the equation. Given pairs of values f, x I can use a minimisation algorithm to find a, b, c.

However, let's say I have data for which there is some fixed value of c. It would be useful to define some function f2(x, a, b) = f(x, a, b, _).

[ianlancetaylor]

You can already do that, of course, using a function literal. This proposal is simply suggesting a more concise mechanism.

In any case, this proposal is closed. Personally I think that #21498, which explores more general mechanisms, is likely to be a better path forward. Though there is no guarantee that that proposal will be accepted either.

[teivah]

@ianlancetaylor I think https://github.com/golang/go/issues/21498 solves a completely different problem. Anyway, as you said, it's closed.