Non-local returns

[gisborne]

In Ruby, it is natural to write something like this:

MyArray.each{|x| return x if x.hasSomeProperty}

The difference between this sort of anonymous function (a ‘block’) and a regular one is that if you return in a block, it returns from the scope where the block is defined. Once you get used to it, you really miss it in languages that don’t have it.

We would probably need an alternative anonymous function syntax because we wouldn’t want to change the existing behavior.

[munificent]

We do have anonymous functions already, though not with Ruby's syntax. There has been a number of requests to add a more block-like syntax for passing a closure to a function. I would like that too, though it's difficult to find a way to get it into Dart given all of the existing syntax we have.

Is your request here more for non-local returns and less about the block syntax itself? We have discussed non-local returns a number of times over the years. They are very cool, but I don't know how well they would fit in Dart. They may be too surprising for many users to be worth it.

[gisborne]

Yes. Nonlocal returns. On Apr 10, 2020, 15:23 -0700, Bob Nystrom notifications@github.com, wrote:

We do have anonymous functions already, though not with Ruby's syntax. There has been a number of requests to add a more block-like syntax for passing a closure to a function. I would like that too, though it's difficult to find a way to get it into Dart given all of the existing syntax we have. Is your request here more for non-local returns and less about the block syntax itself? We have discussed non-local returns a number of times over the years. They are very cool, but I don't know how well they would fit in Dart. They may be too surprising for many users to be worth it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[gisborne]

Just want to add that there isn't any more reason to think that the folks using Dart would have any more trouble using blocks than folks using Ruby.

I imagine that as compiler features go, it wouldn't be horrific to implement. And syntactically, we could have:

regular lambda: () {} block: || {}

It really is nice that a block behaves more like a language feature than a function.

[eernstg]

We could use | argument | {...}, or we could use a syntax that marks any specific return statement as a non-local return, that doesn't matter for the following topic. I'm trying out the second alternative below, just to illustrate that we can have both in the same function if we want, and also to illustrate that the issue I'm describing is applicable to a slightly generalized notion of non-local returns, not just the most basic one.

One reason why non-local returns may be a less-than-optimal match for Dart is that they are very likely to come with a run-time error which is difficult to detect statically (as in undecidable in the general case).

bool Function(int) globalG;
bool b = false;

void f(bool Function(int) g) {
  globalG = g;
}

void foo() {
  f((i) {
    if (b) foo.return; // Non-local return, returning from `foo`.
    return true; // Return a value to the caller, `f`.
  });
}

void main() {
  foo();
  b = true;
  globalG(42); // Throws!
}

The problem is that the invocation of globalG will call the function literal that has the non-local return, but when it tries to non-locally return from foo, there is no such invocation on the call stack. We could also have used the most common kind of non-local return (like f.return true;), but that's going to throw for the same reason (there is no f on the call stack).

In order to make this kind of mechanism statically safe there would need to be a static analysis ensuring that those function objects will never be invoked after the point where a function like foo or f has returned. For example, we can't allow such function objects to be stored in a top-level variable (or indeed in any variable, local/instance/static/top-level, which is alive after those returns).

[lrhn]

Another approach is to not allow such functions to escape, which require keeping them marked as special at every step.

To be able to do a safe non-local return (or other control flow), it's imperative that the closure doing so doesn't escape. The stack frame it returns from or jumps into must still be on the stack when that closure is called.

That suggests that there is a special kind of functions which can only be passed down as argument to invocations further down the stack, but can never be stored anywhere else. All you can do with such a "stack local" function is to call it, store it in a variable which remembers that it is such a function, or pass it further down the call stack to arguments expecting such a function. It must never be forgotten that the function is stack local, and non-local variables cannot contain stack local variables.

Basically, it's a parameter property: The value of a specially declared parameter can be "stack local", and it can never escape, and therefore it must never be forgotten.

It's probably OK to use a non-stack-local function where a stack local function is expected, not escaping won't hurt it.

So a stack local parameter is like a supertype for the non-stack-local type. Let's say we write local for it:

class Iterable<E> {
  void forEach(local void Function(E element) action) {
     for (var element in this) action(element);
  }
  bool any(local bool Function(E element) test) {
    forEach((e) local {                // strawman syntax!
      if (test(e)) any.return true;
    });
  }
  // ...
}

You can't abstract over stack-locality, it's a property of the parameter (or possibly a local variable). Either you support it or you don't.

It's a compile-time error to assign the value of a stack local variable to a non-stack-local variable. All you can do is assign or pass it to a stack-local local variable or function parameter.

A Function(local T) is a supertype of Function(T). You can override a stack-local parameter to be normal in a subclass. It's a compile-time error to go in the other direction, even using covariant. It's function parameter property, not a type.

But then there is async, which just doesn't work. An async, async* or sync* method cannot have a stack-local variable. The stack context won't survive past the first await of the async function, and not even into the body at all for the rest.

That's pretty restrictive. We could probably introduce an async-equivalent version, if an async function calls another async function and immediately awaits the result, which means that an async function can have a stack local parameter, but it can only be called from inside another async function, and only if result is awaited immediately. Still not type-safe because we cannot distinguish an async function from a non-async function returning a Future.

So, very much at-odds with out async/await story and asynchrony in general. Shucks.

[munificent]

Just want to add that there isn't any more reason to think that the folks using Dart would have any more trouble using blocks than folks using Ruby.

I can think of four:

1. Ruby is a heavily object-oriented language where every named callable-thing declaration is declaring a method. That method becomes the terminating boundary for any non-local returns within it. Blocks and lambdas are understood by users to be a fairly different concept with different behavior. So Ruby has two things: methods which terminate non-local returns and blocks/lambdas where returns are non-local.

   In Dart, top-level functions and local functions are functions and not methods. Users are already used to non-method callable-things that don't have non-local returns. If we did non-local returns, users would have to reason about three things: methods which terminate non-local returns; functions which aren't methods but also terminate non-local returns; and blocks, which aren't functions and return non-locally.

2. Dart users are already extremely used to writing functions (with normal return semantics) for all of the common use cases where you would use a block with non-local returns in Ruby.

3. Dart doesn't have implicit returns. In Ruby, return is rare and clearly stands out as performing control flow since the normal way to have a function or method yield a value is to just have it be the last expression in the body. In Dart, every time a function yields a value, you use an explicit return (or a => body).

4. Imperative code is well-supported and encouraged by Dart. We have a lot of rich control-flow constructs and it's idiomatic to use them. In Ruby, the typical way to iterate a sequence is using a block, and if you want to abort in the middle and have the surrounding method return a value, you need a non-local return. In Dart, the typical way to iterate a sequence is with a for-in loop, and if you want to exit early and yield a value, you just return and it does what you want.

I think non-local returns are a cool feature, but I think for Dart, they would mostly be a solution in search of a problem.