How to terminate a coroutine early and call `finally` blocks?

[darkleaf]

JS generators have the return method:

var fn_gen = function* () { 
  try {
    yield
  }
  finally {
    console.log("finally")
  }
};

var gen = fn_gen();
gen.next()
// => {value: undefined, done: false}
gen.return()
// => finally  <= the finally block has been executed 
// => {value: undefined, done: true}

Can I do the same thing with the cloroutine? I thought about exceptions but anyone can catch Throwable.

[leonoel]

I can't really imagine an easy way to do that, but IMO it's more about the language than it's about coroutines. Unlike JS, clojure doesn't have return, so cloroutine inherits this limitation. Exceptions come close, you could use that trick to return from an arbitrary place in an ordinary clojure function but it's generally a sign that your code should be rewritten in a more clojure-friendly style.

[darkleaf]

It is not about the return statement. Maybe you misunderstood my idea?

(defn ! [x] x)
(defn handle-! [] nil)

(cr {! handle-!}
  (try 
     (! nil)
     (! :some-value)
     :finish
     (finally
       (prn "finally"))))

The coroutine will return nil, :some-value, :finish.

When coroutine returns nil I want to stop it and get all finally blocks called as if the coroutine weren't contain other breaks.

I can't really imagine an easy way to do that

Yes, the current API can't provide such functionality.

[leonoel]

I understand what you're trying to achieve. cloroutine doesn't aim to provide more than the ability to split the evaluation of a clojure expression on arbitrary break points. It aims to be a low-level facility on top of which higher level constructs can be build, such as generators. It doesn't aim to change the clojure language. Adding a return-like facility to the cloroutine API, while technically possible, would break some assumptions one could make about clojure expressions. In your example, I would find it very confusing if the try block could return successfully without having evaluated all of its body expressions. In javascript, it's not an issue because the language already allows to return from any point. Obviously, if a return-like statement would be added to the clojure language at some point, it would be natural for cloroutine to support it as well.

[darkleaf]

First of all, are we talking about the return statement or the return method of a js generator?

I don't want to add the return expression to clojure. I want to correctly stop a coroutine in the middle of it execution. The cloroutine library has support for exceptions but it hasn't holistic one in my mind. Maybe I'm wrong. Maybe a coroutine shouldn't work with exceptions at all but a js generators does.

I would find it very confusing if the try block could return successfully without having evaluated all of its body expressions.

(cr {! handle-!}
  (try ;; point 1
     (! nil) ;; point 2
     (! :some-value)
     :finish
     (finally
       (prn "finally"))))

In my example, the try block doesn't "return" a value. Look, the coroutine returns nil at the point 2, and then I want to call an imaginary method coroutine.return() wich just would run all of finally blocks. The whole try form at the point 1 don't produce a value. It's like throwing an exception:

(cr {! handle-!}
  (try ;; point 1
     (! nil) ;; point 2
     (throw (SOME_HALT_EXCEPTION. ...))
     (! :some-value)
     :finish
     (finally
       (prn "finally"))))

Look at the more complex generator usage:

var fn_gen = function* () { 
  yield 0;
  try {
    yield 1;
    yield 2
  }
  finally {
    console.log("finally")
  }
};

var gen = fn_gen();

gen.next()
// {value: 0, done: false}
gen.next()
// {value: 1, done: false}
gen.return()
// finally
// {value: undefined, done: true}

There are no return statements. The return method just closes the generator properly early.

Or maybe you are talking about the difference between a try statement and a try expression? I think there are no diffrence in this context. We just stop an execution.

PS. It's a little bit hard to express my thoughts in English. Sorry about it.

[darkleaf]

But maybe I can just throw InterruptedException. But which I should use in javascript?

I want something like a Thread#interrupt in Java.

[leonoel]

First of all, are we talking about the return statement or the return method of a js generator?

Both. We need the former to implement the latter.

There are no return statements. The return method just closes the generator properly early.

gen.return() works as if a return statement was automagically inserted before giving control back to the generator, or at least that's the mental model I have about it.

Or maybe you are talking about the difference between a try statement and a try expression? I think there are no diffrence in this context.

I don't see a difference either, try is an expression in clojure and a statement in javascript but that doesn't really matter in this context.

We just stop an execution.

To be precise, we ask the execution to gracefully shutdown. Clearly, the execution must continue, because we want the finally blocks to be evaluated.

PS. It's a little bit hard to express my thoughts in English. Sorry about it.

No worries. I'm not english-native either.

But maybe I can just throw InterruptedException. But which I should use in javascript? I want something like a Thread#interrupt in Java.

Exceptions work, and they're probably the right tool for this job. Use whatever type you want and make this behavior explicit. Allowing the generator to catch it is a feature, not a bug.

TBH I fail to understand the rationale for gen.return(). It's not clear to me what should happen if we reach more yield statements in a finally block after having called gen.return(). The interruption of a logical process must be cooperative, java learned it the hard way, and Thread#stop was deprecated for good reasons.

[darkleaf]

It's not clear to me what should happen if we reach more yield statements in a finally block after having called gen.return()

It wouldn't be done after gen.return().

var fn_gen = function* () { 
  try {
    try {
      yield 1;
    }
    finally {
      console.log("internal finally 1")
      yield 2;
      console.log("internal finally 2")
    }
  }
  finally {
    console.log("outer finally")
  }
};

var gen = fn_gen();
gen.next()
// {value: 1, done: false}
gen.return();               <<<< return <<<<<
// internal finally 1
// {value: 2, done: false}  <<<< done: false <<<<<

/* !!!!!!!!!!!!! */
gen.next()
// internal finally 2
// outer finally
// {value: undefined, done: true}

Allowing the generator to catch it is a feature, not a bug.

Can you provide an example?

A finally block with a break are weird in the cloroutine. It throws only on the third coroutine call. Maybe it is a bug?

(defn ! [] :ok)
(defn handle-! [])

(let [coroutine (cr {! handle-!}
                    (try
                      (prn "step 1")
                      (throw (ex-info "halt" {}))
                      (prn "step 2")
                      (finally
                        (!)
                        (prn "step 3")
                        (!)
                        (prn "finally"))))]
  (coroutine)
  (coroutine)
  (coroutine))

;; "step 1"
;; "step 3"
;; "finally"
;; Execution error (ExceptionInfo) at darkleaf.effect.core/eval17693$cr17694-block-1 (REPL:14).
;; halt

Maybe I'll look at the compiled code and can suggest how to call finally blocks without an exception. Anyway it just a state machine, so we can just jump to a finally block.

[leonoel]

Can you provide an example?

The meaning of an interruption depends on the process. If the process is some kind of computation that eventually produces a result, then interrupting it before completion would most likely result in an error. On the other hand, if it's a situated program (e.g a web server handling requests), then interrupting it is the normal way to terminate and it should probably not result in an error. Throwing an exception on interruption is good design because the process can catch it and give some meaning to it, maybe turn it into a successful termination. You can't do that with a return-like interruption mechanism.

A finally block with a break are weird in the cloroutine. It throws only on the third coroutine call. Maybe it is a bug?

It's the intended behavior. What do you find weird about it ?

Maybe I'll look at the compiled code and can suggest how to call finally blocks without an exception. Anyway it just a state machine, so we can just jump to a finally block.

You're right, but to be very clear I'm not interested in this feature. I'm fine with current design, I like its simplicity and I want to keep cloroutine semantics on par with the clojure language.

[darkleaf]

It's the intended behavior. What do you find weird about it ?

I think it should throw at the first call:

;; "step 1"
;; Execution error (ExceptionInfo) at darkleaf.effect.core/eval17693$cr17694-block-1 (REPL:14).
;; halt

[leonoel]

Compare it with the thread version :

(def ^:dynamic *queue*)

(defn ! [] (.put *queue* (fn [] :ok)))

(defn cr [f]
  (let [q (java.util.concurrent.SynchronousQueue.)]
    (.start (Thread. #(.put q (try (let [x (binding [*queue* q] (f))] (fn [] x))
                                   (catch Throwable t (fn [] (throw t)))))))
    #((.take q))))

(let [coroutine (cr #(try
                       (prn "step 1")
                       (throw (ex-info "halt" {}))
                       (prn "step 2")
                       (finally
                         (!)
                         (prn "step 3")
                         (!)
                         (prn "finally"))))]
  (coroutine)
  (coroutine)
  (coroutine))
;; "step 1"
;; "step 3"
;; "finally"
;; Execution error

[darkleaf]

Compare it with the thread version :

I was wrong.

---

I'm fine with current design

This is an answer. Thanks for the discussion. It was cool.