Support for generators

[Akeboshiwind]

To upvote this issue, give it a thumbs up. See this list for the most upvoted issues.

Is your feature request related to a problem? Please describe. I'm trying to play with https://motioncanvas.io/ which uses generators as a declarative interface for animations. Currently this is tricky to do.

Describe the solution you'd like I think the simplest solutions would be three function/macros:

• js/yield
• js/yield*
• js/function* (not a good name but something like that)

Describe alternatives you've considered I've tried writing my own macros, but my-fn didn't output something that worked:

(defmacro my-fn [[& params] & body]
  `(js* "function* ~{} {\n~{}\n}"
          ~params
          (do ~@body)))

[borkdude]

Syntax-wise, I've discovered the convention in CLJS is to write keywords in JS as: js-in, js-debugger, this is why js/await now is written as js-await, so I propose: js-yield and js-yield* for those things. js/ should be exclusively used for accessing the global JS environment.

For async functions we use metadata and no special syntax: (defn ^:async foo [])

So I propose something like:

(defn ^:generator foo [])

or maybe shorter:

(defn ^:gen foo [])

or maybe:

(defn ^:yield foo [])

or maybe

(defn ^:* foo [])

although ^:* will be confusing so let's not go there.

One challenge is that let and do are compiled as IIFEs and those expressions can contain yield and/or may have a useful return value. We can solve that as follows:

function* foo () {
  yield 1;
  yield* [2, 3];
  let x = (function* () {
    yield 6;
    yield* [7, 8, 9];
    return "This is X";
  })();
  while(true) {
    const { value, done } = x.next();
    if (done) {
      x = value;
      break;
    } else {
      yield value;
    }
  }
  console.log('x', x);
}

Each IIFE generator value must be consumed and the return value must be assigned to the appropriate name.

[Akeboshiwind]

This also works FYI:

function* foo () {
  yield 1;
  yield* [2, 3];
  let x = undefined;
  yield* (function* () {
    yield 6;
    yield* [7, 8, 9];
    x = "This is X";
  })();
  console.log(x);
}

[Akeboshiwind]

I think that all makes sense! I'd have a preference for :gen or :generator as then it's easier to figure out what to google when you're reading some code. I also presume there would be a fn variant?

[borkdude]

Yes: (fn ^:gen []) perhaps?

[lilactown]

One challenge is that let and do are compiled as IIFEs and those expressions can contain yield and/or may have a useful return value.

Doesn't async have this problem too? I thought we solved that by not emitting IIFEs when inside an async fn, but maybe I'm wrong. I would imagine there being a runtime impact on creating so many generators/async boundaries without the authors knowledge.

[borkdude]

I thought we solved that by not emitting IIFEs when inside an async fn

Wrong, an async IIFE is emitted and then awaited upon.

[borkdude]

I would imagine there being a runtime impact on creating so many generators/async boundaries without the authors knowledge

It's either that or not support it at all (like it is currently). It's not like we're introducing regressions since it's not supported at all right now.

[borkdude]

Note that CoffeeScript doesn't work with yield in do blocks (yet): https://github.com/jashkenas/coffeescript/issues/5461 I consider that a bug.

[borkdude]

There is a TC39 proposal for do blocks and a babel transform which expands into an IIFE, so same issue as above.

import { transformSync } from '@babel/core';

const text = `
function* foo () {
    let x = do {
        yield 1;
        return 2;
    }
    yield x;
}

foo();
`

const js = transformSync(text, {
    plugins: ["@babel/plugin-proposal-do-expressions"],
  })?.code ?? ""

console.log(js);
const xs = eval(js);
console.log([...xs]);

$ node do.mjs
function* foo() {
  let x = yield* function* () {
    yield 1;
    return 2;
  }();
  yield x;
}
foo();
[ 1, 2 ]

So it seems simply prefixing the IIFE with yield* is all we need to do.

[borkdude]

The babel transform has one optimization: it detects if implicit IIFEs (like squint/cherry has for do and let) use async/yield internally and if not, a normal function is produced. We could apply this optimization later on.

[lilactown]

Makes sense to me! Maybe some day we'll get first class do-blocks and can stop worrying about this stuff 😓

[borkdude]

Demo: https://squint-cljs.github.io/squint/?src=KGRlZm4gXjpnZW4gZm9vIFtdCiAgKGpzLXlpZWxkIDEpCiAgKGpzLXlpZWxkKiBbMiAzXSkKICAobGV0IFt4IChpbmMgMyldCiAgICAoanMteWllbGQgeCkpCiAgKGxldCBbeCAoZG8gKGpzLXlpZWxkIDUpCiAgICAgICAgICAgIDYpXQogICAgKGpzLXlpZWxkIHgpKSkKCih2ZWMgKGZvbykp

[Akeboshiwind]

Amazing as always!