Down level compile memo (tooling)

[Jack-Works]

Moved from https://github.com/bakkot/do-expressions-v2/issues/9. About how to compile this to ES2021 correctly.

I wrote a memo https://blog.jackworks.dev/2021/memo-transfrom-do-expressions/ to make sure I got things right before I actually start the coding. Please point out any edge case I missed thanks!

[nicolo-ribaudo]

Not mentioned in the memo, but be careful with super.* when wrapping do expressions with yield with a generator iife!

[Jack-Works]

@nicolo-ribaudo hi can you give me a code example?

[nicolo-ribaudo]

class A extends B {
  *method() {
    let res = do {
      yield super.foo();
    }
  }
}

This doesn't work if you transform it as

class A extends B {
  *method() {
    var _Completion;

    let res = ((yield* function*(){
      _Completion = yield super.foo();
      return _completion;
    }.call(this, arguments)), _Completion)
  }
}

because super. changes its meaning when moved inside a different function.

[Jack-Works]

Thanks, it's a nice catch!

Uncaught SyntaxError: 'super' keyword unexpected here

I'll think about how to handle this correctly. Are there any other similar syntax elements that cannot / will change meaning if I do this? I know function.sent proposal might be but it's not landed yet.

[nicolo-ribaudo]

I think only new.target and arguments (but this one is easier, just pass arguments to .call).

[Jack-Works]

I don't have a clue on how to fix super.* calls in this pattern. I cannot hoist those super.* access cause they might have side effects. Maybe hoist them as a function?

class A extends B {
  *method() {
    var _Completion;
    var _super_foo = (...args) => super.foo(...args)

    let res = ((yield* function*(){
      _Completion = yield _super_foo();
      return _completion;
    }.call(this, arguments)), _Completion)
  }
}

[nicolo-ribaudo]

Yeah Babel does something similar: https://babeljs.io/repl#?browsers=chrome%2080&build=&builtIns=false&spec=false&loose=false&code_lz=MYGwhgzhAECC0FMAeAXBA7AJjAQtA3gFDTQBUAtgigBYD2mAFAJQHEnQhXQBOCMAvNEy1W7dgDNa3aAwDcslgCNeYANay2YgJ4BLBCEzQIAVwAOCbgDpJtZhrEBfNk4dA&debug=false&forceAllTransforms=false&shippedProposals=true&circleciRepo=&evaluate=false&fileSize=false&timeTravel=false&sourceType=module&lineWrap=true&presets=env%2Cstage-1&prettier=false&targets=&version=7.13.11&externalPlugins=

(but it's broken because it doesn't inject the generator)

[Jack-Works]

I think only new.target and arguments (but this one is easier, just pass arguments to .call).

Consider this code:

function x(a, b) {
    function y(a, b) {
        arguments[1] = 2
    }
    y.call(this, arguments)
    console.log(b)
}

The assignment doesn't "popup" to the outer scope so it's a wrong transformation. I think I can ban it as a TypeScript error and use this wrong transformation since no one use arguments today.

[Jack-Works]

@nicolo-ribaudo hi thanks! I have fixed it in my memo. And I add another section Part 0 to see if we can conver it without a function wrapper.

For example for the common case:

const rnd = do {
    let tmp = rand()
    tmp * tmp;
}

We can convert it into

let _a
const rnd = ((_a = rand()), _a * _a);

[Jack-Works]

Wait, how can I down-level compile do expressions in the class fields position? 🤔

class X {
    #a = do {
        if (super.x) a; else b;
    }
}

Maybe into this?

var _super_x, completion_value
class X {
    #a = ((_super_x = () => super.x), (() => { ... })(), completion_value)
}

[nicolo-ribaudo]

That might cause problems if the do-expression contains a new X itself. I think the sacred thing to do is to store _super_x and completion_value as private fields.

[Jack-Works]

so if it is not a private field but a public field, I think it is not acceptable to introduce a new private field for code that not using a private field (cause down-level compiling are based on WeakMap)

class X {
    #a_super_x = () => super.x
    #a_completion_value
    a = (((setCompletionValue) => { ... })(x => this.#a_completion_value = x), this.#a_completion_value)
}

[Jack-Works]

hmm, I think I need to revisit my compiling method. it seems like my memo don't work for class private/public fields

[Jack-Works]

Is there a non-static version for https://github.com/tc39/proposal-class-static-block? If so I can compile this easily into things like

class X {
    a;
    init block {
        // ...
        this.a = completion_a
    }
}

[ljharb]

The non-static thing is “inside the constructor”

[nicolo-ribaudo]

The non-static version is the constructor.

However, in class fields you cannot have yield/await so this might be enough?

class X {
    #a = (() => {
        var _completion_value;
        if (super.x) _completion_value = a; else _completion_value = b;
        return _completion_value;
    })();
}

[Jack-Works]

Oh wait I forgot super is not a function scoped thing (like arguments)

[ljharb]

It’s exactly like arguments, in that arrow functions don’t bind either.

[Jack-Works]

I have to say if we have an arrow version of the generator function & async generator function things will be easier

[ljharb]

Can you use super inside a nested non-arrow generator function such that it matters?

[Jack-Works]

Not related to the super case, just cause I need to take care about this arguments that won't cross the normal function but do across the arrow version

[Jack-Works]

https://github.com/bakkot/do-expressions-v2/issues/9#issuecomment-768803688

The transformation of control flow jumps is tricker than I think.

Consider this code:

const x = do {
    try { if (y) break; 1; } catch {}
}

It will accidentally catch the "breaking" signal behind the transformation so I need to also add a "re-throw" code in every catch user writes.

[Jack-Works]

What about this compile result?

function f(y) {
    const x = do {
        if (y) return 2
        1
    }
    return x
}

Pros of this manner: How to transform yield inside do expression is already known. And yield is not able to accidentally caught by the userland like the try-catch way.

function f(y) {
    const returnSign = { value: undefined }
    function* f_inner() {
        const x = do {
            if (y) ((returnSign.value = 2), (yield returnSign));
            1
        }
        yield x
    }
    for (const _a of f_inner()) {
        if (_a === returnSign) return returnSign.value
        return _a
    }
}

Further into

function f(y) {
    const returnSign = { value: undefined }
    function* f_inner() {
        var completionValue
        const x = ((yield* (function* () {
            if (y) ((returnSign.value = 2), (yield returnSign));
            completionValue = 1
        })()), completionValue)
        yield x
    }
    for (const _a of f_inner()) {
        if (_a === returnSign) return returnSign.value
        return _a
    }
}

[bakkot]

I think you can make that work, but introducing a generator where none was previously adds a lot of overhead - more than just rewriting catch blocks to add a rethrow.

[Jack-Works]

Ok... I found out this code

switch (expr) {
    case 1:
        let x = do {
            if (cond) break
            1;
        }
        console.log(x)
        function f() {
            return x
        }
        break
    case 2:
        x = f()
}

This basically not translatable in a reasonable way. Is it enough to analyze all things in the current scope and hoist them, or other edge cases?

[Jack-Works]

Because switch doesn't have a "block" for me that I can write those things:

switch (expr) {
    try {
        case ....
    } catch (e) {...}
}

I need to convert them clause-by-clause but that make cross-clause definitions harder to handle with

[pitaj]

Isn't this

switch (expr) {
    case 1:
        let x = do {
            if (cond) break
            1;
        }
        console.log(x)
        function f() {
            return x
        }
        break
    case 2:
        x = f()
}

equivalent to this?

{
  let x;
  function f() {
    return x
  }
  switch (expr) {
    case 1: {
      x = do {
        if (cond) break
        1;
      }
      console.log(x)
    } break
    case 2: {
      x = f()
    }
  }
}

So it would be possible to hoist all declarations to a containing block and convert each case as a block?

[Jack-Works]

Yes, I hope this kind of analysis does not miss any edge case in the switch statement.

[Jack-Works]

Ok I got two more edge cases...

function f(x = do { if (expr) return; 1 }) {}

for (const { prop = do { if (expr) continue; 1 } } of arr) {}

Is that allowed today?

[bakkot]

I haven't written spec text for how to deal with those cases yet. My intention is that break and continue within loop heads (as in your second example) will be forbidden, but I haven't decided about return in parameter lists (as in your first). Seems fine to forbid both for now, at least.

[Jack-Works]

Seems fine to forbid both for now, at least.

Yes, it will make the transformer easier. And I don't think any one will need it

[Jack-Works]

Some progress on control flow changes within do expression. (Note I temporarily disabled the transform of do expression because it does not work well with the control flow transform).

[Ontopic]

Loving this. Any idea when this is gonna be semi usable?

[Jack-Works]

Loving this. Any idea when this is gonna be semi usable?

You can use babel to try this. For TypeScript, it is still in progress.

[Jack-Works]

I think most of the transformation work in TypeScript has been done in https://github.com/microsoft/TypeScript/pull/42437. I'll close this.