Hoisting v2

[conartist6]

Hoisting v1 consisted of building a tree and then trying to work out hoisting after the fact. Grammars provided isHoistable ((token) => boolean), and after a tree was constructed leading and trailing hoistables were spliced into their parents match results. I didn't like the solution that much. I didn't like the splicing, and that you could only tell what was happening after the results were built. That said, the solution worked and seems to have been technically sound, which is more than you can say for anything I've tried to replace it with so far.

I will note that when I started ripping out v1, I thought it was unsound because at the time I thought I should be hoisting expression parenthesis. I no longer think that's likely to be correct, but the problem would have been the difficulty of distinguishing between call() and (call()). The algorithm, being poorly integrated with the grammar, there existed the potential to accidentally create a malformed node like call(.

The v2 implementation integrates hoisting fully into the grammar, using startNode and endNode commands that unambiguously identify the correct hoisting.

The v2 implementation has a serious flaw though! You can see it in the log from parsing o = { foo }.

In this situation foo is an Identifier wrapped in a Property here, which means that correct behavior is to take the leading space before the Identifier and hoist it not into the Property but into the parent ObjectExpression. Unfortunately you can see in the log that Property emits startNode before it references Identifier. This has to occur so that ref`key` ends up in the right place, but it also captures the whitespace, which fails to bubble up through Property as it should.

I've tried any number of strategies to resolve this problem, but the parameters for a successful solution are remarkably narrow:

• The startNode command should be issued at the correct time, and by the grammar generator. Getting this right increases the value of the command stream to plugins.
• The solution must be compatible with branching. Matched tokens to be hoisted must still live in the tip state so as to be reversible, meaning that there is no chance to simply do the hoisting (emit tokens into the parent node result) immediately.

[conartist6]

This was pretty cool. I've gotten so used to thinking about things in terms of my nice structure logs that I mocked up the log output before writing the code to produce it. I believe this is the correct solution.

[conartist6]

Implemented.