Fix closure conversion and make CST-to-AST do what Generate-AST does.

[karlosz]

Speeds up partial inlining noticeably.

[Bike]

Clasp built fine using this.

[Bike]

Using this to build Clasp, I got what looked like an miscompiled function using uninitialized data- it'll take me a while to find the culprit.

[Bike]

Rather than recomputing the binding site after the fact with dominance, I think it might be easier to just have CST-to-AST mark LET assignments specially (like a subclass of setq-ast, or whatnot).

[karlosz]

Not only would LET assignments need to be marked, but also variables bound by LAMBDA in the syntax tree. Since there may be an arbitrary number of optimizations made between AST->HIR and closure conversion we should consider the pros and cons of each approach: Using the dominance tree: Pros:

• Requires information only from the flow graph. No need to tie closure conversion to the AST-to-HIR or CST-to-AST/Generate-AST pass.
• Easy to extend and make correct when closed over variables which are defined in branches are introduced into the graph from optimizations.
• The flow graph will always have sensible dominance information. However, it is easy to obtain flow graphs which have no sensible scope tree interpretation.
• No additional AST nodes.

Cons:

• Requires linear walks to compute the dominance tree for each function which creates a closure variable.

Creating new AST nodes/marking the syntax tree. Pros:

• No need to compute any information again during closure conversion with respect to putting CREATE-CELLs in the correct place.
• More obviously 'correct' if done immediately after AST-to-HIR compared to a dominator analysis.

Cons:

• Forces us to do closure conversion before any significant graph modifications or optimizations are done, locking us out of potential optimizations that could be made before closure conversion.
• Keeping the syntax tree information updated as the graph changes is both tedious and in general impossible. (see: the no dominating definer case described above) We may end up rewriting the scope tree with dominators anyway if we change the graph too much before closure conversion.
• Requires extra ast nodes.

So it looks like the decision mainly comes down to whether there will be any serious optimizations worth doing besides inlining before closure conversion.

[karlosz]

http://ix.io/1jQi

Clasp backtrace posted by Bike.

[karlosz]

Generalized for no dominating definer

[karlosz]

This PR also makes generate AST closure convert properly

[karlosz]

Rebased with incremental inlining changes implemented by Bike.