Extend static modeling language syntax

[marcoct]

The implementation of the SML (static modeling language) is currently broken down into stages:

• an intermediate representation (IR) based on a DAG (https://github.com/probcomp/Gen/blob/master/src/static_ir/static_ir.jl)

• a surface syntax that resembles Julia function definition and generates this IR (https://github.com/probcomp/Gen/blob/master/src/dsl/static.jl)

There are many extensions to the surface syntax that would make it less restricted, that would not require changes to the underlying IR. This includes:

• supporting more general types of expressions including y = @trace(normal(0, 1), :x) + 1 as suggested by @ztangent in https://github.com/probcomp/Gen/issues/90, as well as expressions that include multiple @trace calls on the right-hand-side, and let expressions.

• restricted classes of for loops and list comprehensions (could be converted to a map combinator). (supporting if-else-end branching in this way would require introducing a branching combinator; which should be a separate issue)

• relaxing the static single assignment restriction as suggested by @alex-lew (the language implementation would convert to SSA when generating the IR)

• possibly, supporting macros as suggested in https://github.com/probcomp/Gen/issues/185

I am putting these extensions into one issue, because I think that the SML surface syntax code could probably be redesigned with this set of capabilities in mind, while provisioning for yet additional extensions in the future.

[marcoct]

A revisiting of the SML code should also address the poor error messages that are currently thrown during syntax errors involving @trace: https://github.com/probcomp/Gen/issues/90

[AriMKatz]

Would loops based on foldl help, by preserving more structure? see discussion here https://github.com/JuliaLang/julia/pull/35074#issuecomment-597803494

[ztangent]

I've started taking a stab at this in ztangent:sml-refactor! Right now I've just switched over some of the pattern-matching logic to use @capture from MacroTools so it's a little neater, but I've a good sense of how the parsing code is like now and how it might need to change.

[ztangent]

Partially addressed by #214! Still unaddressed:

• Support for restricted classes of for loops and list comprehensions
• Support for macro calls

[femtomc]

One comment I noticed about the extensions outlined above: it looks like they could be addressed by an SSA form analysis and transformation using the IRTools representation (before constructing the static DAG representation).

Although @ztangent has already introduced a number of well-tested extensions using syntax capture - some of the issues (e.g. capturing control flow with an expansion-time bounds check into a combinator) can most easily be handled when all control flow is branch unless statements between basic blocks. The main benefit of this approach is that everything is de-sugared to branch unless (it might also make the analysis performed with syntax capture easier to express as mutation is explicit).

One thing I’m not sure about with this representation is how macro calls are represented. It may be possible to expand and inline calls in the representation.

Edit: I’ll post a few examples when I’m rested today or tomorrow.

Edit2: After looking at the parsing functions for the static DSL, this suggestion may not be a good fit solution. It’s certainly true that you don’t need more complex machinery for determining if there’s a constant loop bound - assuming it’s just a literal. As a first start, it’s likely overkill to setup the infrastructure to do this.

[marcoct]

Support for @assert statements is needed. (Support should be added at the static IR level).

[femtomc]

@marcoct if you provide a usage example, I could likely do this.

[femtomc]

In particular, at parse time, could move the macro into a Expr(:block, ...) inside the thunk we put in JuliaNode instances. This defers the requirement of having a special node to handle macros, for example.