Homoiconicity and Meta-circular evaluation

[sirinath]

Code can be stored in data structures and evaluated at both compile or run time.

[slightknack]

I'm not sure about storing code in data-structures: this requires runtime support for AST transformation steps, which is a bit expensive and honestly a tad confusing (as I've said in the past, static compile-time macros are pretty confusing as-is, so imagine the havoc first-class runtime macros could wreak). As for homoiconicity, although Passerine is not a lisp, I think homoiconicity should be possible. This simplest way to make this possible would be to add support for macro pattern matching in language constructs, so, for instance:

syntax 'apply (args -> value) {
    value args
}

double = x -> x * 2
apply (2 -> double)

would become feasible. Because you could match against any Passerine construct, it would be possible to package arbitrary passerine constructs in user-defined data-types, over which a meta-circular evaluator could be implemented.

[slightknack]

This would also allow us ample opportunity to separate desugaring from macro expansion, which has honestly also been a bit of a pain point in the past.

[slightknack]

We're switching to token-based macros, which allows for code evaluation at compile time. I've thought about homoiconicity some more, and posted this on the discord server the other day:

Homoiconicity is interesting, but people usually focus on why it's good for the wrong reasons. The power of homoiconicity in lisp is read, or the ability to parse code into data. It doesn't matter if code is literally data, what's more important is that code is representable as data in a manner that is easily manipulated by the language. Since lisp is great at lists, representing code as lists for macros makes sense and is powerful. Since passerine is great at, say, adts, representing code as adts for macros makes sense and is equally as powerful. In my opinion, to have a successful macro system in the spirit of lisp, the language must have forms, i.e. lists of identifiers that can be matched to make macros. In lisps, this is (a b c) - passerine also operates on forms, which is why we're able to make a powerful lisp-style macro system in a language that - on the surface - does not look like lisp.

[sirinath]

Might be of interest to further investigate:

• Higher-Order Abstract Syntax,
• Parametric Higher-Order Abstract Syntax,

[slightknack]

The eventual plan is to expose the entire compilation pipeline through the FFI, and allow for compile-time evaluation of any expressions through the macro system.