Simpler primitives than `syntax-case`

[david-christiansen]

Now that we have datatypes and integers, I think we can do better than the built-in syntax-case operator. Here's a proposal:

First, we reflect our ExprF functor in the kernel language as the equivalent of:

(datatype (List A) ...)
(datatype (Syntax-Contents A)
  (list-syntax (List A))
  (integer-syntax Integer)
  (string-syntax String)
  (identifier String))

This requires making lists built-in like Bools.

Next, we can have two operations: open-syntax : (-> Syntax (Syntax-Contents Syntax)) and close-syntax : (-> Syntax Syntax (Syntax-Contents Syntax) Syntax). The former exposes one level down into the syntax object, and the latter takes some contents along with a source of scopes and a source of position information, creating a syntax object.

This is enough to implement something like our current syntax-case, but it also allows macros to manufacture identifiers (e.g. if we wanted to have a macro that created a datatype's signature functor with a -F suffix). It allows us to greatly simplify our evaluator and expander, because we replace builtins with primitive functions. And it ought to allow for some simpler destructuring patterns in some cases.

Thoughts?

[david-christiansen]

One difficulty with this approach is that we risk inefficiency by treating lists as normal inductive types. This is because recursive functions over list syntax will end up calling open-syntax, analyzing the list, and then calling close-syntax prior to recursion. This means a quadratic overhead as we traverse the list to convert between Haskell [Syntax] and the corresponding Klister value objects.

I suppose it's reasonable enough to just support lists as a built-in primitive type rather than using the datatype mechanism.

[david-christiansen]

For clarity, that would mean we'd have a value constructor

 | ValueList [Value]

to avoid the traversals.

[gelisam]

Isn't this already implemented? We now have a Syntax-Contents datatype, as well as open-syntax and close-syntax functions to convert between Syntax and Syntax-Contents.

Shall we close this and open two new tickets for ValueList and for downgrading syntax-case from a builtin to a user-defined macro?