let*: lexical binding special form that doesn't create a recur point

[alandipert]

(fn (x)
  (let (y (- x 1))
     (if (< x 1)
       'done
       (recur (- x 1))))

Assuming that in the above example let is a macro built atop our fn, (recur (- x 1)) would recur to a fn somewhere in the expansion of the let, not the enclosing, visible, desired fn.

Thus, we need let* so that we have the ability to establish a new lexical environment without also creating a recur frame.

We may also want to support sequence destructuring on the left-hand side of let*'s binding form, which should be a list with an even number of things in it.

[quoll]

My first approach was to do simply translate the following:

(let (a (foo) b (bar))
  body)

Into this:

((fn (a) ((fn (b) body) bar) foo)

But that won't handle the recur correctly. OTOH, it's better than not having a let expression at all.

Implementing made me realize that I don't need to go through the fn machinery, and can just expand the environment. I'm a little hazy on how recur interacts with the environment, so while I'm in there I'll have a look to see if I can jump further out than the current frame.

[quoll]

OK, I have implementation attempt no. 1 committed.

For now it's called let, not let*. It does simple var-to-value bindings. There's no unpacking (I'd like to see vector/array syntax and macros before doing any unpacking).

Implementation calls add_bindings to add to the head of current_env. It iterates along the binding list, adding the results of lisp_eval with the var to the current_env on each iteration. This allows it to build bindings based on the results of previous bindings. e.g. (let (a 1 b (+ a 1) c (+ b 1)) c) => 3

There are no changes to the frames or stack, so the recur should jump back to the same place with no effect.

An outstanding issue is that recur does not reset the current_env, meaning that it accumulates environment state. I need to fix this.

[quoll]

Seems that the environment array was designed with this purpose in mind. So current_env is now being set correctly.

However, some benchmarking has shown that this doesn't actually help. Bash segfaults at the same place even though memory has been supposedly "unset".

But we're doing the right thing now. I'll leave this with the name let until we get macros and can build a fully functional let over let*.

[alandipert]

I think the behavior of our new let matches those of Clojure's binding and that we should rename it to that - it's a way to establish dynamic bindings, vs the let* this issue is about, which establishes lexical bindings that can be closed over.

For example:

((let (x 1) (fn () x))) ;; explodes because x can't be closed over

I propose we rename the let introduced with fe7834a316 to binding. I've reopened this card to track the creation of let* which is a different thing.