Lisp-Forth communication

[larsbrinkhoff]

It would be nice to have a communication protocol between Lisp and Forth. Something in the vein of swank.

But maybe Forth itself is almost enough for this. Just extended with some implementation-specific words.

[larsbrinkhoff]

One particular case is inspecting the state of the Forth interpreter/compiler.

The standard interface to the Forth interpreter works with chunks of parsing areas which can be a full line or a block. Note that parsing words can consume text from the parsing area.

I have an idea that it should be able to add a hook between interpreting each word. E.g. Gforth has a deferred word called PARSED which is called for each token. At this point, it would be possible to insert code which records STATE, >IN, GET-ORDER, GET-CURRENT, HERE, the stack depth, etc.

I believe that with carnal knowledge, some kind of hook like that would be possible to implement for most Forths.

[larsbrinkhoff]

Looking at >IN in between each word would make it possible to e.g. indent IF and POSTPONE IF properly without having special-purpose code for POSTPONE. Or ['] or other parsing words, including comments.

Possibly, the stack depth can be used to find balanced expressions. This could control indentation and sexp cursor motion.

All this of couse only works if the buffer text is sent to the Forth interpreter.

[Reepca]

So how would the buffer text being sent work? Would proper indentation only be assured for text that has already been sent, so for example I use C-M-x and the current definition gets properly indented, or would there be some sort of constant inquiring about information as typing is done (perhaps after each space)? Since immediate words can do pretty much anything, how could information about the effect they would have be retrieved without changing the program state in an unintended way? Maybe if Forths had a sort of sandboxing environment we can run code in wherein it can all be undone easily enough and the changes recorded? MARKER could probably handle most of that, but it could still yield unintuitive results (I'm typing this immediate word that records how many times it's been compiled, but it's randomly higher when I use emacs?)

I'm curious how this would work. I could see static analysis of the stack effect working most of the time, but sending each word to the interpreter and then seeing what happens would only work (I think) if either immediate words didn't have side-effects or we used some elaborate sandboxing scheme.

Am I making sense?

[larsbrinkhoff]

I suppose we should do the right thing for all standard words. But if a user defines a new control-flow word, that would not be handled properly until Emacs receives some information about it.

[larsbrinkhoff]

I have created a new branch with Forth code to record state changes when the text interpreter runs. My idea is that this would be enabled on a line-by-line basis. As you say, immediate words can consume text from the parsing area, so we can't insert code in between words. However, I'm hoping most Forths can be made to run some hook around, or before, or after each word. In the worst case, we can provide a custom text interpreter.

My sample code defines two words, start-introspection and stop-introspection. They would be added around every line being sent to Forth.

[larsbrinkhoff]

Note that I'm talking about analysing the compile-time control stack. Not the runtime stacks.

[Reepca]

Oh, that makes a lot more sense then.