hikari-no-yume
commented
9 years ago If we're to make proper "constructors" for abstract token and opcode types, we should probably make a proper pretty-print format, rather than just using JSON.
Also, it occurs to me that once a function body is in the FuncValue format you can't really do further processing on it. So it should probably stay as a plain list initially so the AST can be optimised. That or we could stop hiding the content :p
mathroc
Currently it's a bit of an ad hoc mess and that causes problems. The tokeniser/lexer emits a stream of
*Values, opcodes ({type: "invoke"}and{type: "variable"}) and weird brace placeholders ({type: "["}and{type: "]"}. The "parser" (really, it just groups braces) finds the brace pairs and makesFuncValues out of them, but otherwise the output is the same as the lexer's. The "executor" works on these opcodes and values.I don't like this situation where we're mixing three different things. It means that the executor has to deal with all the different value types, despite this being completely unimportant to it; execution only really needs three true opcodes: push literal, push variable, pop and invoke. It doesn't actually care about what values are pushed onto the stack, just that something's being pushed.
So, we should refactor the lexer and parser. The lexer should emit "tokens", most likely of four types: open brace, close brace, invoke, literal. The parser should emit "opcodes", most likely just the three "true" opcodes I described before.
This would make things less messy.
I tried to do this earlier but gave up because I combined the parser and lexer in the process, which unfortunately breaks both
test.htmland the REPL.