corth-lang / Corth

A self-hosted stack based language like Forth
MIT License
7 stars 1 forks source link

Storing tokens for bootstrapping #45

Open HuseyinSimsek7904 opened 6 months ago

HuseyinSimsek7904 commented 6 months ago

When the compiler is bootstrapping, it has to regenerate all of the tokens each time it iterates. This could be avoided if the tokens were collected at first, then in each iteration the tokens were reused. This would improve the compilation time.

However, there should also be an option to make the compiler recreate the tokens every time. This is important because if the lexer was changed that would require the tokens to be recollected at every iteration which would otherwise create problems.