This PR updates the at-least-6-months old problem of keeping the positions.
The old positions required the lexer to keep track of the line and column numbers while it is trying to interpret the tokens. This creates an overhead in the lexer which slows the compiler a lot. This could have been fixed very simply by implementing positions as integers that keep the number of character instead of position structs that keep both the line and column numbers as 64-bit integers. The position integers can then be interpreted by the debugger to display the error information.
To implement this, I removed the line-no and char-no variables. Then whenever a position was required, I simply used the ftell procedure to get the current position.
This simple change allowed me to remove a net of 200 lines of code. And the compiler now compiles itself in ~1.075 seconds which used to take ~1.145 seconds.
This PR updates the at-least-6-months old problem of keeping the positions.
The old positions required the lexer to keep track of the line and column numbers while it is trying to interpret the tokens. This creates an overhead in the lexer which slows the compiler a lot. This could have been fixed very simply by implementing positions as integers that keep the number of character instead of position structs that keep both the line and column numbers as 64-bit integers. The position integers can then be interpreted by the debugger to display the error information.
To implement this, I removed the
line-noandchar-novariables. Then whenever a position was required, I simply used theftellprocedure to get the current position.This simple change allowed me to remove a net of 200 lines of code. And the compiler now compiles itself in ~1.075 seconds which used to take ~1.145 seconds.