In accordance with the specification, the suffix of a number should end when it matches a valid suffix. Since the lexer attempts to produce better errors, it treats all alphanumeric characters after a number and until a break as a whole suffix.
Solution
Fixing this would require keeping track of all currently defined macro names, so that if we come across one in a suffix, we produce a number token followed by an ident token, to allow the parser to correctly expand the macro. At the very least, we only need to parse the name of a macro which will be the first token, (if even present). We would also need to parse undefine directives.
glsl-lsp version: 0.0.1
Describe the bug The lexer currently fails to correctly handle one edge case in regards to numeric suffixes.
The following source string:
should result in the following token stream:
however, it currently results in:
In accordance with the specification, the suffix of a number should end when it matches a valid suffix. Since the lexer attempts to produce better errors, it treats all alphanumeric characters after a number and until a break as a whole suffix.
Solution Fixing this would require keeping track of all currently defined macro names, so that if we come across one in a suffix, we produce a number token followed by an ident token, to allow the parser to correctly expand the macro. At the very least, we only need to parse the name of a macro which will be the first token, (if even present). We would also need to parse undefine directives.