Currently line-continuation characters are not syntax highlighted at all - we don't even produce any tokens that cover their spans in the source code. Obviously, it'd be nice to properly colour them.
Line-continuation characters are "transparent" from the perspective of the token stream, but that means they aren't iterated over by the parser (and hence no syntax tokens are produced for them).
We obviously want to colour them properly, but we can't have them be tokens in the token stream, so the only other solution is to manually track the spans chronologically within the lexer (as a vector probably), and return this vector along with the metadata. Then the parser can insert the tokens in the correct places during syntax highlighting.
Currently line-continuation characters are not syntax highlighted at all - we don't even produce any tokens that cover their spans in the source code. Obviously, it'd be nice to properly colour them.
Line-continuation characters are "transparent" from the perspective of the token stream, but that means they aren't iterated over by the parser (and hence no syntax tokens are produced for them).
We obviously want to colour them properly, but we can't have them be tokens in the token stream, so the only other solution is to manually track the spans chronologically within the lexer (as a vector probably), and return this vector along with the metadata. Then the parser can insert the tokens in the correct places during syntax highlighting.