Open almann opened 4 years ago
parse_text_simple.go#L65-L170 shows where the processing of escapes are happening. It duplicates the lexing code a bit, since the lexing code has to effectively parse the escapes when doing the normalization. That said, I don't see this processing all the escapes (particularly \u
, \U
, and \x
), so I think we need to take a closer look at this logic and its factoring.
While investigating #3. It seems that escapes pass through verbatim in text parsing; they are recognized by the lexer, but not replaced with the runes (code points) they represent.
E.g. adding the following to
lex_test.go
:Fails as follows:
Either I misunderstand the responsibility of the lexer, or this is a bug and we should make sure escapes are being processed correctly.