franko / luajit-lang-toolkit

A Lua bytecode compiler written in Lua itself for didactic purposes or for new language implementations
Other
655 stars 91 forks source link

Incorrect escape sequence parsing #15

Closed mpeterv closed 10 years ago

mpeterv commented 10 years ago

Hello! It seems that there are some off-by-one errors in lexer which result in characters being dropped or added near some escape sequences.

\z:

$ cat testcase.lua
"foo\z
bar"
$ luajit run-lexer.lua testcase.lua
TK_string   foozar
TK_eof

(expected TK_string foobar)

\\n:

$ cat testcase.lua
"foo\
bar"
$ luajit run-lexer.lua testcase.lua
TK_string   foo

ar
TK_eof

(expected TK_string foo bar)

\d, \dd (less than 3 digits decimal escape sequences):

$ cat testcase.lua
"\97"
$ luajit run-lexer.lua testcase.lua
luajit: LLT-ERRORtestcase.lua:1: unfinished string near '"a'
stack traceback:
    [C]: in function 'error'
    ./lexer.lua:33: in function 'error_lex'
    ./lexer.lua:43: in function 'lex_error'
    ./lexer.lua:306: in function 'read_string'
    ./lexer.lua:426: in function 'llex'
    ./lexer.lua:459: in function 'next'
    run-lexer.lua:8: in main chunk
    [C]: at 0x0804be80

(expected TK_string a)

franko commented 10 years ago

Hello,

I think I've fixed the problem with the commit above. I didn't have a lot of time to test the modification but now seems to be ok.

mpeterv commented 10 years ago

It does work now, thanks!