Bad LexToken(STRING) or invalid javascript?

[nuklea]

$ echo "var SohoPriceSlideData = { min_price : '\3,000' };" | slimit

Illegal character "'" at 1:39 after LexToken(COLON,':',1,37)
Illegal character '\\' at 1:40 after LexToken(COLON,':',1,37)
Illegal character "'" at 1:46 after LexToken(NUMBER,'000',1,43)

Traceback (most recent call last):
  File "/home/nuklea/.virtualenvs/project/bin/slimit", line 9, in <module>
    load_entry_point('slimit==0.7.4', 'console_scripts', 'slimit')()
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/minifier.py", line 69, in main
    text, mangle=options.mangle, mangle_toplevel=options.mangle_toplevel)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/minifier.py", line 38, in minify
    tree = parser.parse(text)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/parser.py", line 116, in p_error
    self._raise_syntax_error(token)
  File "/home/nuklea/.virtualenvs/project/lib/python2.7/site-packages/slimit/parser.py", line 89, in _raise_syntax_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (SEMI, ';') at 1:48 between LexToken(NUMBER,'000',1,43) and LexToken(RBRACE,'}',1,48)

[nuklea]

@dkrnl ты про javascript или про slimit?

[rspivak]

Thanks. I'll look into it.

[qur2]

Just stumbled on the same issue:

echo "a='oui\0'" | slimit

Illegal character "'" at 1:2 after LexToken(EQ,'=',1,1)
Illegal character '\\' at 1:6 after LexToken(ID,'oui',1,3)
Illegal character "'" at 1:8 after LexToken(NUMBER,'0',1,7)
Traceback (most recent call last):
  File "~/.virtualenvs/the-env/bin/slimit", line 8, in <module>
    load_entry_point('slimit==0.7.4', 'console_scripts', 'slimit')()
  File "~/.virtualenvs/the-env/lib/python2.7/site-packages/slimit/minifier.py", line 69, in main
    text, mangle=options.mangle, mangle_toplevel=options.mangle_toplevel)
  File "~/.virtualenvs/the-env/lib/python2.7/site-packages/slimit/minifier.py", line 38, in minify
    tree = parser.parse(text)
  File "~/.virtualenvs/the-env/lib/python2.7/site-packages/slimit/parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)
  File "~/.virtualenvs/the-env/lib/python2.7/site-packages/ply/yacc.py", line 265, in parse
    return self.parseopt_notrack(input,lexer,debug,tracking,tokenfunc)
  File "~/.virtualenvs/the-env/lib/python2.7/site-packages/ply/yacc.py", line 1047, in parseopt_notrack
    tok = self.errorfunc(errtoken)
  File "~/.virtualenvs/the-env/lib/python2.7/site-packages/slimit/parser.py", line 116, in p_error
    self._raise_syntax_error(token)
  File "~/.virtualenvs/the-env/lib/python2.7/site-packages/slimit/parser.py", line 89, in _raise_syntax_error
    self.lexer.prev_token, self.lexer.token())
SyntaxError: Unexpected token (NUMBER, '0') at 1:7 between LexToken(ID,'oui',1,3) and None

However, this gives 3 Illegal character warnings but succeeds:

$ echo "a='\0'" | slimit

Illegal character "'" at 1:2 after LexToken(EQ,'=',1,1)
Illegal character '\\' at 1:3 after LexToken(EQ,'=',1,1)
Illegal character "'" at 1:5 after LexToken(NUMBER,'0',1,4)
a=0;

Same here:

$ echo "'\0'" | slimit

Illegal character "'" at 1:0 after None
Illegal character '\\' at 1:1 after None
Illegal character "'" at 1:3 after LexToken(NUMBER,'0',1,2)
0;

With minimal string, I get the warnings and no output:

$ echo "'\'" | slimit

Illegal character "'" at 1:0 after None
Illegal character '\\' at 1:1 after None
Illegal character "'" at 1:2 after None

[mtdb]

same issue, testing with 'sting'.replace('some', '\0').split('\0');