search

scrapinghub / js2xml

Convert Javascript code to an XML document
MIT License
186 stars 23 forks source link

Parse function breaks when there’s a line ending in a string #32

Closed ariasuni closed 6 years ago

ariasuni commented 6 years ago 
In: js2xml.parse('"test\ntest"')
Illegal character '"' at 1:0 after None
Illegal character '"' at 1:10 after LexToken(ID,'value',1,6)
Out: <Element program at 0x7f023df51648>

In: js2xml.parse('"this is a test\nvalue"')
Illegal character '"' at 1:0 after None
Traceback (most recent call last):

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2910, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-37-e4d16cf2f1c9>", line 1, in <module>
    js2xml.parse('"this is a test\nvalue"')

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/js2xml/__init__.py", line 17, in parse
    tree = _parser.parse(text, debug=debug)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/js2xml/parser.py", line 36, in parse
    result = super(CustomParser, self).parse(text, debug=debug)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/slimit/parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/ply/yacc.py", line 333, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/ply/yacc.py", line 1201, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/ply/yacc.py", line 192, in call_errorfunc
    r = errorfunc(token)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/slimit/parser.py", line 116, in p_error
    self._raise_syntax_error(token)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/slimit/parser.py", line 89, in _raise_syntax_error
    self.lexer.prev_token, self.lexer.token())

  File "<string>", line unknown
SyntaxError: Unexpected token (ID, 'is') at 1:6 between LexToken(THIS,'this',1,1) and LexToken(ID,'a',1,9)

In: js2xml.parse('"this is\na test value"')
Illegal character '"' at 1:0 after LexToken(ID,'a',1,9)
Traceback (most recent call last):

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2910, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)

  File "<ipython-input-38-44296e838d48>", line 1, in <module>
    js2xml.parse('"this is\na test value"')

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/js2xml/__init__.py", line 17, in parse
    tree = _parser.parse(text, debug=debug)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/js2xml/parser.py", line 36, in parse
    result = super(CustomParser, self).parse(text, debug=debug)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/slimit/parser.py", line 93, in parse
    return self.parser.parse(text, lexer=self.lexer, debug=debug)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/ply/yacc.py", line 333, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/ply/yacc.py", line 1201, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/ply/yacc.py", line 192, in call_errorfunc
    r = errorfunc(token)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/slimit/parser.py", line 116, in p_error
    self._raise_syntax_error(token)

  File "/home/ariasuni/projets/magasins_v2/.venv/lib/python3.6/site-packages/slimit/parser.py", line 89, in _raise_syntax_error
    self.lexer.prev_token, self.lexer.token())

  File "<string>", line unknown
SyntaxError: Unexpected token (ID, 'is') at 1:6 between LexToken(THIS,'this',1,1) and LexToken(ID,'a',1,9)
ariasuni commented 6 years ago

I tried to use slimit directly and noticed it has the same bug…

As it’s an upstream bug, I opened a bug report there: https://github.com/rspivak/slimit/issues/100.

ariasuni commented 6 years ago

Following the discussion at the slimit bug tracker, I’m closing this.