Incorrect line numbers given in traceback when running a xonsh script

[battaglia01]

Suppose you have a xonsh script with a syntax error in it. A traceback will be generated properly, but it will sometimes incorrectly point to the wrong line in the file if preceded by lines in subprocess mode. The problem seems to be that blank lines and comments are ignored.

For example:

$ cat test.xsh
#!/usr/bin/env xonsh

echo "This is line 3"
print ("This is line 4")
x = "This is a string where I forget the closing quote on line 5
echo "This is line 6"

$ ./test.xsh 
Traceback (most recent call last):
  [trimmed traceback]
SyntaxError: ./test.xsh:3:5: ('code: "This is line 3"',)
echo "This is line 3"
     ^

It says the error is on line 3, character 5, but it's really on line 5.

However, suppose that first echo call is changed to another print call. Then we get the correct line:

$ cat test.xsh
#!/usr/bin/env xonsh

print("This is line 3")
print ("This is line 4")
x = "This is a string where I forget the closing quote on line 5
echo "This is line 6"

$ ./test.xsh 
Traceback (most recent call last):
  [trimmed traceback]
SyntaxError: ./test.xsh:5:4: "
x = "This is a string where I forget the closing quote on line 5
    ^

Lastly, suppose we replace the print call on line 4 with a call in subprocess mode to echo. Then the error is now on line 4:

$ cat test.xsh
#!/usr/bin/env xonsh

print("This is line 3")
echo "This is line 4"
x = "This is a string where I forget the closing quote on line 5
echo "This is line 6"
$ ./test.xsh 
Traceback (most recent call last):
  [trimmed traceback]
SyntaxError: ./test.xsh:4:5: ('code: "This is line 4"',)
echo "This is line 4"
     ^

For reference, here is the entire traceback, which (besides the last part with the line) is identical in all three runs:

$ ./test.xsh 
Traceback (most recent call last):
  File "/usr/local/bin/xonsh", line 3, in <module>
    main()
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 20837, in main
    _failback_to_other_shells(args, err)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 20801, in _failback_to_other_shells
    raise err
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 20835, in main
    return main_xonsh(args)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 20876, in main_xonsh
    loc=None, mode='exec')
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 2438, in run_script_with_cache
    ccode = compile_code(filename, code, execer, glb, loc, mode)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 2397, in compile_code
    filename=filename)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 20054, in compile
    transform=transform)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 20023, in parse
    tree, input = self._parse_ctx_free(input, mode=mode, filename=filename)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 20123, in _parse_ctx_free
    raise original_error from None
  File "/usr/local/lib/python3.5/dist-packages/xonsh/__amalgam__.py", line 20110, in _parse_ctx_free
    debug_level=(self.debug_level > 2))
  File "/usr/local/lib/python3.5/dist-packages/xonsh/parsers/base.py", line 349, in parse
    tree = self.parser.parse(input=s, lexer=self.lexer, debug=debug_level)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/ply/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/ply/ply/yacc.py", line 1199, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/ply/ply/yacc.py", line 193, in call_errorfunc
    r = errorfunc(token)
  File "/usr/local/lib/python3.5/dist-packages/xonsh/parsers/base.py", line 2760, in p_error
    column=p.lexpos))
  File "/usr/local/lib/python3.5/dist-packages/xonsh/parsers/base.py", line 482, in _parse_error
    raise err
SyntaxError: ./test.xsh:4:5: ('code: "This is line 4"',)
echo "This is line 4"
     ^

[scopatz]

Thanks for reporting this @battaglia01! This is rather strange, but reproducible.

[scopatz]

Potential fix in #2738