Failed to recognize semicolons on python file

[williamjamir]

Hi,

The parser does not recognize the semicolons on python files. I don't know how often people use semicolons as separator on python, but I think that it's a good idea to provide some support for such cases =)

Here it's a small reproducible example of the error:

File: a=(1+1);print(a)

Script

import pasta
from pasta.augment import rename

path = r'test.py'

with open(path) as file:
        tree = pasta.parse(file.read())

Error:

Traceback (most recent call last):

  File "script.py", line 7, in <module>
    tree = pasta.parse(file.read())
  File "/opt/pasta/pasta/__init__.py", line 25, in parse
    annotator.visit(t)
  File "/opt/pasta/pasta/base/annotate.py", line 1044, in visit
    super(AstAnnotator, self).visit(node)
  File "/opt/pasta/pasta/base/annotate.py", line 115, in visit
    super(BaseVisitor, self).visit(node)
  File "/home/william/miniconda3/envs/google_pasta_py34/lib/python3.4/ast.py", line 245, in visit
    return visitor(node)
  File "/opt/pasta/pasta/base/annotate.py", line 79, in wrapped
    f(self, node, *args, **kwargs)
  File "/opt/pasta/pasta/base/annotate.py", line 166, in visit_Module
    self.generic_visit(node)
  File "/home/william/miniconda3/envs/google_pasta_py34/lib/python3.4/ast.py", line 253, in generic_visit
    self.visit(item)
  File "/opt/pasta/pasta/base/annotate.py", line 1044, in visit
    super(AstAnnotator, self).visit(node)
  File "/opt/pasta/pasta/base/annotate.py", line 115, in visit
    super(BaseVisitor, self).visit(node)
  File "/home/william/miniconda3/envs/google_pasta_py34/lib/python3.4/ast.py", line 245, in visit
    return visitor(node)
  File "/opt/pasta/pasta/base/annotate.py", line 43, in wrapped
    f(self, node, *args, **kwargs)
  File "/opt/pasta/pasta/base/annotate.py", line 509, in visit_Expr
    self.visit(node.value)
  File "/opt/pasta/pasta/base/annotate.py", line 1044, in visit
    super(AstAnnotator, self).visit(node)
  File "/opt/pasta/pasta/base/annotate.py", line 115, in visit
    super(BaseVisitor, self).visit(node)
  File "/home/william/miniconda3/envs/google_pasta_py34/lib/python3.4/ast.py", line 245, in visit
    return visitor(node)
  File "/opt/pasta/pasta/base/annotate.py", line 43, in wrapped
    f(self, node, *args, **kwargs)
  File "/opt/pasta/pasta/base/annotate.py", line 626, in visit_Call
    self.visit(node.func)
  File "/opt/pasta/pasta/base/annotate.py", line 1044, in visit
    super(AstAnnotator, self).visit(node)
  File "/opt/pasta/pasta/base/annotate.py", line 115, in visit
    super(BaseVisitor, self).visit(node)
  File "/home/william/miniconda3/envs/google_pasta_py34/lib/python3.4/ast.py", line 245, in visit
    return visitor(node)
  File "/opt/pasta/pasta/base/annotate.py", line 43, in wrapped
    f(self, node, *args, **kwargs)
  File "/opt/pasta/pasta/base/annotate.py", line 743, in visit_Name
    self.token(node.id)
  File "/opt/pasta/pasta/base/annotate.py", line 1094, in token
    token_val, token.src, token.start[0], token.line))
pasta.base.annotate.AnnotationError: Expected 'print' but found ';'
line 1: a=(1+1);print(a)

[soupytwist]

Wow... this is a huge oversight. Maybe not that difficult to fix though. Thanks for reporting it!

[soupytwist]

Fixed in 4011e8f.