search

mbdevpl / horast

Human-oriented abstract syntax tree (AST) parser/unparser for Python 3 that doesn't discard comments.
Apache License 2.0
16 stars 6 forks source link

# type: ignore leads to AssertionError #3

Open ZdenekM opened 5 years ago

ZdenekM commented 5 years ago

If the following code:

class foo:
    def bar(self):  # whatever comment 
        pass
    bar.__baz__ = None  # type: ignore

is passed to horast.parse, an AssertionError is raised:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/zdenal/.local/lib/python3.6/site-packages/horast/parser.py", line 19, in parse
    tree = insert_comment_tokens(code, tree, comment_tokens)
  File "/home/zdenal/.local/lib/python3.6/site-packages/horast/ast_comments.py", line 27, in insert_comment_tokens
    path_to_anchor, before_anchor = find_in_ast(code, tree, nodes, scope)
  File "/home/zdenal/.local/lib/python3.6/site-packages/horast/ast_tools.py", line 144, in find_in_ast
    scopes = get_ast_node_scopes(code, nodes)
  File "/home/zdenal/.local/lib/python3.6/site-packages/horast/ast_tools.py", line 72, in get_ast_node_scopes
    assert len(ast_nodes) == len(nodes), (len(ast_nodes), len(nodes))
AssertionError: (12, 13)

If I replace # type: ignore by normal comment, it works. Also, if # whatever comment is removed, it works. Tested with horast version 0.3.1. Any ideas? Many thanks!

mbdevpl commented 5 years ago

Indeed, I can confirm the error. For the time being I've created a branch which reproduces the error - I think the error might boil down to a too-strict assertion, i.e. maybe the code would just work with the assertion disabled. Have you tried running your code with -O (i.e. optimization flag passed to python that disables assertions)? Does it work then, or brakes somewhere else?

ZdenekM commented 5 years ago

The issue seems to be gone with -O flag.

ZdenekM commented 4 years ago

Any chance that this will be fixed soon? :) Thanks...

mbdevpl commented 4 years ago

Yeah, I'm working on this issue (and some related ones that emerged) right now. Feel free to test with current issue_3 branch - should be fixed there, but I'll do some more tests before merging and publishing new version.

ZdenekM commented 4 years ago

I can confirm that the issue is fixed in issue_3 branch. Any chance that it will be merged and released to PyPI soon?

mbdevpl commented 4 years ago

Yes, sorry for the wait! And thanks for the verification. I'll get down to finalize those fixes soon!

ZdenekM commented 4 years ago

@mbdevpl It would be great if you can make a new release containing this fix. The project I'm working on depends on horast and this issue causes some troubles :-) Many thanks!