t3rn0 / ast-comments

Extension to the built-in ast module. Finds comments in source code and adds them to the parsed tree.
MIT License
31 stars 10 forks source link

Break docstring if comment is above it #16

Closed hovi closed 1 year ago

hovi commented 1 year ago

Hi, the comments usually work great, but when there is comment directly above docstring, the docstring is broken, illustrated by this tests:

import ast_comments as ast
import pytest

comment_below = '''
"""
Multiline string
"""
# comment
'''.strip()

def test_this_one_is_fine():
    assert comment_below == ast.unparse(ast.parse(comment_below))

comment_above = '''
# comment
"""
Multiline string
"""
'''.strip()

@pytest.mark.xfail(reason="This one is failing")
def test_this_one_is_failing():
    assert comment_above == ast.unparse(ast.parse(comment_above))
t3rn0 commented 1 year ago

Hello @hovi. I think it's expected behavior. Docstring is a simple string constant + it's the first Constant node in a Module, FunctionDef, etc. (ast._Unparser.get_raw_docstring) (actually there is no need for a multiline string here: any string will be considered as a docstring). When you place a node before a docstring (any node, including Comment node), the string ceases to be a docstring. That's why it's unparsed as a regular string. I'm not sure that Comment nodes should change this rule.

Also, parse-unparse roundtrips with your examples still provide (tree-)equivalent source code.

import test_unparse

test_unparse._test_unparse(comment_below)   # no AssertionError
test_unparse._test_unparse(comment_above)   # no AssertionError