Docstrings messed up in annotated files

[marcoeilers]

The output files we produce that contain the original program with added type annotations has messed up docstrings. For example, we get something like this:

'\n    Flattens a dict of lists, i.e., concatenates all lists for the same keys.\n    '

which presumably lookes like this before:

'''
Flattens a dict of lists, i.e., concatenates all lists for the same keys.
'''

[mostafa-abdullah]

This has to do with the astunparse package.

[mostafa-abdullah]

I think it also gets rid of normal comments, because the AST itself doesn't contain the comments

[marcoeilers]

Yeah normal comments disappear. Okay, so this means there's basically nothing we can do about it, right?

[mostafa-abdullah]

For the comments, yes there is nothing we can do because the builtin ast parser itself gets rid of the comments. For the docstrings, I tried to look for a way to configure the astunparse package to keep the docstrings but was in vain. Maybe in upcoming releases they will fix it.

[mostafa-abdullah]

Well after inspecting the AST, the strings 'abc', "abc" and """abc""" correspond to the exact same node in the AST given by the builtin ast module. So it will be impossible for the astunparse unparser to figure out which of the strings in the AST is actually a docstring.

[marcoeilers]

Okay. Not our problem then.