Python 3 code generation

[gvanrossum]

We need a flag to generate Python 3 annotations instead of Python 2 style.

[impredicative]

Is a flag necessary? Can the tool do it based on the version of Python being used? Python 3.5+ users probably don't care about Python 2 style; certainly I don't. I can see the diff in other ways.

[gvanrossum]

The flag is not the important part. What we need to write is the code that inserts Python 3 style annotations, and that's not a trivial thing.

[gsemet]

+1 on the python 3.5+ annotation support. Would be so cool !

PS: you mean you mainly use python2 at dropbox?!!!

[kura]

So I guess this is the reason on Python 3.6 I get the following stack trace? Or do I need to create an issue for this?

  File "/home/kura/.virtualenvs/blackhole/bin/pyannotate", line 11, in <module>
    sys.exit(main())
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/__main__.py", line 45, in main
    generate_annotations_json(infile, tf.name)
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/main.py", line 37, in generate_annotations_json
    arg_types, return_type = infer_annotation(item.type_comments)
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/infer.py", line 38, in infer_annotation
    arg_types, return_type = parse_type_comment(comment)
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/parse.py", line 196, in parse_type_comment
    return Parser(comment).parse()
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/parse.py", line 205, in __init__
    self.tokens = tokenize(comment)
  File "/home/kura/.virtualenvs/blackhole/lib/python3.6/site-packages/pyannotate_tools/annotations/parse.py", line 188, in tokenize
    raise ParseError(original)
pyannotate_tools.annotations.parse.ParseError: Invalid type comment: (method-wrapper) -> bool

[gvanrossum]

@kura Can you file that as a separate bug? It's a simple fix, apparently there's an internal type whose name is method-wrapper. We need to figure out what to use instead (perhaps Callable?).

[kura]

@gvanrossum Will do!

[jabdoa2]

It will currently break existing annotation (for mypy):

Before:

    def get_express_config(self, value: str):
        """Parse short config version."""

After:

    def get_express_config(self, value: str):
        # type: (Union[int, str]) -> Dict[str, Any]
        """Parse short config version."""

Mypy dislikes this:

mypy mpf
mpf/config_players/coil_player.py:45: error: Function has duplicate type signatures

[gvanrossum]

Oh, that's a separate bug! It should not attempt to add an annotation if a Py3 annotation is already present.

[gvanrossum]

(Note: as a work-around for the original issue one could use https://github.com/ilevkivskyi/com2ann to post-process the Python 2 style annotations generated by PyAnnotate into Python 3 style annotations.)

[The-Compiler]

FWIW looks like Instagram released MonkeyType which can generate stub files or Python 3 annotations. I haven't tried it yet, though.

[gvanrossum]

I've tried it a little. Maybe we could write a script that converts our type_info.json into there sqlite3 (default) database format.

[omermazig-zz]

com2ann does not seems to work for this purpose. One of its tests specifically requires that it will ignore "#type : str".

I have not found a different way to process the Python 2 style annotations into Python 3 style annotations

[gvanrossum]

Perhaps you could write a script that translates the type_info.json file into the database format supported by MonkeyType, and use their backend?

[omermazig-zz]

Actually, we ended up using the Instagram tool exclusively. It's pretty good, and the use of it also triggered us to migrate our projects to work with python 3.6 (It's one of its requirements). You guys should join forces on this IMO.

[gvanrossum]

I would have loved to, but Instagram only cares about Python 3 and we need Python 2 support for our own use of this tool.

[bluebird75]

I have started working on python 3 style annotations. Work is happening here : https://github.com/bluebird75/pyannotate on the master branch.

This is work in progress. At this point, it only supports return value annotations on very simple case.

[gvanrossum]

Thank you. Have you signed the Dropbox CLA yet? It's very simple: https://opensource.dropbox.com/cla/

On Wed, Jun 27, 2018 at 11:45 PM Bluebird75 notifications@github.com wrote:

I have started working on python 3 style annotations. Work is happening here : https://github.com/bluebird75/pyannotate on the master branch.

This is work in progress. At this point, it only supports return value annotations on very simple case.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dropbox/pyannotate/issues/4#issuecomment-400930065, or mute the thread https://github.com/notifications/unsubscribe-auth/ACwrMsw0Jc-pxNYhvRhmPE4Eo0hp0o_0ks5uBHuegaJpZM4Qd0vR .

-- --Guido van Rossum (python.org/~guido)

[bluebird75]

CLA signed.

Current status is that simple cases of argument are now annotated. So far, it has been easier than I expected. We'll see how it goes...

[gvanrossum]

OK, looking forward to the result!

[gvanrossum]

I've finally released pyannotate 1.0.7. Sorry for the delay!