dropbox / pyannotate

Auto-generate PEP-484 annotations
Apache License 2.0
1.43k stars 59 forks source link

Skip functions whose name starts with '<' #22

Closed gvanrossum closed 6 years ago

gvanrossum commented 6 years ago

Those include lambdas, generator functions and comprehensions, and possibly more (the code indicates <module> as a possible name that's filtered out in a later stage).

ethanhs commented 6 years ago

I have a local branch of typycal that filters genexpr, listcomp, setcomp, dictcomp, and module. I believe those are all of the locally scoped frames that can/should be ignored, based on my experiments.

gvanrossum commented 6 years ago

I feel dumb but I have to ask -- what's special about these? Is there something special about the frames or is it a flaw in inspect?

ethanhs commented 6 years ago

I feel dumb but I have to ask -- what's special about these?

Not at all! This is one of the more subtle corners of Python, I only learned about this several months ago.

I have an older version of typycal that filtered these for performance reasons, nothing is gained from doing inspection on them (imo).

I think it causes an error for pyannotate because names in generators in Python 3 are in their own scope eg:

>>>i = 5
>>>[i for i in range(30)]
...
>>>i
5

The error in #21 is you are expecting that a variable in a generator is a local in the exterior frame, when it is not.

This is slightly confounded by the fact that in Python 2 variables are not scoped separately for generators, and i will be 29 after the comprehension in the above example.

So for:

def foo():
    return [i for i in range(30)]

foo.__code__.co_varnames is ('i',) in Python 2, but () on Python 3.

gvanrossum commented 6 years ago

So I'm still not sure how this works but it seems a dict comprehension in Py2 is also a code object, but it seems to be confusing inspect.getargvalues(). E.g. from the original repro:

def method():
    d = {1: {1: 2}}
    return {
        i: {
            (i, k): l
            for k, l in j.items()
        }
        for i, j in d.items()
    }

when tracing into the outer for the first time, inspect.getargvalues() returns

ArgInfo(args=['j'], varargs=None, keywords=None, locals={'.0': <listiterator object at 0x102990d90>})

and in the inner, it returns

ArgInfo(args=[['k', 'l']], varargs=None, keywords=None, locals={'.0': <listiterator object at 0x1029b0990>, 'i': 1})

I think this indicates a discrepancy between the names used for locals and the reported arg names ('j' in the first case, and ['k', 'l'] in the second -- that's right, the arg name is a list of strings, which can happen in Py2 if the arg definition is a tuple).

I'm just going to stop researching and start filtering out all functions whose name starts with '<'.