search

serge-sans-paille / beniget

Extract semantic information about static Python code
BSD 3-Clause "New" or "Revised" License
69 stars 21 forks source link

Using beniget to parse stubs produce errors #55

Open tristanlatr opened 1 year ago

tristanlatr commented 1 year ago

Here are the warnings that beniget reports (with PEP563 enabled) when parsing the typeshed builtins module and all its dependencies. 

W: unbound identifier '_ScandirIterator' at os:734:74
W: unbound identifier 'TraceFunction' at _typeshed:306:59
W: unbound identifier 'Callable' at typing:159:25
W: unbound identifier '_ClassInfo' at builtins:1391:59
W: unbound identifier '_ClassInfo' at builtins:1393:41
W: unbound identifier 'AbstractContextManager' at contextlib:38:33
W: unbound identifier 'ellipsis' at types:606:19
W: unbound identifier 'CDLL' at ctypes:13:31
W: unbound identifier '_CData' at ctypes:14:27
W: unbound identifier '_FuncPointer' at ctypes:88:49
W: unbound identifier 'Array' at ctypes:125:40
W: unbound identifier 'A' at re:167:12
W: unbound identifier 'I' at re:170:17
W: unbound identifier 'L' at re:172:13
W: unbound identifier 'M' at re:174:16
W: unbound identifier 'S' at re:176:13
W: unbound identifier 'X' at re:178:14
W: unbound identifier 'U' at re:180:14
W: unbound identifier 'T' at re:182:15
W: unbound identifier 'Enum' at enum:37:52
W: unbound identifier 'IntFlag' at enum:201:11
W: unbound identifier 'KEEP' at enum:229:48
W: unbound identifier 'Message' at email.message:15:31

Most of them are because cyclic definitions of type aliases, or using a forward reference in a TypeVar bound argument.

We could introduce a new constructor flag that would signify whether we're inside a stub module and change a few behaviour regarding name resolution for a few specific scenarios.

Tell me what you think,

tristanlatr commented 1 year ago

Here are a few of the problematic code fragments:

https://github.com/python/typeshed/blob/81b8211d0edeed0a39253e096aaab86e1699e25b/stdlib/os/__init__.pyi#L733C1-L736C33

class _ScandirIterator(Iterator[DirEntry[AnyStr]], AbstractContextManager[_ScandirIterator[AnyStr]]):
    ...

https://github.com/python/typeshed/blob/81b8211d0edeed0a39253e096aaab86e1699e25b/stdlib/typing.pyi#L182C1-L183C1

_F = TypeVar("_F", bound=Callable[..., Any])
Callable = object
tristanlatr commented 1 year ago

Any thoughts, @serge-sans-paille ?

tristanlatr commented 1 month ago

Here is an implementation: https://github.com/serge-sans-paille/beniget/commit/9ff36a0c7931fd2150609158574fe176e896deaa and a test that includes checking whether the builtins.pyi stub is well processed: https://github.com/serge-sans-paille/beniget/commit/aee2834a54206fc4a34e63d775c79380f0b4e337

tristanlatr commented 4 weeks ago

@serge-sans-paille could you please think whether you’d like to have this feature as part of beniget?