google / pytype

A static type analyzer for Python code
https://google.github.io/pytype
Other
4.71k stars 274 forks source link

ValueError: Unresolved class: '_typeshed.StrOrBytesPath' (--protocols) #1066

Open cebtenzzre opened 2 years ago

cebtenzzre commented 2 years ago

Software versions

Python 3.9.7 pytype 2021.11.24 (not present on 2021.11.18)

Reproducer

import os

Description

Running pytype --protocols repr.py generates a repr.pyi with this content:


from typing import Any
def __getattr__(name: Any) -> Any: ...
# Caught error in pytype: Unresolved class: '_typeshed.StrOrBytesPath'
# Traceback (most recent call last):
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/io.py", line 146, in check_or_generate_pyi
#     errorlog, result, ast = generate_pyi(
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/io.py", line 104, in generate_pyi
#     errorlog, (mod, builtins) = _call(analyze.infer_types, src, options, loader)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/io.py", line 58, in wrapper
#     return f(*args, **kwargs)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/io.py", line 70, in _call
#     return errorlog, analyze_types(
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/analyze.py", line 121, in infer_types
#     ast = convert_structural.convert_pytd(ast, builtins_pytd, protocols_pytd)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/convert_structural.py", line 280, in convert_pytd
#     mapping, result = solve(ast, builtins_pytd, protocols_pytd)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/convert_structural.py", line 208, in solve
#     builtins_pytd = visitors.LookupClasses(builtins_pytd)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/pytd/visitors.py", line 237, in LookupClasses
#     target.Visit(VerifyLookup(ignore_late_types))
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/pytd/parse/node.py", line 92, in Visit
#     return _Visit(self, visitor, *args, **kwargs)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/pytd/parse/node.py", line 110, in _Visit
#     return _VisitNode(node, visitor, *args, **kwargs)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/pytd/parse/node.py", line 202, in _VisitNode
#     new_child = _VisitNode(child, visitor, *args, **kwargs)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/pytd/parse/node.py", line 161, in _VisitNode
#     new_child = _VisitNode(child, visitor, *args, **kwargs)
# <snip>
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/pytd/parse/node.py", line 161, in _VisitNode
#     new_child = _VisitNode(child, visitor, *args, **kwargs)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/pytd/parse/node.py", line 184, in _VisitNode
#     status = visitor.Enter(node, *args, **kwargs)
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/pytd/base_visitor.py", line 199, in Enter
#     return self.enter_functions[node.__class__.__name__](
#   File "/home/cebtenzzre/.local/lib/python3.9/site-packages/pytype/pytd/visitors.py", line 257, in EnterClassType
#     raise ValueError("Unresolved class: %r" % node.name)
# ValueError: Unresolved class: '_typeshed.StrOrBytesPath'

Stepping through the call stack with pdb reveals that pytd.visitors.LookupClasses is trying to parse posix.posix_spawn, which has an argv parameter typed as _ExecVArgs, which references StrOrBytesPath. Reverting https://github.com/python/typeshed/commit/f30d6f6fd0444690404b00420b3f843c854b03ac or removing the _ExecVArgs import (and replacing references to it with Any) makes this particular backtrace go away, leaving the usual is_partial backtrace (which I apparently worked around but never reported).

rchen152 commented 2 years ago

I bet that this is another circular dependency issue; we've had a lot of problems with resolving types in os in the past due to this.