search

google / pytype

A static type analyzer for Python code
https://google.github.io/pytype
Other
4.71k stars 274 forks source link

Unexpected import-error failure #1434

Open fvdnabee opened 1 year ago

fvdnabee commented 1 year ago

For the following snippet:

import google.protobuf.message

def print_fields(msg: google.protobuf.message.Message):
    print(msg.DESCRIPTOR.fields)

pytype (tested 2023.5.24 and 2023.5.8), to my surprise, fails with an import-error:

❯ pytype main.py
Computing dependencies
Analyzing 1 sources with 0 local dependencies
ninja: Entering directory `.pytype'
[1/1] check main
FAILED: /home/fvdnabee/tmp/pytype_importerror/.pytype/pyi/main.pyi
/home/fvdnabee/tmp/pytype_importerror/.venv/bin/python -m pytype.single --imports_info /home/fvdnabee/tmp/pytype_importerror/.pytype/imports/main.imports --module-name main --platform linux -V 3.10 -o /home/fvdnabee/tmp/pytype_importerror/.pytype/pyi/main.pyi --analyze-annotated --nofail --quick /home/fvdnabee/tmp/pytype_importerror/main.py
File "/home/fvdnabee/tmp/pytype_importerror/main.py", line 1, in <module>: Can't find module 'google'. [import-error]

For more details, see https://google.github.io/pytype/errors.html#import-error
ninja: build stopped: subcommand failed.
Leaving directory '.pytype'

The interpreter can import google just fine however:

❯ python -c 'import google; print("success")' || echo 'import failed'
success

Is this a known limitation of pytype of the type stubs? Is there a work-around (apart from disabling the import-error)? I tried adding an init file at ../lib/python3.10/site-packages/google/__init__.py, but the errors persists. I've pushed my example here.

Verbose output follows:

`pytype -v 2 main.py` ```shell ❯ pytype -v2 main.py INFO No config file found. Using default configuration. INFO Configuration: exclude = set() inputs = {'/home/fvdnabee/tmp/pytype_import_error/main.py'} keep_going = False jobs = 1 output = '/home/fvdnabee/tmp/pytype_import_error/.pytype' platform = 'linux' pythonpath = ['/home/fvdnabee/tmp/pytype_import_error'] python_version = '3.10' disable = [] report_errors = True overriding_parameter_count_checks = False overriding_renamed_parameter_count_checks = False use_enum_overlay = False strict_none_binding = False use_fiddle_overlay = False no_return_any = False enable_cached_property = False precise_return = False protocols = False strict_import = False strict_parameter_checks = False strict_primitive_comparisons = False Computing dependencies INFO Source tree: source: /home/fvdnabee/tmp/pytype_import_error/main.py deps: [/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/descriptor.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/descriptor_pb2.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/containers.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/enum_type_wrapper.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/extension_dict.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/message.pyi] source: [/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/descriptor.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/descriptor_pb2.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/containers.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/enum_type_wrapper.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/extension_dict.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/message.pyi] deps: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/builtins/typing.pytd [/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/_collections_abc.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/_ctypes.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/_typeshed/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/_winapi.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/abc.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/array.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/codecs.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/collections/abc.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/contextlib.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/ctypes/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/charset.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/contentmanager.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/header.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/message.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/policy.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/genericpath.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/importlib/abc.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/importlib/machinery.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/importlib/metadata/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/importlib/metadata/_meta.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/io.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/mmap.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/ntpath.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/os/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/os/path.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/pathlib.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/pickle.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/posixpath.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/re.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/sre_compile.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/sre_constants.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/sre_parse.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/subprocess.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/types.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/typing_extensions.pyi] /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/message_listener.pyi /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/python_message.pyi /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/type_checkers.pyi source: [/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/_collections_abc.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/_ctypes.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/_typeshed/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/_winapi.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/abc.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/array.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/codecs.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/collections/abc.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/contextlib.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/ctypes/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/charset.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/contentmanager.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/header.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/message.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/policy.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/genericpath.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/importlib/abc.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/importlib/machinery.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/importlib/metadata/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/importlib/metadata/_meta.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/io.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/mmap.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/ntpath.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/os/__init__.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/os/path.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/pathlib.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/pickle.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/posixpath.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/re.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/sre_compile.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/sre_constants.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/sre_parse.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/subprocess.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/types.pyi->/home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/typing_extensions.pyi] deps: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/builtins/typing.pytd /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/builtins/builtins.pytd /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/stdlib/collections/__init__.pytd /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/errors.pyi /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/stdlib/enum.pytd source: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/message_listener.pyi source: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/python_message.pyi source: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stubs/protobuf/google/protobuf/internal/type_checkers.pyi deps: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/builtins/typing.pytd source: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/builtins/builtins.pytd deps: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/builtins/typing.pytd source: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/stdlib/collections/__init__.pytd deps: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/builtins/typing.pytd source: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/typeshed/stdlib/email/errors.pyi source: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/stdlib/enum.pytd deps: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/builtins/typing.pytd source: /home/fvdnabee/tmp/pytype_import_error/.venv/lib/python3.10/site-packages/pytype/stubs/builtins/typing.pytd INFO ------------- Starting pytype run. ------------- INFO infer command: /home/fvdnabee/tmp/pytype_import_error/.venv/bin/python -m pytype.single --imports_info $imports --module-name $module --platform linux -V 3.10 -o $out --no-report-errors --nofail --quick $in INFO check command: /home/fvdnabee/tmp/pytype_import_error/.venv/bin/python -m pytype.single --imports_info $imports --module-name $module --platform linux -V 3.10 -o $out --analyze-annotated --nofail --quick $in INFO check main imports: /home/fvdnabee/tmp/pytype_import_error/.pytype/imports/main.imports deps: () output: /home/fvdnabee/tmp/pytype_import_error/.pytype/pyi/main.pyi Analyzing 1 sources with 0 local dependencies ninja: Entering directory `.pytype' [1/1] /home/fvdnabee/tmp/pytype_import_error/.venv/bin/python -m pytype.single --imports_info /home/fvdnabee/tmp/pytype_import_error/.pytype/imports/main.imports --module-name main --platform linux -V 3.10 -o /home/fvdnabee/tmp/pytype_import_error/.pytype/pyi/main.pyi --analyze-annotated --nofail --quick /home/fvdnabee/tmp/pytype_import_error/main.py FAILED: /home/fvdnabee/tmp/pytype_import_error/.pytype/pyi/main.pyi /home/fvdnabee/tmp/pytype_import_error/.venv/bin/python -m pytype.single --imports_info /home/fvdnabee/tmp/pytype_import_error/.pytype/imports/main.imports --module-name main --platform linux -V 3.10 -o /home/fvdnabee/tmp/pytype_import_error/.pytype/pyi/main.pyi --analyze-annotated --nofail --quick /home/fvdnabee/tmp/pytype_import_error/main.py File "/home/fvdnabee/tmp/pytype_import_error/main.py", line 1, in : Can't find module 'google'. [import-error] For more details, see https://google.github.io/pytype/errors.html#import-error ninja: build stopped: subcommand failed. Leaving directory '.pytype' ```
fvdnabee commented 1 year ago

Using a from xxx import statement is a work-around. I still feel like the OP is a bug though.

diff --git a/main.py b/main.py
index 781b723..fc5f5e7 100644
--- a/main.py
+++ b/main.py
@@ -1,4 +1,4 @@
-import google.protobuf.message
+from google.protobuf import message

-def print_fields(msg: google.protobuf.message.Message):
+def print_fields(msg: message.Message):
     print(msg.DESCRIPTOR.fields)

This is also a work-around:

diff --git a/main.py b/main.py
index 781b723..c326060 100644
--- a/main.py
+++ b/main.py
@@ -1,4 +1,6 @@
 import google.protobuf.message
+import google.protobuf.pyext

 def print_fields(msg: google.protobuf.message.Message):
     print(msg.DESCRIPTOR.fields)

As well as:

diff --git a/main.py b/main.py
index 781b723..36b51f5 100644
--- a/main.py
+++ b/main.py
@@ -1,4 +1,6 @@
+import google
 import google.protobuf.message

+
 def print_fields(msg: google.protobuf.message.Message):
     print(msg.DESCRIPTOR.fields)