higherorderfunctor
commented
5 years ago I am also seeing this issue when running with the plugin. I cannot reproduce with the plugin disabled. It is very intermittent, however, when hooked to a linter, it triggers quite often. I resorted to turning off incremental mode, but this quite slow while linting. The first manual run after clearing the cache is always successful. Walking lookup.py:lookup_fully_qualified, the modules dict appears to be lacking some of the project's sub-modules causing the walk to fail.
Below is some output after clearing the cache and running manually to rule out concurrency issues with the linter. After stepping into one of my root modules, it fails to find the models sub-module.
[mypy]
plugins = mypy_django_plugin.main
python_version = 3.7
check_untyped_defs = True
disallow_any_decorated = True
disallow_any_generics = True
disallow_any_unimported = True
disallow_incomplete_defs = True
disallow_subclassing_any = True
disallow_untyped_defs = True
follow_imports = silent
ignore_missing_imports = True
show_error_context = True
warn_redundant_casts = True
warn_return_any = True
warn_unused_ignores = True
disallow_untyped_decorators = True
#incremental=False
[mypy_django_plugin]
django_settings = config.settings.developmentTraceback (most recent call last):
File "/home/higherorderfunctor/workspace/myproject/ENV/bin/mypy", line 24, in <module>
sys.exit(console_entry())
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/__main__.py", line 7, in console_entry
main(None)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/main.py", line 84, in main
res = build.build(sources, options, None, flush_errors, fscache)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/build.py", line 163, in build
result = _build(sources, options, alt_lib_path, flush_errors, fscache)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/build.py", line 218, in _build
graph = dispatch(sources, manager)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/build.py", line 2461, in dispatch
process_graph(graph, manager)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/build.py", line 2754, in process_graph
process_fresh_modules(graph, prev_scc, manager)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/build.py", line 2832, in process_fresh_modules
graph[id].fix_cross_refs()
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/build.py", line 1803, in fix_cross_refs
self.options.use_fine_grained_cache)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 25, in fixup_module
node_fixer.visit_symbol_table(tree.names)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 88, in visit_symbol_table
self.visit_type_info(value.node)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 45, in visit_type_info
self.visit_symbol_table(info.names)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 90, in visit_symbol_table
value.node.accept(self)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/nodes.py", line 832, in accept
return visitor.visit_var(self)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 133, in visit_var
v.type.accept(self.type_fixer)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/types.py", line 625, in accept
return visitor.visit_instance(self)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 157, in visit_instance
a.accept(self)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/types.py", line 625, in accept
return visitor.visit_instance(self)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 150, in visit_instance
inst.type = lookup_qualified_typeinfo(self.modules, type_ref, self.allow_missing)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 240, in lookup_qualified_typeinfo
node = lookup_qualified(modules, name, allow_missing)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 254, in lookup_qualified
stnode = lookup_qualified_stnode(modules, name, allow_missing)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/fixup.py", line 263, in lookup_qualified_stnode
return lookup_fully_qualified(name, modules, raise_on_missing=not allow_missing)
File "/home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/lookup.py", line 47, in lookup_fully_qualified
assert key in names, "Cannot find %s for %s" % (key, name)
AssertionError: Cannot find models for ipam.models.block.Block
> /home/higherorderfunctor/workspace/myproject/ENV/lib/python3.7/site-packages/mypy/lookup.py(47)lookup_fully_qualified()
-> assert key in names, "Cannot find %s for %s" % (key, name)
(Pdb) import pprint
(Pdb) pprint.PrettyPrinter(indent=4).pprint(sorted(modules.keys()))
[ 'Crypto',
'Crypto.Cipher',
'Crypto.Cipher.Blowfish',
'Crypto.Cipher.blockalgo',
'__future__',
'_ast',
'_importlib_modulespec',
'_markupbase',
'abc',
'argparse',
'ast',
'attr',
'attr.converters',
'attr.exceptions',
'attr.filters',
'attr.validators',
'builtins',
'codecs',
'collections',
'collections.abc',
'contextlib',
'copy',
'core',
'core.admin',
'core.exceptions',
'core.models.readonly',
'core.routers',
'core.tests.utils',
'core.utils',
'core.utils.datatypes',
'core.utils.datatypes.either',
'core.utils.datatypes.lift',
'core.utils.modeltools',
'dataclasses',
'datetime',
'decimal',
'django',
'django.apps',
'django.apps.config',
'django.apps.registry',
'django.conf',
'django.conf.global_settings',
'django.conf.urls',
'django.contrib',
'django.contrib.admin.decorators',
'django.contrib.admin.models',
'django.contrib.admin.views',
'django.contrib.auth',
'django.contrib.auth.backends',
'django.contrib.auth.base_user',
'django.contrib.auth.models',
'django.contrib.auth.signals',
'django.contrib.auth.tokens',
'django.contrib.auth.validators',
'django.contrib.contenttypes',
'django.contrib.contenttypes.fields',
'django.contrib.contenttypes.models',
'django.contrib.messages.constants',
'django.contrib.messages.storage.base',
'django.contrib.postgres',
'django.contrib.postgres.fields.array',
'django.contrib.postgres.fields.citext',
'django.contrib.postgres.fields.hstore',
'django.contrib.postgres.fields.jsonb',
'django.contrib.postgres.fields.mixins',
'django.contrib.postgres.fields.ranges',
'django.contrib.sessions',
'django.contrib.sessions.backends',
'django.contrib.sessions.backends.base',
'django.core',
'django.core.cache',
'django.core.cache.backends',
'django.core.cache.backends.base',
'django.core.checks',
'django.core.checks.messages',
'django.core.checks.registry',
'django.core.exceptions',
'django.core.files',
'django.core.files.base',
'django.core.files.images',
'django.core.files.storage',
'django.core.files.temp',
'django.core.files.uploadedfile',
'django.core.files.uploadhandler',
'django.core.files.utils',
'django.core.handlers',
'django.core.handlers.base',
'django.core.handlers.wsgi',
'django.core.management',
'django.core.management.base',
'django.core.management.color',
'django.core.paginator',
'django.core.serializers',
'django.core.serializers.base',
'django.core.serializers.json',
'django.core.serializers.python',
'django.core.servers',
'django.core.servers.basehttp',
'django.core.validators',
'django.core.wsgi',
'django.db',
'django.db.backends',
'django.db.backends.base',
'django.db.backends.base.base',
'django.db.backends.base.client',
'django.db.backends.base.creation',
'django.db.backends.base.features',
'django.db.backends.base.introspection',
'django.db.backends.base.schema',
'django.db.backends.base.validation',
'django.db.backends.ddl_references',
'django.db.backends.sqlite3',
'django.db.backends.sqlite3.base',
'django.db.backends.utils',
'django.db.migrations',
'django.db.migrations.migration',
'django.db.migrations.operations',
'django.db.migrations.operations.base',
'django.db.migrations.operations.fields',
'django.db.migrations.operations.models',
'django.db.migrations.operations.special',
'django.db.migrations.state',
'django.db.models',
'django.db.models.aggregates',
'django.db.models.base',
'django.db.models.deletion',
'django.db.models.expressions',
'django.db.models.fields',
'django.db.models.fields.files',
'django.db.models.fields.mixins',
'django.db.models.fields.proxy',
'django.db.models.fields.related',
'django.db.models.fields.related_descriptors',
'django.db.models.fields.related_lookups',
'django.db.models.fields.reverse_related',
'django.db.models.functions.comparison',
'django.db.models.functions.datetime',
'django.db.models.functions.text',
'django.db.models.functions.window',
'django.db.models.indexes',
'django.db.models.lookups',
'django.db.models.manager',
'django.db.models.options',
'django.db.models.query',
'django.db.models.query_utils',
'django.db.models.signals',
'django.db.models.sql',
'django.db.models.sql.compiler',
'django.db.models.sql.datastructures',
'django.db.models.sql.query',
'django.db.models.sql.subqueries',
'django.db.models.sql.where',
'django.db.transaction',
'django.db.utils',
'django.dispatch',
'django.dispatch.dispatcher',
'django.forms',
'django.forms.boundfield',
'django.forms.fields',
'django.forms.forms',
'django.forms.formsets',
'django.forms.models',
'django.forms.renderers',
'django.forms.utils',
'django.forms.widgets',
'django.http',
'django.http.cookie',
'django.http.request',
'django.http.response',
'django.middleware',
'django.template',
'django.template.backends',
'django.template.backends.base',
'django.template.base',
'django.template.context',
'django.template.defaultfilters',
'django.template.defaulttags',
'django.template.engine',
'django.template.exceptions',
'django.template.library',
'django.template.loader_tags',
'django.template.loaders',
'django.template.loaders.base',
'django.template.response',
'django.template.smartif',
'django.template.utils',
'django.test',
'django.test.client',
'django.test.runner',
'django.test.testcases',
'django.test.utils',
'django.urls',
'django.urls.base',
'django.urls.conf',
'django.urls.converters',
'django.urls.exceptions',
'django.urls.resolvers',
'django.urls.utils',
'django.utils',
'django.utils.datastructures',
'django.utils.dateparse',
'django.utils.datetime_safe',
'django.utils.deprecation',
'django.utils.encoding',
'django.utils.functional',
'django.utils.html',
'django.utils.module_loading',
'django.utils.safestring',
'django.utils.six',
'django.utils.text',
'django.utils.timezone',
'django.utils.translation.trans_real',
'django.utils.tree',
'django.utils.version',
'django.views.decorators',
'django.views.decorators.http',
'django.views.generic.base',
'email',
'email.charset',
'email.contentmanager',
'email.errors',
'email.header',
'email.message',
'email.policy',
'enum',
'functools',
'gettext',
'html',
'html.parser',
'http',
'http.cookies',
'http.server',
'import_export',
'import_export.formats',
'import_export.formats.base_formats',
'import_export.instance_loaders',
'import_export.results',
'import_export.signals',
'import_export.tmp_storages',
'import_export.widgets',
'importlib',
'importlib.abc',
'io',
'ipam',
'ipam.admin',
'ipam.admin.models',
'ipam.admin.models.netelement_snmp_admin',
'ipam.admin.models.snmp_admin',
'ipam.admin.resources',
'ipam.apiv1',
'ipam.apiv1.serializers',
'ipam.apiv1.urls',
'ipam.apiv1.views',
'ipam.apiv1.views.netelement',
'ipam.migrations',
'ipam.models.rir_organization',
'ipam.tests',
'ipam.tests.base',
'ipam.tests.models',
'ipam.tests.models.base',
'ipam.tests.models.test_admin',
'ipam.tests.models.test_agent',
'ipam.tests.models.test_container',
'ipam.tests.models.test_container_closure',
'ipam.tests.models.test_netelement',
'ipam.utils',
'itertools',
'json',
'json.decoder',
'json.encoder',
'logging',
'mmap',
'model_utils.choices',
'model_utils.fields',
'model_utils.managers',
'model_utils.tracker',
'mypy_extensions',
'numbers',
'operator',
'os',
'os.path',
'posix',
'pprint',
're',
'rest_framework',
'rest_framework.schemas.generators',
'rest_framework.settings',
'rest_framework.utils',
'rest_framework.utils.field_mapping',
'rest_framework.utils.model_meta',
'simple_history.exceptions',
'simple_history.manager',
'simple_history.signals',
'socket',
'socketserver',
'sqlite3',
'sqlite3.dbapi2',
'string',
'sys',
'tempfile',
'threading',
'time',
'traceback',
'types',
'typing',
'typing_extensions',
'unittest',
'unittest.mock',
'uuid',
'warnings',
'wsgiref',
'wsgiref.handlers',
'wsgiref.headers',
'wsgiref.simple_server',
'wsgiref.types',
'wsgiref.util']
(Pdb) key
'models'
(Pdb) pprint.PrettyPrinter(indent=4).pprint(sorted(names))
[ '__doc__',
'__file__',
'__name__',
'__package__',
'default_app_config',
'utils']
(Pdb) print(modules.get(head))
MypyFile:1(
/home/higherorderfunctor/workspace/myproject/ipam/__init__.py)
(Pdb) print(modules.get(head).names)
SymbolTable(
default_app_config : Gdef/Var (ipam.default_app_config) : builtins.str
utils : Gdef/MypyFile (ipam.utils))
If I install
pytest-xdistand run tests withpytest -n8, sometimes (half the time?) the testmodels_imported_inside_init_file_one_to_one_fieldfails with a failure like: