search

pylint-dev / astroid

A common base representation of python source code for pylint and other projects
https://pylint.readthedocs.io/projects/astroid/en/latest/
GNU Lesser General Public License v2.1
529 stars 275 forks source link

[2.0 bugs] A lookup in numpy infinitely recurses due to an interaction between a C extension and Python module #262

Open pylint-bot opened 8 years ago

pylint-bot commented 8 years ago

Originally reported by: BitBucket: ceridwenv, GitHub: @ceridwen

test_numpy_crash fails (at least on Python 3) because, while trying to process an instance of ufunc, it looks up ufunc's type which has a declared module of numpy. It thus imports numpy's __init__.py, which contains from .core import *, and then tries to look up ufunc in that module. Eventually, it encounters a C extension, tries to analyze it using ast_from_object, and finds ufunc, only ufunc, being a type implemented in C, declares that its module is numpy, which sets off the infinite recursion because ast_from_object creates an ImportFrom node for it.

There are a number of possible fixes, I'm not immediately sure which would be best.

pylint-bot commented 8 years ago

Original comment by Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

What fixes do you have in mind?

pylint-bot commented 8 years ago

Original comment by BitBucket: ceridwenv, GitHub: @ceridwen:

I don't have any good ones, I'm afraid. One solution is to simply abandon all attempts to determine which module an object came from and, when building a mock AST for a live object, treat all objects as if they came from the same module. This definitely has its advantages, as handling imports is a major source of complexity in the ast_from_object code and the reason it needs to import manager and has to do shenanigans with lazy objects. However, this loses a lot of information and makes ast_from_object much less useful for analyzing dependencies involving C extensions. Another solution is to try to explicitly detect cycles like this, which will be complicated and hard to get right because the cycles involved can be arbitrarily complicated, and leaves open the question of, assuming it's found a cycle, how does it decide where to put the object?