search

pylint-dev / pylint

It's not just a linter that annoys you!
https://pylint.readthedocs.io/en/latest/
GNU General Public License v2.0
5.34k stars 1.14k forks source link

AttributeError: module <module> has no attribute '__path__' #7592

Closed hnhegde closed 2 years ago

hnhegde commented 2 years ago

Bug description

The crash gets triggered by any file that contains:

from decouple import config

I am using python-decouple version 3.6. 

$ grep python-decouple requirements.txt 
python-decouple==3.6

pylint crashed with a AstroidError and with the following stacktrace:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 590, in _spec_from_modpath
    found_spec = spec.find_spec(modpath, [context])
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 392, in find_spec
    finder, spec = _find_spec_with_path(
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 361, in _find_spec_with_path
    raise ImportError(f"No module named {'.'.join(module_parts)}")
ImportError: No module named decouple.config
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 782, in _lint_file
    check_astroid_module(module)
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1049, in check_astroid_module
    retval = self._check_astroid_module(
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1099, in _check_astroid_module
    walker.walk(node)
  File "/usr/local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 93, in walk
    self.walk(child)
  File "/usr/local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 90, in walk
    callback(astroid)
  File "/usr/local/lib/python3.10/site-packages/pylint/checkers/imports.py", line 523, in visit_importfrom
    self._add_imported_module(node, f"{imported_module.name}.{name}")
  File "/usr/local/lib/python3.10/site-packages/pylint/checkers/imports.py", line [833](https://gitlab.com/agiliti-techpartners/squaredeal/square-deal-services/-/jobs/3149213017#L833), in _add_imported_module
    importedmodname = astroid.modutils.get_module_part(
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 438, in get_module_part
    file_from_modpath(
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 334, in file_from_modpath
    return file_info_from_modpath(modpath, path, context_file).location
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 384, in file_info_from_modpath
    return _spec_from_modpath(modpath, path, context)
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 593, in _spec_from_modpath
    found_spec = spec.find_spec(modpath, path)
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 392, in find_spec
    finder, spec = _find_spec_with_path(
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 354, in _find_spec_with_path
    spec = finder_instance.find_module(
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 203, in find_module
    if util.is_namespace(modname) and modname in sys.modules:
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/util.py", line 42, in is_namespace
    found_spec = _find_spec_from_path(
  File "/usr/local/lib/python3.10/importlib/util.py", line 58, in _find_spec_from_path
    return _find_spec(name, path)
  File "<frozen importlib._bootstrap>", line 945, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1448, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1222, in __init__
  File "<frozen importlib._bootstrap_external>", line 1238, in _get_parent_path
AttributeError: module 'decouple' has no attribute '__path__'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 747, in _lint_files
    self._lint_file(fileitem, module, check_astroid_module)
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 784, in _lint_file
    raise astroid.AstroidError from e
astroid.exceptions.AstroidError

. ***** Module square-deal-services.square_deal_app.api.views.leads_api square_deal_app/api/views/leads_api.py:1:0: F0002: square_deal_app/api/views/leads_api.py: Fatal error while checking 'square_deal_app/api/views/leads_api.py'. Please open an issue in our bug tracker so we address this. There is a pre-filled template that you can use in '/root/.cache/pylint/pylint-crash-2022-10-10-08-57-37.txt'. (astroid-error)

Configuration

No response

Command used

pylint --load-plugins pylint_django --fail-under $PYLINT_THRESHOLD --rcfile=.pylintrc square_deal_app

Pylint output

pylint crashed with a ``AstroidError`` and with the following stacktrace:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 590, in _spec_from_modpath
    found_spec = spec.find_spec(modpath, [context])
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 392, in find_spec
    finder, spec = _find_spec_with_path(
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 361, in _find_spec_with_path
    raise ImportError(f"No module named {'.'.join(module_parts)}")
ImportError: No module named decouple.config
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 782, in _lint_file
    check_astroid_module(module)
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1049, in check_astroid_module
    retval = self._check_astroid_module(
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 1099, in _check_astroid_module
    walker.walk(node)
  File "/usr/local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 93, in walk
    self.walk(child)
  File "/usr/local/lib/python3.10/site-packages/pylint/utils/ast_walker.py", line 90, in walk
    callback(astroid)
  File "/usr/local/lib/python3.10/site-packages/pylint/checkers/imports.py", line 523, in visit_importfrom
    self._add_imported_module(node, f"{imported_module.name}.{name}")
  File "/usr/local/lib/python3.10/site-packages/pylint/checkers/imports.py", line 833, in _add_imported_module
    importedmodname = astroid.modutils.get_module_part(
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 438, in get_module_part
    file_from_modpath(
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 334, in file_from_modpath
    return file_info_from_modpath(modpath, path, context_file).location
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 384, in file_info_from_modpath
    return _spec_from_modpath(modpath, path, context)
  File "/usr/local/lib/python3.10/site-packages/astroid/modutils.py", line 593, in _spec_from_modpath
    found_spec = spec.find_spec(modpath, path)
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 392, in find_spec
    finder, spec = _find_spec_with_path(
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 354, in _find_spec_with_path
    spec = finder_instance.find_module(
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/spec.py", line 203, in find_module
    if util.is_namespace(modname) and modname in sys.modules:
  File "/usr/local/lib/python3.10/site-packages/astroid/interpreter/_import/util.py", line 42, in is_namespace
    found_spec = _find_spec_from_path(
  File "/usr/local/lib/python3.10/importlib/util.py", line 58, in _find_spec_from_path
    return _find_spec(name, path)
  File "<frozen importlib._bootstrap>", line 945, in _find_spec
  File "<frozen importlib._bootstrap_external>", line 1448, in find_spec
  File "<frozen importlib._bootstrap_external>", line 1222, in __init__
  File "<frozen importlib._bootstrap_external>", line 1238, in _get_parent_path
AttributeError: module 'decouple' has no attribute '__path__'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 747, in _lint_files
    self._lint_file(fileitem, module, check_astroid_module)
  File "/usr/local/lib/python3.10/site-packages/pylint/lint/pylinter.py", line 784, in _lint_file
    raise astroid.AstroidError from e
astroid.exceptions.AstroidError

### Expected behavior

pylint to complete linting and provide the score. 

### Pylint version

```shell
pylint==2.15.2
pylint-django==2.5.3
pylint-plugin-utils==0.7
astroid==2.12.9

$ python -V
Python 3.10.7

OS / Environment

Ubuntu

Additional dependencies

absl-py==1.2.0
aiohttp==3.8.1
aiosignal==1.2.0
amqp==5.1.1
arabic-reshaper==2.1.3
asgiref==3.5.2
asn1crypto==1.5.1
astroid==2.12.9
async-timeout==4.0.2
atomicwrites==1.4.1
attrs==22.1.0
autopep8==1.7.0
billiard==3.6.4.0
boto3==1.24.68
botocore==1.27.68
cached-property==1.5.2
cachetools==5.2.0
celery==5.2.7
certifi==2022.6.15
cffi==1.15.1
chardet==5.0.0
charset-normalizer==2.1.1
click==8.1.3
click-didyoumean==0.3.0
click-plugins==1.1.1
click-repl==0.2.0
colorclass==2.2.2
coreapi==2.3.3
coreschema==0.0.4
cryptography==38.0.1
cssselect2==0.6.0
curlify==2.2.1
defusedxml==0.7.1
dill==0.3.5.1
Django==4.1.1
django-celery-results==2.4.0
django-cors-headers==3.13.0
django-crontab==0.7.1
django-jsonfield==1.4.1
django-mathfilters==1.0.0
django-pgconnection==1.0.3
django-pghistory==2.4.0
django-pgtrigger==4.5.2
django-rest-framework==0.1.0
django-rest-swagger==2.2.0
django-sesame==3.1
django-ttdb==0.5
djangorestframework==3.13.1
djangorestframework-simplejwt==5.2.0
docopt==0.6.2
drf-excel==2.1.0
drf-pdf==0.2.0
drf-renderer-xlsx==1.0.0
einops==0.4.1
et-xmlfile==1.1.0
facebook-business==14.0.0
filelock==3.8.0
frozenlist==1.3.1
fsspec==2022.8.2
future==0.18.2
fuzzywuzzy==0.18.0
google-auth==2.11.0
google-auth-oauthlib==0.4.6
googlemaps==4.6.0
greenlet==1.1.3
grpcio==1.48.1
gunicorn==20.1.0
haversine==2.7.0
html5lib==1.1
huggingface-hub==0.9.1
idna==3.3
isort==5.10.1
itypes==1.2.0
jdcal==1.4.1
Jinja2==3.1.2
jmespath==0.10.0
jsonschema==4.15.0
julius==0.2.6
kombu==5.2.4
lazy-object-proxy==1.7.1
lxml==4.9.1
Markdown==3.4.1
MarkupSafe==2.1.1
mccabe==0.7.0
mir-eval==0.7
mock==4.0.3
mpu==0.23.1
multidict==6.0.2
numpy==1.23.2
oauthlib==3.2.0
openapi-codec==1.3.2
openpyxl==3.0.10
oscrypto==1.3.0
packaging==21.3
pandas==1.4.4
pathlib-mate==1.0.3
pb-bss-eval==0.0.2
pdfrw==0.4
pesq==0.0.4
Pillow==9.2.0
pip-upgrader==1.4.15
platformdirs==2.5.2
prettytable==3.4.1
prompt-toolkit==3.0.31
protobuf==3.19.4
psycopg2-binary==2.9.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycodestyle==2.9.1
pycountry==22.3.5
pycparser==2.21
pyDeprecate==0.3.2
pyHanko==0.13.2
pyhanko-certvalidator==0.19.5
PyJWT==2.4.0
pylint==2.15.2
pylint-django==2.5.3
pylint-plugin-utils==0.7
pyparsing==3.0.9
PyPDF3==1.0.6
pyrsistent==0.18.1
pystoi==0.3.3
python-bidi==0.4.2
python-dateutil==2.8.2
python-decouple==3.6
pytorch-lightning==1.7.5
pytorch-ranger==0.1.1
pytz==2022.2.1
pytz-deprecation-shim==0.1.0.post0
PyYAML==6.0
qrcode==7.3.1
reportlab==3.6.11
requests==2.28.1
requests-oauthlib==1.3.1
retrying==1.3.3
rsa==4.9
s3transfer==0.6.0
scipy==1.9.1
sentry-sdk==1.9.8
simplejson==3.17.6
six==1.16.0
SoundFile==0.10.3.post1
SQLAlchemy==1.4.41
sqlalchemy-mate==1.4.28.3
sqlparse==0.4.2
stripe==2.60.0
svglib==1.4.1
tensorboard==2.10.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.1
terminaltables==3.1.10
tinycss2==1.1.1
toml==0.10.2
tomli==2.0.1
tomlkit==0.11.4
torch==1.12.1
torch-optimizer==0.1.0
torch-stoi==0.1.2
torchaudio==0.12.1
torchmetrics==0.7.3
tqdm==4.64.1
typing_extensions==4.3.0
tzdata==2022.2
tzlocal==4.2
uritemplate==4.1.1
uritools==4.0.0
urllib3==1.26.12
uszipcode==1.0.1
vine==5.0.0
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==2.2.2
wrapt==1.14.1
xhtml2pdf==0.2.8
xlrd==2.0.1
yarl==1.8.1
jacobtylerwalls commented 2 years ago

Thanks for the report. We may need to add a wider catch to is_namespace in astroid.

It would be of tremendous help if you could reproduce against astroid 2.12.11 or astroid main branch, both of which have changes to the is_namespace method.

Also, do you have any namespace packages or other directories that are called decouple?

The failure point in the stacktrace looks very similar to the one described in python/cpython#93334, except that it's AttributeError instead of KeyError. I'm wondering if https://github.com/python/cpython/pull/98100 would resolve the root issue in importlib, but I know that's a lot to ask of a bug reporter, feel free to just answer my first question about astroid versions :-)

hnhegde commented 2 years ago

Checked against astroid==2.12.11. Same issue.

Reg. decouple, I couldn't identify the root cause. There was this SO thread that said:

You might have decouple installed in additional to python-decouple (two different packages).

But, I confirmed that's not the case with me.