prospector-dev / prospector

Inspects Python source files and provides information about type and location of classes, methods etc
GNU General Public License v2.0
1.95k stars 173 forks source link

[BUG] Not working with the last version of Pylint #465

Closed sbrunner closed 2 years ago

sbrunner commented 2 years ago

Describe the bug Just not working with the last version of PyLint

To Reproduce Steps to reproduce the behavior:

  1. Install last version of Prospector and Pylint
  2. Run prospector -X in an empty folder

=>

/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/setuptools/distutils_patch.py:25: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
  warnings.warn(
Traceback (most recent call last):
  File "/home/sbrunner/.asdf/installs/python/3.8.6/bin/prospector", line 8, in <module>
    sys.exit(main())
  File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/prospector/run.py", line 183, in main
    prospector.execute()
  File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/prospector/run.py", line 69, in execute
    messages += tool.run(found_files)
  File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/prospector/tools/pylint/__init__.py", line 259, in run
    self._linter.check(self._args)
  File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 996, in check
    self._check_files(
  File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1031, in _check_files
    with self._astroid_module_checker() as check_astroid_module:
  File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1150, in _astroid_module_checker
    checker.open()
  File "/home/sbrunner/.asdf/installs/python/3.8.6/lib/python3.8/site-packages/pylint/checkers/unsupported_version.py", line 51, in open
    self._py36_plus = py_version >= (3, 6)
TypeError: '>=' not supported between instances of 'NoneType' and 'tuple'

Expected behavior No output, no error

Screenshots not applicable

Environment (please complete the following information):

Additional context python3.8 -m pip freeze:

affine==2.3.0
aiocache==0.11.1
alembic==1.6.5
appdirs==1.4.4
argcomplete==1.12.3
arrow==1.1.0
astor==0.7.1
astroid==2.9.0
attrs==21.2.0
auto-changelog==0.5.3
azure-core==1.15.0
azure-storage-blob==12.9.0
Babel==2.9.1
backcall==0.2.0
bandit==1.7.0
bashcolor==1.1.1
bcrypt==3.2.0
Beaker==1.11.0
binaryornot==0.4.4
black==21.10b0
bleach==3.3.0
boto3==1.17.88
botocore==1.20.88
bottle==0.12.19
c2c.template==2.3.0
-e git+ssh://git@github.com/camptocamp/c2cciutils.git@b41af7949f102cd7206302956b062f95d73ae8da#egg=c2cciutils
c2cgeoform==2.1.23
-e git+ssh://git@github.com/camptocamp/c2cgeoportal.git@4442a8d28c6e0a3e5f8680c73153866f1a090beb#egg=c2cgeoportal_admin&subdirectory=admin
-e git+ssh://git@github.com/camptocamp/c2cgeoportal.git@4442a8d28c6e0a3e5f8680c73153866f1a090beb#egg=c2cgeoportal_commons&subdirectory=commons
-e git+ssh://git@github.com/camptocamp/c2cgeoportal.git@4442a8d28c6e0a3e5f8680c73153866f1a090beb#egg=c2cgeoportal_geoportal&subdirectory=geoportal
-e git+ssh://git@github.com/camptocamp/c2cwsgiutils.git@57746499c8830547fc4d7564277515e130b936ab#egg=c2cwsgiutils
CacheControl==0.12.6
cached-property==1.5.2
cachetools==4.2.2
cachy==0.3.0
cee-syslog-handler==0.6.0
Cerberus==1.3.4
certifi==2021.5.30
cffi==1.14.5
Chameleon==3.9.1
changelog-builder==0.1.7
chardet==4.0.0
cleo==0.8.1
click==7.1.2
click-log==0.3.2
click-plugins==1.1.1
cligj==0.7.2
clikit==0.6.2
codespell==2.1.0
colander==1.8.3
ColanderAlchemy==0.3.4
colorama==0.4.4
commitizen==2.17.12
confuse==1.5.0
conventional==0.5.3
conventional-commit==0.4.2
cookiecutter==1.7.3
crashtest==0.3.1
cryptography==3.4.7
cycler==0.10.0
decli==0.5.2
decorator==4.4.2
deform==2.0.15
defusedxml==0.7.1
-e git+ssh://git@github.com/sbrunner/deskew.git@5158d9e6f2692bea507ab81d7aac8e4b044557df#egg=deskew
distlib==0.3.2
distro==1.5.0
docformatter==1.4
docker==5.0.0
docker-compose==1.29.2
dockerpty==0.4.1
docopt==0.6.2
docutils==0.17.1
dodgy==0.2.1
dogpile.cache==1.1.3
dotty-dict==1.3.0
dparse==0.5.1
easygui==0.98.2
EasyProcess==0.3
EditorConfig==0.12.3
entrypoint2==0.2.4
filelock==3.0.12
Fiona==1.8.20
flake8==3.8.4
flake8-polyfill==1.0.2
GeoAlchemy2==0.9.1
geojson==2.5.0
git-changelog==0.4.2
gitdb==4.0.7
GitPython==3.1.18
gobang==0.1.0
google-api-core==1.29.0
google-api-python-client==2.2.0
google-auth==1.30.1
google-auth-httplib2==0.1.0
google-auth-oauthlib==0.4.4
googleapis-common-protos==1.53.0
graphite==0.70
graphyte==1.6.0
greenlet==1.1.0
gunicorn==20.1.0
html5lib==1.1
httplib2==0.19.1
hupper==1.10.3
idna==2.10
imageio==2.9.0
importlib-metadata==4.5.0
influxdb==5.3.1
iniconfig==1.1.1
inotify==0.2.10
invoke==1.6.0
ipython==7.27.0
iso8601==0.1.14
isodate==0.6.0
isoparser==0.3
isort==5.8.0
jedi==0.18.0
jeepney==0.6.0
Jinja2==2.11.3
jinja2-time==0.2.0
jira==3.0.1
jmespath==0.10.0
jsonref==0.2
jsonschema==3.2.0
jsonschema-gentypes==0.9.3
jsonschema2md==0.2.0.post1
keyring==21.8.0
kiwisolver==1.3.1
kubernetes==17.17.0
lark-parser==0.11.3
lazy-object-proxy==1.4.3
lingua==4.14
lnetatmo==2.1.0
lockfile==0.12.2
Mako==1.1.4
mappyfile==0.9.1
Markdown==3.3.4
markdown-table==2020.12.3
MarkupSafe==2.0.1
matplotlib==3.4.2
matplotlib-inline==0.1.2
mccabe==0.6.1
mp3play==0.1.15
msgpack==1.0.2
msrest==0.6.21
mss==6.1.0
munch==2.5.0
mypy==0.901
mypy-extensions==0.4.3
networkx==2.5.1
node-vm2==0.3.7
nose==1.3.7
numpy==1.21.0
oauthlib==3.1.1
opencv-python==4.5.3.56
opentsdb-py==0.6.0
orderedmultidict==1.0.1
OWSLib==0.24.1
packaging==20.9
pandas==1.2.5
papyrus==2.4
paramiko==2.7.2
parso==0.8.2
passwordgenerator==1.4
PasteDeploy==2.1.1
pastel==0.2.1
pathspec==0.9.0
pbr==5.6.0
pdfminer==20191125
pdfminer.six==20200517
pdfplumber==0.5.28
pep517==0.11.0
pep8-naming==0.10.0
peppercorn==0.6
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.2.0
pip-shims==0.5.3
pipenv==2021.11.15
pipfile==0.0.2
pkginfo==1.7.0
plaster==1.0
plaster-pastedeploy==0.7
platformdirs==2.2.0
playsound==1.3.0
plette==0.2.3
pluggy==0.13.1
poetry==1.1.8
poetry-core==1.0.4
polib==1.1.1
poyo==0.5.0
prompt-toolkit==3.0.19
prospector==1.5.1
protobuf==3.17.2
psycopg2==2.8.6
psycopg2-binary==2.9.1
ptyprocess==0.7.0
py==1.10.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycairo==1.20.1
pycodestyle==2.6.0
pycparser==2.20
pycryptodome==3.10.1
pydocstyle==6.1.1
pyflakes==2.2.0
pygame==2.0.1
Pygments==2.9.0
PyGObject==3.40.1
pyguetzli==1.0.9
PyJWT==2.1.0
pylev==1.4.0
pylint==2.12.1
pylint-celery==0.3
pylint-django==2.4.4
pylint-flask==0.6
pylint-plugin-utils==0.6
PyNaCl==1.4.0
pyotp==2.6.0
pyparsing==2.4.7
PyPDF2==1.26.0
pyperclip==1.8.2
pyproj==3.1.0
pyramid==1.10.8
pyramid-beaker==0.8
pyramid-chameleon==0.3
pyramid-debugtoolbar==4.9
pyramid-jinja2==2.8
pyramid-mako==1.1.0
pyramid-multiauth==0.9.0
pyramid-tm==2.4
pyroma==3.2
pyrsistent==0.17.3
pyscreenshot==3.0
pytest==6.2.4
python-dateutil==2.8.1
python-dotenv==0.17.1
python-editor==1.0.4
python-gettext==4.0
python-gitlab==2.9.0
python-magic==0.4.22
python-semantic-release==7.16.2
python-slugify==4.0.1
python-vlc==3.0.12118
pytz==2021.1
pyupgrade==2.19.4
PyWavelets==1.1.1
PyYAML==5.4.1
pyyaml-include==1.2.post2
questionary==1.9.0
rasterio==1.2.6
readme-renderer==29.0
redis==3.5.3
regex==2021.4.4
repoze.lru==0.7
requests==2.25.1
requests-oauthlib==1.3.0
requests-toolbelt==0.9.1
requirements-detector==0.7
requirementslib==1.5.16
rfc3986==1.5.0
rsa==4.7.2
ruamel.yaml==0.17.17
ruamel.yaml.clib==0.2.2
runlike==0.7.0
s3transfer==0.4.2
safety==1.10.3
-e git+ssh://git@github.com/sbrunner/scan-to-paperless.git@3af86d80f908d68f9b389ec3e53ec5c3a78da45e#egg=scan_to_paperless
scikit-build==0.11.1
scikit-image==0.18.2
scipy==1.7.0
-e git+ssh://git@github.com/gberaudo/scrumtools.git@73b05150f2e5ea8007d834a7ec0a1dea3aa6dfd4#egg=scrumtools
SecretStorage==3.3.1
semver==2.13.0
sentry-sdk==1.1.0
setoptconf==0.2.0
setoptconf-tmp==0.3.1
setuptools-scm==6.0.1
Shapely==1.7.1
-e git+ssh://git@github.com/camptocamp/shared_config_manager.git@e594d5215ca3f45094c677c45573018f8fa8f538#egg=shared_config_manager&subdirectory=app
shellingham==1.4.0
six==1.16.0
smmap==4.0.0
snowballstemmer==2.1.0
snuggs==1.4.7
sortedcontainers==2.4.0
SQLAlchemy==1.3.23
stevedore==3.3.0
systemd==0.16.1
termcolor==1.1.0
text-unidecode==1.3
texttable==1.6.3
tifffile==2021.7.2
-e git+ssh://git@github.com/camptocamp/tilecloud.git@3cb56fd269b47678fdb9d3a1b4f9980985f0b68e#egg=tilecloud
-e git+ssh://git@github.com/camptocamp/tilecloud-chain.git@2e00f4e443abf797d6e7290e17dbccd579c0dc35#egg=tilecloud_chain
tk==0.1.0
tokenize-rt==4.1.0
toml==0.10.2
tomli==1.1.0
tomlkit==0.7.0
tqdm==4.61.0
traitlets==5.1.0
transaction==3.0.1
transifex-client==0.14.3
translationstring==1.4
twine==3.6.0
typer==0.3.2
types-pkg-resources==0.1.3
types-PyYAML==5.4.3
types-redis==3.5.16
types-requests==2.25.0
typing-extensions==3.10.0.0
Unidecode==1.2.0
universion==0.1.2
UNKNOWN @ file:///home/external/workspace/helm-mapserver
untokenize==0.1.1
uritemplate==3.0.1
urllib3==1.26.5
venusian==3.0.0
virtualenv==20.4.7
virtualenv-clone==0.5.4
vistir==0.5.2
Wand==0.6.7
wcwidth==0.2.5
webencodings==0.5.1
WebOb==1.8.7
websocket-client==0.59.0
whisper==1.1.8
wrapt==1.12.1
yoga==1.0.0
yoga-image-optimizer==1.0.1
yolk3k==0.9
zipp==3.4.1
zope.deprecation==4.4.0
zope.event==4.5.0
zope.interface==5.4.0
zope.sqlalchemy==1.4
zopflipy==1.5
maallaard commented 2 years ago

I have this same error and output using Python 3.9.9 with pip 21.3.1, installed only with prospector, black, pipenv, and all their respective dependencies. I use prospector with integration with vscode, so I am providing below a detailed error-message json, if it helps.

[{
    "owner": "python",
    "code": "failure",
    "severity": 8,
    "message": "Tool pylint failed to run (exception was raised, re-run prospector with -X to see the stacktrace)",
    "source": "prospector - pylint",
    "startLineNumber": 1,
    "startColumn": 1,
    "endLineNumber": 1,
    "endColumn": 1
}]

below is my variation of this error:

% prospector -X       
Traceback (most recent call last):
  File "/usr/local/bin/prospector", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/prospector/run.py", line 183, in main
    prospector.execute()
  File "/usr/local/lib/python3.9/site-packages/prospector/run.py", line 69, in execute
    messages += tool.run(found_files)
  File "/usr/local/lib/python3.9/site-packages/prospector/tools/pylint/__init__.py", line 259, in run
    self._linter.check(self._args)
  File "/usr/local/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 996, in check
    self._check_files(
  File "/usr/local/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1031, in _check_files
    with self._astroid_module_checker() as check_astroid_module:
  File "/usr/local/Cellar/python@3.9/3.9.9/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 1150, in _astroid_module_checker
    checker.open()
  File "/usr/local/lib/python3.9/site-packages/pylint/checkers/typecheck.py", line 918, in open
    self._py310_plus = py_version >= (3, 10)
TypeError: '>=' not supported between instances of 'NoneType' and 'tuple'

and the freeze:

astroid==2.9.0
backports.entry-points-selectable==1.1.1
black==21.11b1
certifi==2021.10.8
click==8.0.3
distlib==0.3.3
dodgy==0.2.1
filelock==3.4.0
flake8==3.9.2
flake8-polyfill==1.0.2
isort==5.10.1
lazy-object-proxy==1.6.0
mccabe==0.6.1
mypy-extensions==0.4.3
pathspec==0.9.0
pep8-naming==0.10.0
pipenv==2021.11.23
platformdirs==2.4.0
prospector==1.5.1
pycodestyle==2.7.0
pydocstyle==6.1.1
pyflakes==2.3.1
pylint==2.12.1
pylint-celery==0.3
pylint-django==2.4.4
pylint-flask==0.6
pylint-plugin-utils==0.6
PyYAML==6.0
regex==2021.11.10
requirements-detector==0.7
setoptconf-tmp==0.3.1
six==1.16.0
snowballstemmer==2.2.0
toml==0.10.2
tomli==1.2.2
typing_extensions==4.0.0
virtualenv==20.10.0
virtualenv-clone==0.5.7
wrapt==1.13.3
cyberfox1 commented 2 years ago

pip install 'pylint<2.12.0'

meteozond commented 2 years ago

What about releasing bugfix version of prospector with frozen pylint dependency?

carlio commented 2 years ago

Working on it :) I'll make a bugfix first, then make a proper release later to fix compatibility.

carlio commented 2 years ago

(Note to self : this was also reported in pylint - https://github.com/PyCQA/pylint/issues/5424)

carlio commented 2 years ago

1.5.2 is on PyPI now, just pinning the version.

CC @Pierre-Sassoulas you've self-assigned this - do you want to work on the compatibility fix? I just made the bugfix release so far to fix people's CI pipelines (hopefully). I'm happy to do it instead if you don't want to, but I don't want to step on your toes!

Pierre-Sassoulas commented 2 years ago

Thank you for doing the hot fix ! :)

I'm happy to do it instead if you don't want to, but I don't want to step on your toes!

I assigned it to me because it looked like a missing call to super() in a class prospector inherit and as I know pylint I could probably fix it without too much effort. And I did not expect you to show up 😉 But I did not have the time to work on it yet. This is a busy week, MR for pylint 2.13 are piling up. I'm relieved to see you stepped up honestly 😄

carlio commented 2 years ago

@Pierre-Sassoulas This appears to fix it- https://github.com/PyCQA/prospector/commit/380972f410dfe9ed7a7de15cd0d6ad8670da3213

For context that reset_options was created a long time ago when if I remember correctly, it was necessary because it clashed with prospector trying to override pylint configuration options.

I've tested it locally, it seems to work fine now with latest pylint so I just removed this legacy code.

Have a quick look in case there's anything you think I'm missing, but it seems that the problem was that pylint now gathers information about the environment it runs in that prospector was then resetting, leaving pylint to break later.

I'll create a dev0 release for this and let people test it before making it a real release.

maallaard commented 2 years ago

I forgot to comment this after I posted my last comment, but I have some results.

I changed my interpreter to python 3.10.0 with the same packages installed as above to experiment, and it worked just fine. I didn't get the same error and it showed the linting warnings for my code as it should.

Sorry if I'm just stating the obvious, just wanted to post this so everyone knows that prospector works with python 3.10

carlio commented 2 years ago

Prospector version 1.5.3.dev0 is now on pypi, as a pre-release. That should work fine, it's still just a pre-release while people can test it.

CC @sbrunner @maallaard @Pierre-Sassoulas

sbrunner commented 2 years ago

@carlio Just tested and that looks good, Thanks :-)

carlio commented 2 years ago

Closing, as 1.5.3 is now a full release which appears to fix this issue.

adamcunnington-mlg commented 2 years ago

@carlio 1.5.3 does not fix this issue. we are still pinning < 2.12, not <= 2.12

carlio commented 2 years ago

Done (properly!) in 1.5.3.1