Pylint version is old for pre-commit with python3.11

[atztogo]

On my computer, tox -e pre-commit fails for python 3.11. This is probably fixed by using newer pylint. The issue is https://github.com/GrahamDumpleton/wrapt/issues/196. The version of pylint is fixed at setup.json, which causes the problem.

But I see the version in setup.json is the same as that in aiida-core. So do we want to update it? The version of the pre-commit hook in .pre-commit-config.yaml seems old. (Other hooks are also old.)

% tox -e pre-commit
.pkg: _optional_hooks> python /Users/togo/.miniforge/envs/parsevasp/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /Users/togo/.miniforge/envs/parsevasp/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: prepare_metadata_for_build_wheel> python /Users/togo/.miniforge/envs/parsevasp/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /Users/togo/.miniforge/envs/parsevasp/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
pre-commit: recreate env because dependencies removed: pylint
pre-commit: remove tox env folder /Users/togo/code/parsevasp/.tox/pre-commit
pre-commit: install_package_deps> python -I -m pip install future lxml numpy 'pre-commit~=2.2' py 'pylint~=2.11.1' pyyaml 'tox>=3.23.0' 'virtualenv>20'
pre-commit: install_package> python -I -m pip install --force-reinstall --no-deps /Users/togo/code/parsevasp/.tox/.tmp/package/8/parsevasp-3.1.0.tar.gz
pre-commit: commands[0]> bash -ec 'pre-commit run --all-files || ( git diff; git status; exit 1; )'
fix double quoted strings....................................................Passed
fix end of files.............................................................Passed
fix python encoding pragma...................................................Passed
mixed line ending............................................................Passed
trim trailing whitespace.....................................................Passed
check yaml...................................................................Passed
check for added large files..................................................Passed
forbid new submodules....................................(no files to check)Skipped
isort........................................................................Passed
flynt........................................................................Passed
yapf.........................................................................Passed
pylint.......................................................................Failed
- hook id: pylint
- exit code: 1

Traceback (most recent call last):
  File "/Users/togo/code/parsevasp/.tox/pre-commit/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
             ^^^^^^^^^^^^
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/pylint/__init__.py", line 21, in run_pylint
    from pylint.lint import Run as PylintRun
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/pylint/lint/__init__.py", line 76, in <module>
    from pylint.lint.parallel import check_parallel
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/pylint/lint/parallel.py", line 8, in <module>
    from pylint import reporters
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/pylint/reporters/__init__.py", line 26, in <module>
    from pylint import utils
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/pylint/utils/__init__.py", line 46, in <module>
    from pylint.utils.ast_walker import ASTWalker
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/pylint/utils/ast_walker.py", line 7, in <module>
    from astroid import nodes
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/astroid/__init__.py", line 51, in <module>
    from astroid.nodes import node_classes, scoped_nodes
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/astroid/nodes/__init__.py", line 27, in <module>
    from astroid.nodes.node_classes import (  # pylint: disable=redefined-builtin (Ellipsis)
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/astroid/nodes/node_classes.py", line 47, in <module>
    from astroid import decorators, mixins, util
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/astroid/decorators.py", line 27, in <module>
    import wrapt
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/wrapt/__init__.py", line 10, in <module>
    from .decorators import (adapter_factory, AdapterFactory, decorator,
  File "/Users/togo/code/parsevasp/.tox/pre-commit/lib/python3.11/site-packages/wrapt/decorators.py", line 34, in <module>
    from inspect import ismethod, isclass, formatargspec
ImportError: cannot import name 'formatargspec' from 'inspect' (/Users/togo/.miniforge/envs/parsevasp/lib/python3.11/inspect.py)

Check .rst files with the same linter used by pypi.org.......................Passed
On branch develop
Your branch is up to date with 'origin/develop'.

nothing to commit, working tree clean
pre-commit: exit 1 (5.77 seconds) /Users/togo/code/parsevasp> bash -ec 'pre-commit run --all-files || ( git diff; git status; exit 1; )' pid=16755
.pkg: _exit> python /Users/togo/.miniforge/envs/parsevasp/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  pre-commit: FAIL code 1 (14.05=setup[8.28]+cmd[5.77] seconds)
  evaluation failed :( (14.09 seconds)

[espenfl]

Okey. So I think it would make sense at this point to follow upstream aiida-core, then aiida-vasp in parsevasp. The test and lint environment etc. for aiida-core is sensible and I do not see us having to do it differently. A huge upside is that it would then be much easier to work in editable mode and run pre-commit on all three packages, which is certainly a use case, in particular for the latter two. Since aiida-vasp depends on aiida-core, otherwise it is not very useful, we can use for instance aiida-core[pre-commit] in the pre-commit extras as a dependency in aiida-vasp. Unfortunately this installs the aiida-core package as well, not just its dependencies. It would be ideal if we could use the same syntax and then tell the installer that we do not want the main package installed, only the listed dependencies. But that is not possible yet. For aiida-vasp we can live with aiida-core being installed in pre-commit and for regular use and tests it is surely needed anyway. But for parsevasp we do not want this. We could rely on the depedencies also used by aiida-core for pre-commit but not aiida-core itself. So right now, we try to follow those dependencies in a hard coded way. Not ideal.

And here, we see this showing up for Python 3.11 for pylint. The wrapt fix should be in 2.13.0, according to: https://github.com/pylint-dev/pylint/issues/5919#issuecomment-1067732808.

[espenfl]

In fact, we need pylint~=2.15 as the wrapt dependency was bumped from there on.