search

pylint-dev / pylint

It's not just a linter that annoys you!
https://pylint.readthedocs.io/en/latest/
GNU General Public License v2.0
5.33k stars 1.14k forks source link

Colorama >=0.4.5 not only required in Windows environment #9170

Open cmorsbach opened 1 year ago

cmorsbach commented 1 year ago

Bug description

The colorama==0.4.4 RecursionError described in #6012 and fixed in #6970 can also occur in non-windows environment. In our case it occurred on a GitLab Linux runner when pylint was executed as part of a GitLab pipeline. For some strange reason, the problem did not occur on the same system when logged in on the Linux shell.

Therefore, I suggest that the colorama>=0.4.5 dependency should also be enforced on Linux operating systems.

Configuration

No response

Command used

pylint --rcfile=.pylint.rc path/to/package -E

Pylint output

Exception in initializer:
Traceback (most recent call last):
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 230, in _process_worker
    initializer(*initargs)
  File "/path/to/venv/lib/python3.9/site-packages/pylint/lint/parallel.py", line 47, in _worker_initialize
    _worker_linter = dill.loads(linter)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 301, in loads
    return load(file, ignore, **kwds)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 287, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 442, in load
    obj = StockUnpickler.load(self)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  [Previous line repeated 972 more times]
RecursionError: maximum recursion depth exceeded
Exception in initializer:
Traceback (most recent call last):
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 230, in _process_worker
    initializer(*initargs)
  File "/path/to/venv/lib/python3.9/site-packages/pylint/lint/parallel.py", line 47, in _worker_initialize
    _worker_linter = dill.loads(linter)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 301, in loads
    return load(file, ignore, **kwds)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 287, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 442, in load
    obj = StockUnpickler.load(self)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  [Previous line repeated 972 more times]
RecursionError: maximum recursion depth exceeded
Exception in initializer:
Traceback (most recent call last):
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 230, in _process_worker
    initializer(*initargs)
  File "/path/to/venv/lib/python3.9/site-packages/pylint/lint/parallel.py", line 47, in _worker_initialize
    _worker_linter = dill.loads(linter)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 301, in loads
    return load(file, ignore, **kwds)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 287, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 442, in load
    obj = StockUnpickler.load(self)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  [Previous line repeated 972 more times]
RecursionError: maximum recursion depth exceeded
Exception in initializer:
Traceback (most recent call last):
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 230, in _process_worker
    initializer(*initargs)
  File "/path/to/venv/lib/python3.9/site-packages/pylint/lint/parallel.py", line 47, in _worker_initialize
    _worker_linter = dill.loads(linter)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 301, in loads
    return load(file, ignore, **kwds)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 287, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 442, in load
    obj = StockUnpickler.load(self)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  [Previous line repeated 972 more times]
RecursionError: maximum recursion depth exceeded
Exception in initializer:
Traceback (most recent call last):
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 230, in _process_worker
    initializer(*initargs)
  File "/path/to/venv/lib/python3.9/site-packages/pylint/lint/parallel.py", line 47, in _worker_initialize
    _worker_linter = dill.loads(linter)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 301, in loads
    return load(file, ignore, **kwds)
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 287, in load
    return Unpickler(file, ignore=ignore, **kwds).load()
  File "/path/to/venv/lib/python3.9/site-packages/dill/_dill.py", line 442, in load
    obj = StockUnpickler.load(self)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  File "/usr/lib/python3/dist-packages/colorama/ansitowin32.py", line 29, in __getattr__
    return getattr(self.__wrapped, name)
  [Previous line repeated 972 more times]
RecursionError: maximum recursion depth exceeded
Traceback (most recent call last):
  File "/path/to/venv/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/path/to/venv/lib/python3.9/site-packages/pylint/__init__.py", line 34, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "/path/to/venv/lib/python3.9/site-packages/pylint/lint/run.py", line 211, in __init__
    linter.check(args)
  File "/path/to/venv/lib/python3.9/site-packages/pylint/lint/pylinter.py", line 679, in check
    check_parallel(
  File "/path/to/venv/lib/python3.9/site-packages/pylint/lint/parallel.py", line 162, in check_parallel
    ) in executor.map(_worker_check_single_file, files):
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 726, in map
    results = super().map(partial(_process_chunk, fn),
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 589, in map
    fs = [self.submit(fn, *args) for args in zip(*iterables)]
  File "/usr/lib/python3.9/concurrent/futures/_base.py", line 589, in <listcomp>
    fs = [self.submit(fn, *args) for args in zip(*iterables)]
  File "/usr/lib/python3.9/concurrent/futures/process.py", line 681, in submit
    raise BrokenProcessPool(self._broken)
concurrent.futures.process.BrokenProcessPool: A child process terminated abruptly, the process pool is not usable anymore

Expected behavior

Run without error

Pylint version

pylint 3.0.1
astroid 3.0.1
Python 3.9.2 (default, Feb 28 2021, 17:03:44) 
[GCC 10.2.1 20210110]

OS / Environment

SMP Debian 5.10.197-1 (2023-09-29) x86_64 GNU/Linux

Additional dependencies

colorama==0.4.4
Pierre-Sassoulas commented 1 year ago

Thank you for opening the issue, if you want you can also fix it and become a contributor. The fix is to remove the windows specificity of the requirements (maybe add a little changelog entry).

cmorsbach commented 1 year ago

Sure thing, looks easy enough if you're fine with having that required for linux.

cmorsbach commented 1 year ago

I just had a look, and in my opinion this problem cannot really come from pylint calling colorama. This is indeed only done if the platform is win32. So the problem must be in one of the other dependencies possibly using colorama. Do you still want this requirement in pylint, then?

Pierre-Sassoulas commented 1 year ago

If it's not strictly necessary, then no 😄 Your traceback still had pylint call in it and seemed pretty convincing though.

cmorsbach commented 1 year ago

Well nothing in the call stack has colorama in it. Still I don't understand how in my linux environment we get past https://github.com/pylint-dev/pylint/blob/main/pylint/reporters/text.py#L230, since I get

$ python -c "import sys; print(f'{sys.platform=}')"
sys.platform='linux'

Well, in the end, I definitely have a case, where I have a problem with pylint, even if it might be exotic. 🤷

gruenich commented 3 days ago

Wouldn't it make sense to ask for version 0.4.6 of colorama for every platform? The release is more than 2 years old and it added support for Python 3.9 (in 0.4.5).

Pierre-Sassoulas commented 1 day ago

Doing that would possibly prevent someone from using the old version of colorama if they want for no real reasons, I don't think we should do it unless we're sure there's an issue. (But we support >= python 3.9 which means pip is not going to bring versions < 0.4.5 anyway, as the current version of pylint is incompatible with python 3.8).