Debian patched python 3.12 fails testing

[yarikoptic]

There is a good number of patches (motivation is unknown to me, didn't track) are in Debian's python 3.12 package. See https://sources.debian.org/patches/python3.12/3.12.7-3/ . Testing of our https://github.com/datalad/datalad/ package started to fail on 3.12 (passed ok before IIRC), and it seems to manifest also in running argcomplete tests:

❯ ./test/test.py -f
E
======================================================================
ERROR: test_action_activation (__main__.TestArgcomplete.test_action_activation)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 250, in test_action_activation
    self.assertEqual(set(self.run_completer(make_parser(), cmd)), set(output))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 139, in run_completer
    completer(parser, output_stream=t, exit_method=sys.exit, **kwargs)
  File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 174, in __call__
    completions = self._get_completions(comp_words, cword_prefix, cword_prequote, last_wordbreak_pos)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 223, in _get_completions
    completions = self.collect_completions(active_parsers, parsed_args, cword_prefix)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 440, in collect_completions
    completions += self._get_option_completions(active_parser, cword_prefix)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 326, in _get_option_completions
    if not self._action_allowed(action, parser):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 336, in _action_allowed
    for conflict_action in parser._action_conflicts.get(action, []):
                           ^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'MonkeyPatchedIntrospectiveArgumentParser' object has no attribute '_action_conflicts'

----------------------------------------------------------------------
Ran 1 test in 0.002s

FAILED (errors=1)
❯ python --version
Python 3.12.7

if you don't have a debian system, but podman or docker do smth like

podman run -it --rm debian:sid

and inside

apt update
apt install -y python3.12 git python-is-python3 python3-pip
cd /tmp
git clone https://github.com/kislyuk/argcomplete/
cd argcomplete/
pip install --break-system-packages  -e .[test]
./test/test.py -f

[duck-rh]

I stumbled on the problem too and other Debian users too: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087455

@yarikoptic you seem to imply that the patches in Debian Python packaging make a difference, did you test on other systems with the same Python version to confirm? (I'll try to see if I can access other systems to confirm too)

[yarikoptic]

Not 100% since I don't remember if I tested against vanilla Python release. But we run tests regularly against 3.12 in conda and related test did not fail until I looked into updating Debian package. We had another test/code also triggered by a patch in Debian package which was picked up from some 3.14a IIRC

[yarikoptic]

FWIW -- with vanilla build of 3.13.0 no such error -- just 1 failing test

```shell ❯ ./test/test.py -f ...................................usage: test.py [-h] --foo FOO {bar} test.py: error: too few arguments usage: test.py [-h] --foo FOO {bar} test.py: error: argument --foo is required ...usage: test.py foo [-h] {bar} test.py foo: error: too few arguments ......x...x.......................x...x.........F ====================================================================== FAIL: test_redirection_completion (__main__.TestBashGlobal.test_redirection_completion) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 1342, in test_redirection_completion self.assertEqual(msg, "success\n") ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^ AssertionError: 'failure\n' != 'success\n' - failure + success ---------------------------------------------------------------------- Ran 87 tests in 85.974s FAILED (failures=1, expected failures=4) ./test/test.py -f 51.82s user 4.85s system 65% cpu 1:26.85 total ❯ which python /home/yoh/proj/misc/argcomplete/venv/3.13.0/bin/python ❯ python --version Python 3.13.0 ```

similarly with fresh vanilla build of 3.12.7

```shell fi ./python -E -c 'import sys ; from sysconfig import get_platform ; print("%s-%d.%d" % (get_platform(), *sys.version_info[:2]))' >platform The necessary bits to build these optional modules were not found: _dbm _gdbm _tkinter nis To find the necessary bits, look in configure.ac and config.log. Checked 111 modules (31 built-in, 75 shared, 1 n/a on linux-x86_64, 0 disabled, 4 missing, 0 failed on import) make -j 8 107.41s user 7.11s system 547% cpu 20.927 total ❯ ❯ ln -s python ./python3.12.7 ❯ ln -s python ./python3.12 ❯ ln -s python ./python3 ❯ export PATH=/home/yoh/proj/misc/cpython-3.12.7:$PATH ❯ cd -2 ~/proj/misc/argcomplete Authors.rst LICENSE.rst Makefile README.rst argcomplete/ common.mk docs/ setup.cfg test/ Changes.rst MANIFEST.in NOTICE SECURITY.md argcomplete.egg-info/ contrib/ pyproject.toml setup.py* venv/ ❯ py=3.12.7; d=venv/$py; python$py -m venv $d && source $d/bin/activate && pip install -e . && pip install pexpect && ./test/test.py -f Obtaining file:///home/yoh/proj/misc/argcomplete Installing build dependencies ... done Checking if build backend supports build_editable ... done Getting requirements to build editable ... done Preparing editable metadata (pyproject.toml) ... done Building wheels for collected packages: argcomplete Building editable for argcomplete (pyproject.toml) ... done Created wheel for argcomplete: filename=argcomplete-3.5.1-0.editable-py3-none-any.whl size=12819 sha256=b774e0b7d0fbe9a9392a050a5762858d6b801c83e949c536176af32d62ecdd13 Stored in directory: /home/yoh/.tmp/pip-ephem-wheel-cache-4j3avk04/wheels/06/87/cf/8a90c55a9e84ae1b6cb791c26deb6b39eb9518210d3a5e12e4 /home/yoh/proj/misc/argcomplete/venv/3.12.7/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'} return cls(files, info_location) Successfully built argcomplete Installing collected packages: argcomplete Successfully installed argcomplete-3.5.1 [notice] A new release of pip is available: 24.2 -> 24.3.1 [notice] To update, run: pip install --upgrade pip Collecting pexpect Using cached pexpect-4.9.0-py2.py3-none-any.whl.metadata (2.5 kB) Collecting ptyprocess>=0.5 (from pexpect) Using cached ptyprocess-0.7.0-py2.py3-none-any.whl.metadata (1.3 kB) Using cached pexpect-4.9.0-py2.py3-none-any.whl (63 kB) Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB) /home/yoh/proj/misc/argcomplete/venv/3.12.7/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'} return cls(files, info_location) Installing collected packages: ptyprocess, pexpect /home/yoh/proj/misc/argcomplete/venv/3.12.7/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'} return cls(files, info_location) Successfully installed pexpect-4.9.0 ptyprocess-0.7.0 [notice] A new release of pip is available: 24.2 -> 24.3.1 [notice] To update, run: pip install --upgrade pip ...................................usage: test.py [-h] --foo FOO {bar} test.py: error: too few arguments usage: test.py [-h] --foo FOO {bar} test.py: error: argument --foo is required ...usage: test.py foo [-h] {bar} test.py foo: error: too few arguments ......x...x.......................x...x.........F ====================================================================== FAIL: test_redirection_completion (__main__.TestBashGlobal.test_redirection_completion) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 1342, in test_redirection_completion self.assertEqual(msg, "success\n") AssertionError: 'failure\n' != 'success\n' - failure + success ---------------------------------------------------------------------- Ran 87 tests in 65.577s FAILED (failures=1, expected failures=4) ./test/test.py -f 32.20s user 4.25s system 54% cpu 1:06.37 total ```

and when I try current main of cpython (v3.14.0a2-70-ga4d4c1ede21) built with "nogil" support -- I get this error. So it is, as I hinted, some patches in debian picked up from current "main" of cpython

```shell Checked 112 modules (34 built-in, 74 shared, 1 n/a on linux-x86_64, 0 disabled, 3 missing, 0 failed on import) make -j 10 269.65s user 29.03s system 1221% cpu 24.444 total ❯ ln -s python ./python3 ❯ ln -s python ./python3.14.0a2+nogil ❯ ./python --version Python 3.14.0a2+ ❯ cd - ❯ export PATH=/home/yoh/proj/misc/cpython-3.14.0a2+70+nogil:$PATH ❯ cd ../argcomplete Authors.rst LICENSE.rst Makefile README.rst argcomplete/ common.mk docs/ setup.cfg test/ Changes.rst MANIFEST.in NOTICE SECURITY.md argcomplete.egg-info/ contrib/ pyproject.toml setup.py* venv/ ❯ py=3.14.0a2+nogil; d=venv/$py; python$py -m venv $d && source $d/bin/activate && pip install -e . && pip install pexpect && ./test/test.py -f python$py -m venv $d 6.66s user 0.13s system 99% cpu 6.798 total Obtaining file:///home/yoh/proj/misc/argcomplete Installing build dependencies ... done Checking if build backend supports build_editable ... done Getting requirements to build editable ... done Preparing editable metadata (pyproject.toml) ... done Building wheels for collected packages: argcomplete Building editable for argcomplete (pyproject.toml) ... done Created wheel for argcomplete: filename=argcomplete-3.5.1-0.editable-py3-none-any.whl size=12819 sha256=95ac218a96599214a1b2f48fa3355b3c0dd01fe19ce3bff3999ab866ae239470 Stored in directory: /home/yoh/.tmp/pip-ephem-wheel-cache-48al5mba/wheels/ac/3d/d9/e3290233d2be26ecfe28f8109144c11fdc602a93e9a7b7adb5 /home/yoh/proj/misc/argcomplete/venv/3.14.0a2+nogil/lib/python3.14t/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'} return cls(files, info_location) Successfully built argcomplete Installing collected packages: argcomplete Successfully installed argcomplete-3.5.1 pip install -e . 7.44s user 0.23s system 95% cpu 8.007 total Collecting pexpect Using cached pexpect-4.9.0-py2.py3-none-any.whl.metadata (2.5 kB) Collecting ptyprocess>=0.5 (from pexpect) Using cached ptyprocess-0.7.0-py2.py3-none-any.whl.metadata (1.3 kB) Using cached pexpect-4.9.0-py2.py3-none-any.whl (63 kB) Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB) /home/yoh/proj/misc/argcomplete/venv/3.14.0a2+nogil/lib/python3.14t/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'} return cls(files, info_location) Installing collected packages: ptyprocess, pexpect /home/yoh/proj/misc/argcomplete/venv/3.14.0a2+nogil/lib/python3.14t/site-packages/pip/_internal/metadata/importlib/_dists.py:77: DeprecationWarning: Unimplemented abstract methods {'locate_file'} return cls(files, info_location) Successfully installed pexpect-4.9.0 ptyprocess-0.7.0 E ====================================================================== ERROR: test_action_activation (__main__.TestArgcomplete.test_action_activation) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 250, in test_action_activation self.assertEqual(set(self.run_completer(make_parser(), cmd)), set(output)) ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^ File "/home/yoh/proj/misc/argcomplete/./test/test.py", line 139, in run_completer completer(parser, output_stream=t, exit_method=sys.exit, **kwargs) ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 174, in __call__ completions = self._get_completions(comp_words, cword_prefix, cword_prequote, last_wordbreak_pos) File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 223, in _get_completions completions = self.collect_completions(active_parsers, parsed_args, cword_prefix) File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 440, in collect_completions completions += self._get_option_completions(active_parser, cword_prefix) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 326, in _get_option_completions if not self._action_allowed(action, parser): ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^ File "/home/yoh/proj/misc/argcomplete/argcomplete/finders.py", line 336, in _action_allowed for conflict_action in parser._action_conflicts.get(action, []): ^^^^^^^^^^^^^^^^^^^^^^^^ AttributeError: 'MonkeyPatchedIntrospectiveArgumentParser' object has no attribute '_action_conflicts' ---------------------------------------------------------------------- Ran 1 test in 0.005s FAILED (errors=1) ```

[kislyuk]

Thanks for reporting this - I can keep this issue open to track solutions but I am not involved with nor have insight into Debian maintenance and patches for argcomplete, so I'm not going to be able to diagnose or resolve this issue directly.