Fix ZSH implementation - missing completion

[flu0r1ne]

The ZSH auto-completion was malfunctioning on a fresh Ubuntu 22.04 installation. This issue originated from several problems within the global auto-completion script.

Initially, COMP_WORDS was employed for matching modules and interpreter arguments. However, COMP_WORDS isn't defined in ZSH; words is the correct variable to use instead. To resolve this, an argument vector with base-1 indexing was created to work with both shells.

Secondly, the __python_argcomplete_scan_head function included a defective read command in ZSH during interpreter matching. Using -k with read in ZSH reads a specified num of bytes, regardless of a newline presence. This issue was addressed by introducing a secondary read command in both shells to retrieve the line. Additionally, in ZSH, regex matches are not stored in BASH_REMATCH unless a compatibility option is activated. This was addressed by enabling this option locally. The difference between base-0 and base-1 indexing was mitigated by incrementing BASH_REMATCH when operating in Bash and indexing correctly.

A ZSH-compatible version of __python_argcomplete_expand_tilde_by_ref was also introduced as the prior version utilized zero indexing and Bash-specific indirection syntax.

Moreover, the indexing and variable expansion of the argument array in the __python_argcomplete_run function were correctly adjusted.

Lastly, the path to a Python script was expanded using tilde notation to enable auto-completion when employing:

python ~/some/script.py

TESTED WITH:

GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu) zsh 5.8.1 (x86_64-ubuntu-linux-gnu) Linux ubuntu-22-04-2-desktop 5.19.0-42-generic

[flu0r1ne]

This really should be tested further before merging. I'm going to test it on a different system later tonight. I don't have an OSX machine unfortunately.

[flu0r1ne]

I have re-tested this patch on a fresh installation of Ubuntu 22.04, and all components performed as expected. Additionally, I am currently testing it on Debian 10, as it is likely to have the most outdated versions of zsh and bash that any user might encounter. I anticipate that some minor adjustments may be required, and I plan to provide an update on this tomorrow.

Testing Procedure:

System

Ubuntu 22.04.2 LTS
GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
zsh 5.8.1 (x86_64-ubuntu-linux-gnu)

Installation

git clone https://github.com/kislyuk/argcomplete.git
cd ~/argcomplete
sudo pip install .
sudo activate-global-python-argcomplete
cd test/test_package
sudo pip install .

Bash Argcomplete (pre-patch testing)

$ bash
$ complete | grep 'argcomplete'
complete -o bashdefault -o default -F _python_argcomplete_global -D
$ test-package <tab>
arg     -h      --help
$ python /home/flu0r1ne/argcomplete/test/test_package/test_module.py <tab>
arg     -h      --help
$ python -m test_module<tab>
arg     -h      --help
$ python ~/argcomplete/test/test_package/test_module.py <tab>
[files]

ZSH Argcomplete (pre-patch testing)

$ zsh
$ echo -E $functions[_python-argcomplete]
[prints function]
$ test-package<tab>
[files]
$ python /home/flu0r1ne/argcomplete/test/test_package/test_module.py<tab>
[files]
$ python -m test_module<tab>
[files]
$ python ~/argcomplete/test/test_package/test_module.py<tab>
[files]

Installation from flu0r1ne/argcomplete

cd ~/argcomplete
git remote add flu0r1ne git@github.com:flu0r1ne/argcomplete.git
git fetch flu0r1ne
git checkout flu0r1ne/fix_global_zsh_completion_stemming_from_bash_incompatibilities
sudo pip install .
sudo activate-global-python-argcomplete

Bash Argcomplete

$ bash
$ complete | grep 'argcomplete'
complete -o bashdefault -o default -F _python_argcomplete_global -D
$ test-package <tab>
arg     -h      --help
$ python /home/flu0r1ne/argcomplete/test/test_package/test_module.py <tab>
arg     -h      --help
$ python -m test_module<tab>
arg     -h      --help
$ python ~/argcomplete/test/test_package/test_module.py <tab>
arg     -h      --help

ZSH Argcomplete

$ zsh
$ echo -E $functions[_python-argcomplete]
[prints function]
$ test-package <tab>
arg         --
--help  -h  -- show this help message and exit
$ python /home/flu0r1ne/argcomplete/test/test_package/test_module.py <tab>
[files]
arg         --
--help  -h  -- show this help message and exit
$ python -m test_module<tab>
[files]
arg         --
--help  -h  -- show this help message and exit
$ python ~/argcomplete/test/test_package/test_module.py <tab>
[files]
arg         --
--help  -h  -- show this help message and exit

[codecov-commenter]

Codecov Report

Patch and project coverage have no change.

Comparison is base (f3f856b) 81.29% compared to head (86b8c86) 81.29%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## develop #433 +/- ## ======================================== Coverage 81.29% 81.29% ======================================== Files 10 10 Lines 770 770 ======================================== Hits 626 626 Misses 144 144 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.

[kislyuk]

Thanks. This looks good overall, I appreciate the effort in tracking down the remaining shellcode paths that are not compatible with zsh.

I see a tab on line 161. Please don't use tabs - please replace them with the appropriate number of spaces.

It looks like the test suite is failing. I haven't looked into why yet. Does it pass for you locally? If not, please take a look at why the tests are failing.

[flu0r1ne]

I have resolved several compatibility issues and corrected a bug where the proper interpreter was not being selected in some instances. For further details, please refer to the associated commit message.

However, I had some difficulty understanding the invocation of _describe as shown below:

_describe "$executable" completions -o nosort

This resulted in seemingly arbitrary completion outcomes:

$ test-module
           -a         -d         -h         nosort
-          _a_11      -D         --help     _tmpd
-2V        arg        -default-  -J         _tmpm

After eliminating the -o nosort flags, the issue was resolved. Nevertheless, I remain uncertain of the original intention behind these flags. They didn't seem to be in the ZSH documentation.

[flu0r1ne]

Here are the results from testing on Debian 10.

Testing Procedure:

System

Debian GNU/Linux 10 (buster)
GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)
zsh 5.7.1 (x86_64-debian-linux-gnu)

Installation

git clone https://github.com/kislyuk/argcomplete.git
cd ~/argcomplete
sudo pip3 install .
sudo activate-global-python-argcomplete
cd test/test_package
sudo pip3 install .

Bash Argcomplete (pre-patch testing)

$ bash
$ complete | grep 'argcomplete'
complete -o bashdefault -o default -F _python_argcomplete_global -D
$ test-package <tab>
[files]
$ python3 /home/flu0r1ne/argcomplete/test/test_package/test_module.py <tab>
arg     -h      --help
$ python3 -m test_module<tab>
arg     -h      --help
$ python3 ~/argcomplete/test/test_package/test_module.py <tab>
[files]

ZSH Argcomplete (pre-patch testing)

$ zsh
$ autoload -Uz compinit && compinit
$ echo -E $functions[_python-argcomplete]
builtin autoload -XU
$ test-package<tab>
[files]
$ python3 /home/flu0r1ne/argcomplete/test/test_package/test_module.py<tab>
[files]
$ python3 -m test_module<tab>
[files]
$ python3 ~/argcomplete/test/test_package/test_module.py<tab>
[files]

Installation from flu0r1ne/argcomplete

cd ~/argcomplete
git remote add flu0r1ne https://github.com/flu0r1ne/argcomplete
git fetch flu0r1ne
git branch debian_testing && git checkout debian_testing
git rebase flu0r1ne/add_debian_patches
git rebase flu0r1ne/fix_python3.7_compat
sudo pip3 install .
sudo activate-global-python-argcomplete

Bash Argcomplete

$ bash
$ complete | grep 'argcomplete'
complete -o bashdefault -o default -F _python_argcomplete_global -D
$ test-package <tab>
arg     -h      --help
$ python3 /home/flu0r1ne/argcomplete/test/test_package/test_module.py <tab>
arg     -h      --help
$ python3 -m test_module <tab>
arg     -h      --help
$ python3 ~/argcomplete/test/test_package/test_module.py <tab>
arg     -h      --help

ZSH Argcomplete

$ zsh
$ autoload -Uz compinit && compinit
$ echo -E $functions[_python-argcomplete]
builtin autoload -XU
$ test-package <tab>
arg         --
--help  -h  -- show this help message and exit
$ python3 /home/flu0r1ne/argcomplete/test/test_package/test_module.py <tab>
[files]
arg         --
--help  -h  -- show this help message and exit
$ python3 -m test_module<tab>
[files]
arg         --
--help  -h  -- show this help message and exit
$ python3 ~/argcomplete/test/test_package/test_module.py <tab>
[files]
arg         --
--help  -h  -- show this help message and exit

[flu0r1ne]

I previously rebased develop. However, I noticed that Github is listing two files as having changed. I believe these commits will apply without any issues, but please let me know if there's anything else I need to do.

[kislyuk]

Thank you! I will test this after merging your Python 3.7 importlib backport fix. It will take a while to test, but I'm hoping I can complete the testing and cut a release sometime this weekend.

[kislyuk]

Released in v3.1.0