Global completion support broken for current `console_scripts` installs

[rpatterson]

The support from argcomplete for globally completing all scripts that support argcomplete doesn't currently work for scripts installed via current setuptools' console_scripts entry point because they don't match the regular expression argcomplete uses to detect such scripts, at least when installed with modern tooling. I examined the echoed commands run by /etc/bash_completion.d/python-argcomplete.sh:

$ complete | grep -- '-project-structure'
$ set -x
...
$ ./.tox/py311/bin/py-project-structure [TAB]
+ local executable=./.tox/py311/bin/py-project-structure
+ __python_argcomplete_expand_tilde_by_ref executable
+ '[' . = '~' ']'
+ local ARGCOMPLETE=0
+ [[ ./.tox/py311/bin/py-project-structure == python* ]]
+ [[ ./.tox/py311/bin/py-project-structure == pypy* ]]
+ which ./.tox/py311/bin/py-project-structure
++ which ./.tox/py311/bin/py-project-structure
+ local SCRIPT_NAME=./.tox/py311/bin/py-project-structure
+ [[ 0 == 1 ]]
+ [[ 0 == 2 ]]
+ grep -q function
+ type -t _completion_loader
+ _completion_loader ./.tox/py311/bin/py-project-structure '' ./.tox/py311/bin/py-project-structure
...

Looking at /etc/bash_completion.d/python-argcomplete.sh, I see that the matching commands all have (...) >/dev/null 2>&1 so I'm assuming that's why we don't see those echoed. I simulated those commands:

$ SCRIPT_NAME=./.tox/py311/bin/py-project-structure
$ head -c 1024 "$SCRIPT_NAME" | egrep --quiet "(PBR Generated)|(EASY-INSTALL-(SCRIPT|ENTRY-SCRIPT|DEV-SCRIPT))"
$ echo $?
1
$ python-argcomplete-check-easy-install-script $SCRIPT_NAME
Command 'python-argcomplete-check-easy-install-script' not found, did you mean:
  command 'python-argcomplete-check-easy-install-script3' from deb python3-argcomplete (1.8.1-1.5)
Try: sudo apt install <deb name>
$ python-argcomplete-check-easy-install-script3 $SCRIPT_NAME
$ echo $?
1

FWIW, I think it's a shame that setuptools, pip, or whoever is currently responsible for installing console_scripts no longer leaves any "fingerprint" comment in the scripts it generates. Such a comment is useful for more than just this purpose. For example, it gives sysadmins or container image users who are not Python developers a chance to figure out where a script came from. Looking at the generated scripts, it would be pretty hard to write the whole file regular expression that would be necessary to identify such scripts without false positives and negatives. So I don't have any helpful suggestions for that.

Even if the match did work, however, $ python-argcomplete-check-easy-install-script still fails to identify the script as supporting completion. Note also that the Debian/Ubuntu package installs that check script with a different name than /etc/bash_completion.d/python-argcomplete.sh references.

In summary, the global support from argcomplete is currently broken for console_scripts in several ways.

[rpatterson]

Here's the workaround I'm using at the expense of Windows compatibility.

[kislyuk]

Global completion works fine with console_scripts (except on Windows, which is not actively supported outside the parts that emulate Linux with complete fidelity). When pip/PyPA stopped placing package markers in console_scripts autogenerated files for wheel installs, we developed a pkg_resources metadata based solution (#241).

If you think there's a bug, please post a complete, standalone reproduction.

[rpatterson]

except on Windows, which is not actively supported outside the parts that emulate Linux with complete fidelity

Of course. By "at the expense of Windows compatibility" I meant that the workaround means the script will no longer work as an executable as installed on Windows, not that the workaround breaks completion that worked on Windows without the workaround.

we developed a pkg_resources metadata based solution (#241).

Thanks for the link! I noticed that I had /etc/bash_completion.d/python-argcomplete.sh from the python3-argcomplete Ubuntu package and that it didn't include the changes from that PR. When I removed that package and installed it via $ pip install, tab completion works. So the issue is an out of date distro/OS package.

It looks like a current version is in the next releases of Debian and Ubuntu so I'm guessing reporting bugs there isn't appropriate. I used the distro/OS package because I wanted the easiest or least intrusive way to enable this on all the hosts I manage. Considering that argcomplete is user oriented in the sense that various user oriented Python packages mention and link to the argcomplete docs for how to enable global completion, maybe a warning in the install docs about using anything other than $ pip install would save users some confusion and time?

[rpatterson]

Meant to include versions:

$ apt policy python3-argcomplete           
python3-argcomplete:                                                                                     
  Installed: 1.8.1-1.5                                                                                   
  Candidate: 1.8.1-1.5                                                                                   
  Version table:                                                                                         
 *** 1.8.1-1.5 500                                                                                       
        500 http://apt.pop-os.org/ubuntu jammy/universe amd64 Packages                                   
        500 http://apt.pop-os.org/ubuntu jammy/universe i386 Packages                                    
        100 /var/lib/dpkg/status                                                                         
$ pip freeze | grep argcomplete                                                                                                                     
argcomplete==2.0.0