[question] Any method to install shell completion automatically after `pip install`?

Freed-Wu commented 1 year ago

As https://github.com/huggingface/huggingface_hub/pull/1207 concerned, Just like entry_point will create /usr/bin/XXX, can user pip install and get shell completion /usr/share/zsh/site-functions/_XXX, /usr/share/bash-completion/completions/XXX, etc automatically? That is to create a setuptools plugin to do this work?

Freed-Wu commented 1 year ago

This is just my 2c:

Solution 1

For developer

They only need add one entry_point named shell_completion to pyproject.toml or setup.cfg or setup.py:

[project.entry-points.shell-completion]
# completion by shtab
foo:bash = "foo --print-completion bash"
foo:zsh = "foo --print-completion zsh"
...
# or completion by click
foo:bash = "_FOO_COMPLETE=bash_source foo"
foo:zsh = "_FOO_COMPLETE=zsh_source foo"
foo:fish = "_FOO_COMPLETE=fish_source foo"
# or like pip
foo:bash = "foo completion --bash"
foo:zsh = "foo completion --zsh"
foo:fish = "foo completion --fish"

After python -m build, foo-0.0.1.dist-info/entry_points.txt (this file is in a wheel) will be

# this is a dosini
[shell-completion]
foo:bash = foo --print-completion bash
foo:zsh = foo --print-completion zsh
...

For the packager

Such as archlinux, nix, msys2, termux or homebrew, they usually package a python project by python-installer:

# download wheel file from PYPI or github release
python -m install --destdir=/a/tmp/dir foo-0.0.1-py3-none-any.whl
# compress /a/tmp/dir to a foo.pkg.tar.zst (archlinux and msys2)
# or other format (such as deb, rpm)

So only python-installer support parsing shell-completion, that is

When it met foo:bash = foo --print-completion bash, it generate a bash completion script by foo --print-completion bash(it can be variable according to OS or whether other package is installed), then according to foo:bash or foo:zsh or foo:fish, it generate foo, _foo, foo.fish according to shell standards, and according to OS, it generate

/usr/share/bash-completion/completions/foo for linux, msys2's msys
/usr/local/share/bash-completion/completions/foo for macOS, BSD
/mingw32/share/bash-completion/completions/foo for msys2's mingw32
mingw64, clang32, clang64 is similar.

For zsh, it is /usr/share/zsh/site-functions/_foo, and /usr/share/fish/vendor_completions.d/foo.fish for fish.

So packagers don't need write any code like https://github.com/archlinux/svntogit-community/blob/packages/python-black/trunk/PKGBUILD#L51-L64

For User

They can install package by pip install --user foo, pip don't use python-build and python-installer, it download wheel from pypi or create one wheel from source code cloned by git and install by itself. So pip also need support this function like python-installer.

Other

Some package like shtab can be extra_requires, which make it may not exist when create shell completion scripts. When foo:bash = a_cmd_exit_with_non_zero, it should give a warning and tell user what happened. If user reinstall by pip install foo[completion] these shell scripts should be created.

Solution 2

When developer use python-build or user use pip install git+URL to create a wheel, it can read pyproject.toml and generate foo-XXsh.

 dist/foo-0.0.1.whl
├──  foo
└──  foo-0.0.1.dist-info
   ├──  shell-completion
   │   ├──  foo-bash
   │   ├──  foo-fish
   │   └──  foo-zsh
   ├──  entry_points.txt
   ├──  LICENSE
   ├──  METADATA
   ├──  RECORD
   ├──  top_level.txt
   └──  WHEEL

Then python-installer, pip will copy/move/symlink these file to correct path for different shell and different OS.

Question

Which solution is more probable to be accepted by the most people (developer, packager, user)? @Wauplin

Moreover

python -m foo can also supported by shell completion. Like git, git allow extension: if git-foo exist in $PATH, git foo will call git-foo. The zsh completion of git define many git-XXX's completions.

For zsh, the first line define the compleiton script is for which program. Such as

#compdef foo

is for foo. We can write python's zsh completion script and let python -m foo call completion of python-m-foo and

#compdef foo python-m-foo

The advantage is it can support those module contain __main__.py but don't have any console script.

Conclusion

I don't know how many people know press <TAB> can get shell completion. It not only reduce the number of user's hittings, but also improve user experience. If python can have better support for shell completion, I think those who often press <TAB> will be happier.

PS: Does there exist any forum that can discuss this thought? PS2: My English is not very good. Any typo is not offense. :smile:

casperdcl commented 1 year ago

Hey, thanks for the feedback! So you'd like pip install thing to auto-detect the shell(s) on the local machine & run the post-pip-install completion setup?

Seems quite complex to cover all cases (different platforms/environments/shells); e.g. what if someone pip installs into a throwaway conda environment on a machine that has both zsh and bash? Some trickery with environment files?

Usually, packaging systems implement post-install completion setup, e.g. snapcraft tab-completion and dh_bash-completion. AFAIK no pure Python packaging system implements this.
For now, IMO better to just document & let the end-user decide (see e.g. https://dvc.org/doc/install/completion#what-shell-do-you-have)?

Still, happy to accept experimental suggestions if you think it's possible to auto-setup things.

Freed-Wu commented 1 year ago

you'd like pip install thing to auto-detect

https://github.com/huggingface/huggingface_hub/pull/1207 proposed to do so and it looks like hard because pip is not a system packager.

no pure Python packaging system implements this

I see.

packaging systems implement post-install

Thanks for introduction.

IMO better to just document & let the end-user decide

I believe most end-user want a shell completion. So an out-of-box may be their need.

Thanks for your answer and looks like you have answer a more detailed answer in https://github.com/huggingface/huggingface_hub/pull/1207. So close this now.

iterative / shtab