Docker completion scripts should be easier to install

[silvin-lubecki]

There is no simple and built-in way to install the completion scripts (see the bash script installation).

A better way would be to output the scripts with a docker command, like kubectl does:

$ docker completion bash
...
# install the zsh completion
$ echo "source <(docker completion zsh)" >> ~/.zshrc

[silvin-lubecki]

cc @vdemeester @thaJeztah @albers

[thaJeztah]

Don't the .deb and .rpm packages already install the completion scripts?

[albers]

Bash completion is included in the packages. Don't know about zsh.

For bash, the instructions could be simplified so that they only mention the normal plugin mechanism using the bash_completion.d directory.

There are a lot of problems associated with MacOS. It ships with horribly outdated versions of bash and bash-completion (see http://meta.ath0.com/2012/02/05/apples-great-gpl-purge/) so that every developer has to fiddle with the system to get it usable. Of course there are several ways to do this, complicating install instructions.

For these cases, a docker completion bash|zsh would be very useful.

[thaJeztah]

The difference with kubectl is that they use generated bash completion scripts (using Cobra); useful, but not always the best quality.

In our case, we'd have to figure out how to include the completion scripts in our binaries (downside: updating the bash-completion scripts can only be done if the binary is updated); perhaps similar to how we embed the Compose File schemas? https://github.com/docker/cli/blob/3a8ef767f806a77aace6ae030b1570087ee0a8b5/cli/compose/schema/bindata.go#L191

[vdemeester]

It would also allow to quickly test the completion by just compiling the binary from the PR :stuck_out_tongue:

@thaJeztah yes that's more or less what @silvin-lubecki and I were thinking, embedding the completions script into the binary :angel:

[albers]

This would aslo be a very handy for docker-compose and docker-machine, too. As they do not have packages, installing the completions always involves separate downloads. This could be as easy as docker-compose completion bash > /etc/bash_completion.d/docker-compose.

[thaJeztah]

Even more awesome; publish updated versions as images on Docker Hub (default version embedded);

docker completion update --component=compose > /etc/bash_completion.d/docker-compose

🤔

[silvin-lubecki]

I filled this issue when I was trying to test the new bash completion on docker stack deploy, and it was really painful... It would ease testing and also make life easier for users with their own built cli, I guess.

[albers]

@silvin-lubecki I'm very sorry for that.

[silvin-lubecki]

Lol don't be, it's none of your fault 😉

[ryankemper]

Are there any plans to work on this?

I want simple docker completion zsh, docker completion bash commands so I can source the commands in my config. The existing options are to submodule the entire docker cli repo (really, really bad) or manually curl the completion file and stick it in .zsh/completion (better, but still bad).

Sure, I could write a script to idempotently put the completion file it grabs over https in the right spot, but that's a lot of hassle and I still can't easily track the updates from master without running the script every X minutes. It'd be much more elegant to have the completion command natively.

[thaJeztah]

@ryankemper wondering; how did you install docker? Is this on Linux or Windows/Mac?

The .deb and .rpm packages should install the completion scripts;

https://github.com/docker/docker-ce-packaging/blob/a119f42ef86517dc79e0b3749acdb457a63dea6d/rpm/SPECS/docker-ce-cli.spec#L64-L70

https://github.com/docker/docker-ce-packaging/blob/a119f42ef86517dc79e0b3749acdb457a63dea6d/deb/common/rules#L20-L29

[ryankemper]

@thaJeztah I installed on macos, so I think that involved actually installing a dmg since I haven't found a brew command to do so.

However, I maintain my dotfiles as cross-platform and idempotent, so I would like to be able to always access the completion script natively via a command as long as Docker is installed. That way I don't have to have separate arch linux, macos, fedora, etc install commands; I tend to use a number of different *nix based OS in different contexts so it's important for me to have the most simple system possible.

I'd be willing to try implementing the docker completion $SHELL command myself, but I'll be pretty busy throughout the next few days so I might not be able to start in earnest for a while.

[Thanks for the quick response and the package links, by the way. In the interim, I'm just going to curl the file and manually source it in my config]

[loffelmacher]

I've spent too much time on getting zsh completion to work, and with no success.

[thaJeztah]

Giving this some more thought; I think we should limit scope for now (i.e., the docker completion update https://github.com/docker/cli/issues/1257#issuecomment-409598542 would be cool, but add quite some complexity), and:

• use a flag (--completion) instead of a subcommand (docker completion). I think it makes more sense as a flag, as it's not something a user would be typing on a daily base (more likely it would be either done once, or called as part of, e.g. .profile or .bashrc)
  • the flag would accept a "shell" value (--completion=<bash|zsh|fish|powershell>)
  • potentially we could try to auto-detect the current shell (i.e. --completion would use the detected shell), but we should only do that if it's working well, and doesn't add much complexity (I recall detection was often hit and miss in docker-machine and Docker Toolbox)
• we could start by embedding the current completion scripts in the CLI at compile time, and print them when using the flag; We already use a similar approach to embed the docker compose "json-schema" files; https://github.com/docker/cli/blob/5ad1d4d4c896f7afddd8e3e4ca0d838a1660e36a/cli/compose/schema/bindata.go#L1
• the .deb and .rpm packages could either continue using the current approach (bundle the completion files from this repository, and install them), or execute docker --completion=bash > /path/to/output in the "post-install" step
• documentation will be needed (especially for users that install the static binaries, which currently don't contain the completion scripts)
  • similar to the docker compose documentation

To be discussed:

• the official docker images on Docker Hub could also be updated to include the completion scripts (for users that run them interactively); https://github.com/docker-library/docker/blob/7db7eccb7b7507b1a6ba3ef81c6f37b16149cc55/Dockerfile-dind.template
• completion scripts for plugins:
  • add (optional) --completion flag to each plugin?
  • have a way to "roll up" all completion scripts, including those of plugins, to have a 1-step install to install all of them (--completion components=all,shell=bash ?).
    • is it even possible to "concatenate" all of these to a single file? likely not

[ulyssessouza]

Hello,

I was having a look on how to bypass completion requests from docker/cli and I was wondering why it's not auto-generated from cobra. Is it that it wasn't an option at the time? For the dynamic completion, it can be done through cobra's ValidArgsFunction.

@albers Since you've written the most of the completion scripts, I would like to know if you anything about it.

[thaJeztah]

Is it that it wasn't an option at the time?

At least it wasn't available (the initial bash completion scripts were written before Cobra even existed). So far we've avoided the auto-generated ones, because we had very-well written Bash completion scripts (other shells have varying degrees of quality), but perhaps (with some assistance in the code), it would be possible to generate for Bash, but (imo) it would really depend on the quality of the generated scripts, and the amount of manual patching needed to "make if do the right thing" (thinking, e.g., completion that depends on what OS the daemon runs on; looks like we have some of that).

@ulyssessouza given that you played with the auto-completion generating; how easy would it be for you to generate one from the CLI to see what it looks like?

If the quality for Bash isn't "great", but completion for other shells is equal than what we have currently, we could still decide to "embed" the current Bash completion script (at compile time), and generate the other languages.

[albers]

We have some nice features in the current bash completion script, e.g. completions depending on flags being present on the commandline or not (e.g. docker rm vs docker rm -f). Or completions that can be tuned with env vars. I cannot tell whether it is possible to do things like that in the generate scripts.

If we find someone who is passionate about generated completions and can implement them, we should definitely give it a try, even if it means a minor decay in quality.

I won't be able to do that, can you @ulyssessouza ?

[ulyssessouza]

@albers After some tests I'm able to provide complete dynamic completions even differentiating things like docker rm & docker rm -f. The thing is that it has to be done in one big step for all the commands of the CLI.

For now, we would like to have a simple completion for the compose plugin. So I think the easiest is to add it in the manual bash script for completion.

This PR materializes this work and I would appreciate if can take 2 secs to have a look.

[albers]

I'm currently on vacation. I'll review it next week.

[reitzig]

Seems to have been implemented:

❯ docker version && docker completion fish | head
Client: Docker Engine - Community
 Version:           23.0.3
 API version:       1.42
 Go version:        go1.19.7
 Git commit:        3e7cbfd
 Built:             Tue Apr  4 22:06:10 2023
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          23.0.3
  API version:      1.42 (minimum version 1.12)
  Go version:       go1.19.7
  Git commit:       59118bf
  Built:            Tue Apr  4 22:06:10 2023
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.20
  GitCommit:        2806fc1057397dbaeefbea0e4e17bddfbd388f38
 runc:
  Version:          1.1.5
  GitCommit:        v1.1.5-0-gf19387a
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
# fish completion for docker                               -*- shell-script -*-

function __docker_debug
    set -l file "$BASH_COMP_DEBUG_FILE"
    if test -n "$file"
        echo "$argv" >> $file
    end
end

function __docker_perform_completion

[albers]

@reitzig The feature requested in the OP does exist now, but its presence ist not yet advertised (see output of docker --help | grep completion). The generated completions still require some finishing touches in some places.

In my understanding, this issue should be closed when the completion command is promoted to an official command. @thaJeztah WDYT?

[reitzig]

Ah, good point. Basic documentation is a good minimum requirement. :+1:

The generated completions still require some finishing touches in some places.

Definitely true; at least for fish, image names are not completed. There seem to be a bunch of tickets for several improvements.

[sdondley]

The 200+ line script from docker completion zsh does not seem nearly as good the old 3100+ line script from https://raw.githubusercontent.com/docker/cli/master/contrib/completion/zsh/_docker.

So I'm a little confused how to best implement auto completion.

[marckhouzam]

The 200+ line script from docker completion zsh does not seem nearly as good the old 3100+ line script from https://raw.githubusercontent.com/docker/cli/master/contrib/completion/zsh/_docker.

Is it the fact that the script is much shorter that gives you that impression 🙂? The shorter script no longer hard codes the completions, but instead requests them directly from the docker program. This means completions are now coded in Go instead of in each different shell script language.

So I'm a little confused how to best implement auto completion.

This solution is provided by the Cobra library. Here is the shell completion documentation. I hope it helps: https://github.com/spf13/cobra/blob/main/shell_completions.md#customizing-completions

[marckhouzam]

Hm... I may be completely wrong. I can't tell if Cobra is being used or not to generate the completion scripts...

[sdondley]

@marckhouzam Here's a little background of how I got here: https://stackoverflow.com/questions/76506697/docker-tab-command-completion-not-showing-all-commands

Basically, the latest version of the docker plugin for oh-my-zsh which provides tab completion does not seem to work nearly as well as the previous version. The new version uses docker completion zsh to generate the _docker completion file while the previous version had the 3100 line version of _docker which, as best as I can tell, came from github. The older version, with 3100 lines, works much better.

[marckhouzam]

Someone from the project, knowing the history, would be in a better place to answer this.

From this comment above though, it sounds like there is a need to migrate the features of the completion scripts to the newly available Go completion logic.

Until that is done, my guess is that the original scripts are still to be used instead of the docker completion command?

A similar migration effort was done for kubectl a little while back.

[reitzig]

Quality of the completions is a separate issue, isn't it?

@sdondley I'm not familiar with Cobra, but it seems that some "legacy" commands have been purposefully hiden: cli/command/commands/commands.go#66. I don't know the deprecation plans of Docker CLI, but in principle this seems reasonable to me: the older CLI has a weird mess of top-level shorthand commands and nested commands. Not showing legacy/deprecated commands in completion seems reasonable to me.

That said, the sources seem to suggest the opposite behaviour of what you describe on Stack Overflow: inspect should be hidden but e.g. image shown. huh. :thinking:

FWIW, I've also noted that completion of image and container names doesn't seem to be working as reliably as it used to. Will have to check if that coincides with me using "legacy" commands from that list.