Command-line completion

[ivan-pi]

At some point it would be nice to give fpm command line completion capabilities (like git or other command-line programs).

An example of what the command-completion scripts look like in git can be found here: https://github.com/git/git/tree/328c10930387d301560f7cbcd3351cc485a13381/contrib/completion

A more gentle introduction can be found in various online tutorials, e.g.: https://iridakos.com/programming/2018/03/01/bash-programmable-completion-tutorial

[urbanjost]

bash-shell completion scripts (using complete(1) and compgen(1)) would be very nice, and bash is almost ubiquitous except for some MSWindows environments. It could proceed as a separate project and not require code changes, just a stable CLI interface to describe. So it seems like a great idea.

But I was wondering what prompted the desire, as maybe there are a few other changes to fpm(1) that would help even in the DOS Programming Environment. Assuming it might be the long descriptive keywords

• would short names for the command options help, so you could enter "fpm run -EL -R 'ls -l'" instead of "fpm run --example --list --runner 'ls -l'?

• would something like response files, as described in the M_CLI2 documentation be useful? That was actually for possible use by fpm as proposed by @awvwgk ; but it is not activated and might need some tweeking. As-is you can make abbreviations for your favorite commands and then enter "fpm @itest" instead of "fpm test --compiler ifort --runner time" and so on. It should be relatively platform independent as well.

• I am not a big user of complete and compgen so I am not sure if it is remotely feasible but I am wondering if a CLI parser like M_CLI2, FLAP, kracken, getopt_long_options , ... can generate at least the beginnings of a bash command completion script. If it could, that seems like a nice idea for adding to M_CLI2.

[ivan-pi]

But I was wondering what prompted the desire, as maybe there are a few other changes to fpm(1) that would help even in the DOS Programming Environment. Assuming it might be the long descriptive keywords

I don't really have a full concept for what type of completions.

My favorite would be if fpm build <tab><tab> would list the available targets. Currently whenever I forget the name of my build targets I have to do:

fpm build --list
fpm build <my_target>

It might also be nice to have simple command completions like fpm b<tab> to list commands starting with b.

I also like how git <tab><tab> lists the available subcommands. I think it would be nice if fpm <tab><tab> would do the same.

On the other hand fpm<tab><tab> (no space) could be saved for listing plugins like fpm-search and other future programs.

[urbanjost]

I don't think the targets can be listed by bash completion like a pathname could, but in the current release if you enter a name you know does not exist for run you get a compact list of basenames. In a proposed PR #370 you can enter quoted glob strings like

fpm run '*' 
fpm run '*demo*'

for run and test targets and on the --list argument and (if there is more than one target) you get a list of the basenames. Using \\ in the program itself as an alias for that would probably get trapped by people using completion; but have been thinking that "." might be an alias for "'*'". There is also a --all switch but I am not sure if that is generally supported or not, but it lets you easily do what is the default now without using "special" characters like an asterisk (hence thinking about allowing "." and removing --all). It sounds like those changes help with the problems.

Was thinking of allowing for a menu more like autocompletion but did not add it as as in the discussions it came up that something fancier that would be free (to a greater extent, anyway) to use things like ncurses or shell commands would probably be done better with a plug-in like fpm-search can be used in #364(?).

Note the only auto completion I know of is in the OS or shell, not directly in the programs; but I have not looked that hard.

[everythingfunctional]

I know of one command line parser (for a different language) that has bash completion built in: optparse-applicative. This is actually what I had used for the Haskell implementation, but hadn't turned on the bash completion. So there is precedent for having the bash-completion built in. I don't know all of the intricacies with how bash completion works though, so not sure how difficult it is.

[urbanjost]

Just what I was thinking from what I can tell so far; but more involved than I hoped. Looks like adding something that would just expand keywords and filenames for a regular command would be relatively straight-forward; but doing it with subcommands with M_CLI2 as it currently works totally automatically seems complicated. Maybe writing a program to take in a description of the form cmd [a|b|c]|[--help|--version] like is often in help text (at least in man-pages) would be generically useful. Some interesting possibilities. Unfortunately not trivial ones. At least, not for automating generation of the files. The original proposal seems like the better plan for the foreseeable future.

[LKedward]

A quick and dirty bash completion demo for fpm targets based on this answer.

https://user-images.githubusercontent.com/26024234/109632892-6d30c180-7b3f-11eb-86f9-65939540d2b1.mp4

```bash #!/usr/bin/env bash # # _fpm_run_completion() { _opts=$(ffpm build --list 2>&1 | grep app | grep -v '\.' | cut -d/ -f4|tr '\n' ' ') COMPREPLY=() cur="${COMP_WORDS[COMP_CWORD]}" COMPREPLY=( $( compgen -W "${_opts}" -- ${cur} )) return 0 } complete -F _fpm_run_completion fpm run ```

[ivan-pi]

Cool! I would need a day to understand how or why that bash scripts works :joy:. (hopefully we can find some reviewers for PR's)

How does one ship such a bash-completion script?

[LKedward]

How does one ship such a bash-completion script?

I imagine this is where system package managers become useful, though I know very little about distribution best-practices. The next best option may be to have the install.sh script copy the completion scripts to the correct system location

[awvwgk]

For Unix there are two possibilities:

1. sourcing a script at startup, this is usually placed in /etc/profile.d
2. Running eval "$(fpm bashcompletion)" at shell startup, where fpm-bashcompletion might be a plugin or fpm intrinsic to print the required bash functions

[ivan-pi]

Perhaps slightly tangential, but the Conda installer (or conda init command) typically add a section similar to this one to the ~/.bashrc settings:

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/opt/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
    eval "$__conda_setup"
else
    if [ -f "/opt/miniconda3/etc/profile.d/conda.sh" ]; then
        . "/opt/miniconda3/etc/profile.d/conda.sh"
    else
        export PATH="/opt/miniconda3/bin:$PATH"
    fi
fi
unset __conda_setup
# <<< conda initialize <<<

(For system wide installations a symlink can be made instead like sudo ln -s /opt/conda/etc/profile.d/conda.sh /etc/profile.d/conda.sh):

The purpose is the following:

Expand

``` Key parts of conda's functionality require that it interact directly with the shell within which conda is being invoked. The `conda activate` and `conda deactivate` commands specifically are shell-level commands. That is, they affect the state (e.g. environment variables) of the shell context being interacted with. Other core commands, like `conda create` and `conda install`, also necessarily interact with the shell environment. They're therefore implemented in ways specific to each shell. Each shell must be configured to make use of them. This command makes changes to your system that are specific and customized for each shell. To see the specific files and locations on your system that will be affected before, use the '--dry-run' flag. To see the exact changes that are being or will be made to each location, use the '--verbose' flag. IMPORTANT: After running `conda init`, most shells will need to be closed and restarted for changes to take effect. ```

The contents of the file /opt/miniconda3/etc/profile.d/conda.sh are:

Expand

```bash export CONDA_EXE='/opt/miniconda3/bin/conda' export _CE_M='' export _CE_CONDA='' export CONDA_PYTHON_EXE='/opt/miniconda3/bin/python' # Copyright (C) 2012 Anaconda, Inc # SPDX-License-Identifier: BSD-3-Clause __add_sys_prefix_to_path() { # In dev-mode CONDA_EXE is python.exe and on Windows # it is in a different relative location to condabin. if [ -n "${_CE_CONDA}" ] && [ -n "${WINDIR+x}" ]; then SYSP=$(\dirname "${CONDA_EXE}") else SYSP=$(\dirname "${CONDA_EXE}") SYSP=$(\dirname "${SYSP}") fi if [ -n "${WINDIR+x}" ]; then PATH="${SYSP}/bin:${PATH}" PATH="${SYSP}/Scripts:${PATH}" PATH="${SYSP}/Library/bin:${PATH}" PATH="${SYSP}/Library/usr/bin:${PATH}" PATH="${SYSP}/Library/mingw-w64/bin:${PATH}" PATH="${SYSP}:${PATH}" else PATH="${SYSP}/bin:${PATH}" fi \export PATH } __conda_hashr() { if [ -n "${ZSH_VERSION:+x}" ]; then \rehash elif [ -n "${POSH_VERSION:+x}" ]; then : # pass else \hash -r fi } __conda_activate() { if [ -n "${CONDA_PS1_BACKUP:+x}" ]; then # Handle transition from shell activated with conda <= 4.3 to a subsequent activation # after conda updated to >= 4.4. See issue #6173. PS1="$CONDA_PS1_BACKUP" \unset CONDA_PS1_BACKUP fi \local cmd="$1" shift \local ask_conda CONDA_INTERNAL_OLDPATH="${PATH}" __add_sys_prefix_to_path ask_conda="$(PS1="$PS1" "$CONDA_EXE" $_CE_M $_CE_CONDA shell.posix "$cmd" "$@")" || \return $? rc=$? PATH="${CONDA_INTERNAL_OLDPATH}" \eval "$ask_conda" if [ $rc != 0 ]; then \export PATH fi __conda_hashr } __conda_reactivate() { \local ask_conda CONDA_INTERNAL_OLDPATH="${PATH}" __add_sys_prefix_to_path ask_conda="$(PS1="$PS1" "$CONDA_EXE" $_CE_M $_CE_CONDA shell.posix reactivate)" || \return $? PATH="${CONDA_INTERNAL_OLDPATH}"export CONDA_EXE='/opt/miniconda3/bin/conda' export _CE_M='' export _CE_CONDA='' export CONDA_PYTHON_EXE='/opt/miniconda3/bin/python' # Copyright (C) 2012 Anaconda, Inc # SPDX-License-Identifier: BSD-3-Clause __add_sys_prefix_to_path() { # In dev-mode CONDA_EXE is python.exe and on Windows # it is in a different relative location to condabin. if [ -n "${_CE_CONDA}" ] && [ -n "${WINDIR+x}" ]; then SYSP=$(\dirname "${CONDA_EXE}") else SYSP=$(\dirname "${CONDA_EXE}") SYSP=$(\dirname "${SYSP}") fi if [ -n "${WINDIR+x}" ]; then PATH="${SYSP}/bin:${PATH}" PATH="${SYSP}/Scripts:${PATH}" PATH="${SYSP}/Library/bin:${PATH}" PATH="${SYSP}/Library/usr/bin:${PATH}" PATH="${SYSP}/Library/mingw-w64/bin:${PATH}" PATH="${SYSP}:${PATH}" else PATH="${SYSP}/bin:${PATH}" fi \export PATH } __conda_hashr() { if [ -n "${ZSH_VERSION:+x}" ]; then \rehash elif [ -n "${POSH_VERSION:+x}" ]; then : # pass else \hash -r fi } __conda_activate() { if [ -n "${CONDA_PS1_BACKUP:+x}" ]; then # Handle transition from shell activated with conda <= 4.3 to a subsequent activation # after conda updated to >= 4.4. See issue #6173. PS1="$CONDA_PS1_BACKUP" \unset CONDA_PS1_BACKUP fi \local cmd="$1" shift \local ask_conda CONDA_INTERNAL_OLDPATH="${PATH}" __add_sys_prefix_to_path ask_conda="$(PS1="$PS1" "$CONDA_EXE" $_CE_M $_CE_CONDA shell.posix "$cmd" "$@")" || \return $? rc=$? PATH="${CONDA_INTERNAL_OLDPATH}" \eval "$ask_conda" if [ $rc != 0 ]; then \export PATH fi __conda_hashr } __conda_reactivate() { \local ask_conda CONDA_INTERNAL_OLDPATH="${PATH}" __add_sys_prefix_to_path ask_conda="$(PS1="$PS1" "$CONDA_EXE" $_CE_M $_CE_CONDA shell.posix reactivate)" || \return $? PATH="${CONDA_INTERNAL_OLDPATH}" \eval "$ask_conda" __conda_hashr } conda() { if [ "$#" -lt 1 ]; then "$CONDA_EXE" $_CE_M $_CE_CONDA else \local cmd="$1" shift case "$cmd" in activate|deactivate) __conda_activate "$cmd" "$@" ;; install|update|upgrade|remove|uninstall) CONDA_INTERNAL_OLDPATH="${PATH}" __add_sys_prefix_to_path "$CONDA_EXE" $_CE_M $_CE_CONDA "$cmd" "$@" \local t1=$? PATH="${CONDA_INTERNAL_OLDPATH}" if [ $t1 = 0 ]; then __conda_reactivate else return $t1 fi ;; *) CONDA_INTERNAL_OLDPATH="${PATH}" __add_sys_prefix_to_path "$CONDA_EXE" $_CE_M $_CE_CONDA "$cmd" "$@" \local t1=$? PATH="${CONDA_INTERNAL_OLDPATH}" return $t1 ;; esac fi } if [ -z "${CONDA_SHLVL+x}" ]; then \export CONDA_SHLVL=0 # In dev-mode CONDA_EXE is python.exe and on Windows # it is in a different relative location to condabin. if [ -n "${_CE_CONDA+x}" ] && [ -n "${WINDIR+x}" ]; then PATH="$(\dirname "$CONDA_EXE")/condabin${PATH:+":${PATH}"}" else PATH="$(\dirname "$(\dirname "$CONDA_EXE")")/condabin${PATH:+":${PATH}"}" fi \export PATH # We're not allowing PS1 to be unbound. It must at least be set. # However, we're not exporting it, which can cause problems when starting a second shell # via a first shell (i.e. starting zsh from bash). if [ -z "${PS1+x}" ]; then PS1= fi fi \eval "$ask_conda" __conda_hashr } conda() { if [ "$#" -lt 1 ]; then "$CONDA_EXE" $_CE_M $_CE_CONDA else \local cmd="$1" shift case "$cmd" in activate|deactivate) __conda_activate "$cmd" "$@" ;; install|update|upgrade|remove|uninstall) CONDA_INTERNAL_OLDPATH="${PATH}" __add_sys_prefix_to_path "$CONDA_EXE" $_CE_M $_CE_CONDA "$cmd" "$@" \local t1=$? PATH="${CONDA_INTERNAL_OLDPATH}" if [ $t1 = 0 ]; then __conda_reactivate else return $t1 fi ;; *) CONDA_INTERNAL_OLDPATH="${PATH}" __add_sys_prefix_to_path "$CONDA_EXE" $_CE_M $_CE_CONDA "$cmd" "$@" \local t1=$? PATH="${CONDA_INTERNAL_OLDPATH}" return $t1 ;; esac fi } if [ -z "${CONDA_SHLVL+x}" ]; then \export CONDA_SHLVL=0 # In dev-mode CONDA_EXE is python.exe and on Windows # it is in a different relative location to condabin. if [ -n "${_CE_CONDA+x}" ] && [ -n "${WINDIR+x}" ]; then PATH="$(\dirname "$CONDA_EXE")/condabin${PATH:+":${PATH}"}" else PATH="$(\dirname "$(\dirname "$CONDA_EXE")")/condabin${PATH:+":${PATH}"}" fi \export PATH # We're not allowing PS1 to be unbound. It must at least be set. # However, we're not exporting it, which can cause problems when starting a second shell # via a first shell (i.e. starting zsh from bash). if [ -z "${PS1+x}" ]; then PS1= fi fi ```

[awvwgk]

Conda is a quite particular case because it tries to modify variables like PS1 to display a modified prompt and such.

[LKedward]

For Unix there are two possibilities:

1. sourcing a script at startup, this is usually placed in /etc/profile.d
2. Running eval "$(fpm bashcompletion)" at shell startup, where fpm-bashcompletion might be a plugin or fpm intrinsic to print the required bash functions

git has a script placed in /usr/share/bash-completion/completions/ (on Ubuntu) — can we do something similar (given root permissions), or is this not recommended?

[awvwgk]

If we get with fpm ever installed in the system prefix this would be preferable, otherwise we should place the completion on install at $PREFIX/share/bash-completion/completions, this would be possible with an extra manifest entry in the install table.

[urbanjost]

for reference /etc/bash_completion.d/ is a directory on Red Hat specifically for an admin to put completion scripts.

Along the lines mentioned, a plugin command like fpm-bash could probably customize the environment without changing user prologue files or installing files and spawn a subshell, but would therefore require repeated use instead of a "one-time" setup. Did not actually try it, but something platform-specific could be a plugin written in any language so I think it could just be a bash script.