Bash completion of other commands not working

tiguchi commented 5 years ago

Hi, first of all thanks for this great script!

Unfortunately I ran into an issue where there seems to be a conflict with other Bash completion scripts. I'm not sure if this is some kind of configuration issue, but here's my problem. Bash completion scripts don't work anymore with this extension enabled. For example ssh <TAB> does not list known server names anymore, git <TAB> does not autocomplete git commands etc. It seems _bcpp takes precendence over any other bash completion function.

I'm calling _bcpp with the following arguments:

_bcpp --files --dirs --nocase --readline

I played around with the arguments and it seems removing --files restores the other bash completion features, but unfortunately that makes path completion less useful (e.g. nano /e/d/<TAB> doesn't work anymore).

I did follow installation instructions and made sure that _bcpp is loaded and called after bash_completion scripts are loaded. Only difference is that I added the two setup lines to my ~/.bash_profile script, which in turn is sourced by ~/.bashrc in the very end.

Bash version: GNU bash, version 4.4.19(1)-release
OS: KDE Neon (user edition, latest version)

sio commented 5 years ago

Hello, Thomas! I'm really glad you find this project useful!

The workaround for system's bash-completion needs to be enabled with --override flag which you have explicitly omitted for some reason. Please try adding it and see if that helps.

Now that I think of it the current name may not be the best description for what the flag actually does :) Maybe it would've been better as --wrap or --workaround. What do you think?

tiguchi commented 5 years ago

Hi Vitaly, sorry I forgot to mention, I also used the --default and --override flags before and had the same issue. I actually started out using just the --default flag, this is when I noticed the problem.

I played around with the available bcpp arguments and just figured out that omitting the --files argument fixes the problem.

Following arguments give me exactly the expected behavior:

_bcpp --dirs --nocase --readline --override

It looks like the --files option overrides all previously installed Bash completions?

sio commented 5 years ago

Not exactly. --files defines the default completion - the function to which to fall back if everything else failed. And of course it overwrites whatever the default behavior was before.

I guess the problem is caused by one of the following:

~/.bash_profile actually gets sourced before loading the system's completions. That wouldn't surprise me, after all it was designed to be read before ~/.bashrc if at all.
Your system uses some default completion function that is named differently from the _filedir* functions provided by Debian's bash-completion. In that case my wrapper is not expecting it and can not effectively wrap it.

To debug your case further please provide output of the following commands both on "clean" setup where everything (except bcpp) works fine and after calling _bcpp --defaults:

complete -p
declare (Note that this command will dump values for all of your environment variables and helper functions. By default it should not contain anything sensitive unless you've added it there)

The listings will be rather long, so please add them as separate txt files.

tiguchi commented 5 years ago

Thanks for the debugging instructions. I was a bit concerned about the verbose output of declare, so I simply created a fresh VM install of KDE Neon and reproduced the steps there.

I also made sure to rule out problem number 1.) and added the following two lines to the very end of ~/.bashrc instead of ~/.bash_profile:

source ~/bash-complete-partial-path/bash-completion
_bcpp --defaults

The fresh install of KDE Neon has the same issue with ssh and git autocomplete not working. BTW, KDE Neon is pretty much Ubuntu 18.04 under the hood, including a dedicated repository for KDE.

I diffed the before / after output for complete and noticed that the affected complete statements for git and ssh are missing in the "after" version.

I also diffed the declare output and same here, git related autocomplete functions are missing in the after output.

Complete & Declare Output

complete-after.txt complete-before.txt declare-after.txt declare-before.txt

Inclusion Order

Here's what I could find out about the inclusion order for ~/.bashrc, ~/.bash_profile and bash completion scripts.

1. System-wide Scripts

/etc/profile

# /etc/profile: system-wide .profile file for the Bourne shell (sh(1))
# and Bourne compatible shells (bash(1), ksh(1), ash(1), ...).

if [ "${PS1-}" ]; then
  if [ "${BASH-}" ] && [ "$BASH" != "/bin/sh" ]; then
    # The file bash.bashrc already sets the default PS1.
    # PS1='\h:\w\$ '
    if [ -f /etc/bash.bashrc ]; then
      . /etc/bash.bashrc
    fi
  else
    if [ "`id -u`" -eq 0 ]; then
      PS1='# '
    else
      PS1='$ '
    fi
  fi
fi

if [ -d /etc/profile.d ]; then
  for i in /etc/profile.d/*.sh; do
    if [ -r $i ]; then
      . $i
    fi
  done
  unset i
fi

/etc/bash.bashrc

# System-wide .bashrc file for interactive bash(1) shells.

# To enable the settings / commands in this file for login shells as well,
# this file has to be sourced in /etc/profile.

# If not running interactively, don't do anything
[ -z "$PS1" ] && return

# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize

# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "${debian_chroot:-}" ] && [ -r /etc/debian_chroot ]; then
    debian_chroot=$(cat /etc/debian_chroot)
fi

# set a fancy prompt (non-color, overwrite the one in /etc/profile)
# but only if not SUDOing and have SUDO_PS1 set; then assume smart user.
if ! [ -n "${SUDO_USER}" -a -n "${SUDO_PS1}" ]; then
  PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi

# Commented out, don't overwrite xterm -T "title" -n "icontitle" by default.
# If this is an xterm set the title to user@host:dir
#case "$TERM" in
#xterm*|rxvt*)
#    PROMPT_COMMAND='echo -ne "\033]0;${USER}@${HOSTNAME}: ${PWD}\007"'
#    ;;
#*)
#    ;;
#esac

# enable bash completion in interactive shells
#if ! shopt -oq posix; then
#  if [ -f /usr/share/bash-completion/bash_completion ]; then
#    . /usr/share/bash-completion/bash_completion
#  elif [ -f /etc/bash_completion ]; then
#    . /etc/bash_completion
#  fi
#fi

# sudo hint
if [ ! -e "$HOME/.sudo_as_admin_successful" ] && [ ! -e "$HOME/.hushlogin" ] ; then
    case " $(groups) " in *\ admin\ *|*\ sudo\ *)
    if [ -x /usr/bin/sudo ]; then
    cat <<-EOF
    To run a command as administrator (user "root"), use "sudo <command>".
    See "man sudo_root" for details.

    EOF
    fi
    esac
fi

# if the command-not-found package is installed, use it
if [ -x /usr/lib/command-not-found -o -x /usr/share/command-not-found/command-not-found ]; then
    function command_not_found_handle {
            # check because c-n-f could've been removed in the meantime
                if [ -x /usr/lib/command-not-found ]; then
           /usr/lib/command-not-found -- "$1"
                   return $?
                elif [ -x /usr/share/command-not-found/command-not-found ]; then
           /usr/share/command-not-found/command-not-found -- "$1"
                   return $?
        else
           printf "%s: command not found\n" "$1" >&2
           return 127
        fi
    }
fi

1. .profile

KDE Neon has a ~/.profile script that loads ~/.bashrc if present:

# ~/.profile: executed by the command interpreter for login shells.
# This file is not read by bash(1), if ~/.bash_profile or ~/.bash_login
# exists.
# see /usr/share/doc/bash/examples/startup-files for examples.
# the files are located in the bash-doc package.

# the default umask is set in /etc/profile; for setting the umask
# for ssh logins, install and configure the libpam-umask package.
#umask 022

# if running bash
if [ -n "$BASH_VERSION" ]; then
    # include .bashrc if it exists
    if [ -f "$HOME/.bashrc" ]; then
        . "$HOME/.bashrc"
    fi
fi

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/bin" ] ; then
    PATH="$HOME/bin:$PATH"
fi

# set PATH so it includes user's private bin if it exists
if [ -d "$HOME/.local/bin" ] ; then
    PATH="$HOME/.local/bin:$PATH"
fi

I would take the comment about "only loaded when ~/.bash_profile is missing" with a grain of salt. In my tests on my machine it seems both files are loaded. In the VM I simply don't use a .bash_profile file, so there's only .profile and .bashrc

2. .bashrc

.bashrc loads first system default Bash completions and user-specific ones. In the end it loads ~/.bash_profile

# ~/.bashrc: executed by bash(1) for non-login shells.
# see /usr/share/doc/bash/examples/startup-files (in the package bash-doc)
# for examples

# If not running interactively, don't do anything
case $- in
    *i*) ;;
      *) return;;
esac

# don't put duplicate lines or lines starting with space in the history.
# See bash(1) for more options
HISTCONTROL=ignoreboth

# append to the history file, don't overwrite it
shopt -s histappend

# for setting history length see HISTSIZE and HISTFILESIZE in bash(1)
HISTSIZE=1000
HISTFILESIZE=2000

# check the window size after each command and, if necessary,
# update the values of LINES and COLUMNS.
shopt -s checkwinsize

# If set, the pattern "**" used in a pathname expansion context will
# match all files and zero or more directories and subdirectories.
#shopt -s globstar

# make less more friendly for non-text input files, see lesspipe(1)
[ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)"

# set variable identifying the chroot you work in (used in the prompt below)
if [ -z "${debian_chroot:-}" ] && [ -r /etc/debian_chroot ]; then
    debian_chroot=$(cat /etc/debian_chroot)
fi

# set a fancy prompt (non-color, unless we know we "want" color)
case "$TERM" in
    xterm-color|*-256color) color_prompt=yes;;
esac

# uncomment for a colored prompt, if the terminal has the capability; turned
# off by default to not distract the user: the focus in a terminal window
# should be on the output of commands, not on the prompt
#force_color_prompt=yes

if [ -n "$force_color_prompt" ]; then
    if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
    # We have color support; assume it's compliant with Ecma-48
    # (ISO/IEC-6429). (Lack of such support is extremely rare, and such
    # a case would tend to support setf rather than setaf.)
    color_prompt=yes
    else
    color_prompt=
    fi
fi

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi
unset color_prompt force_color_prompt

# If this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
    PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1"
    ;;
*)
    ;;
esac

# enable color support of ls and also add handy aliases
if [ -x /usr/bin/dircolors ]; then
    test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)"
    alias ls='ls --color=auto'
    #alias dir='dir --color=auto'
    #alias vdir='vdir --color=auto'

    alias grep='grep --color=auto'
    alias fgrep='fgrep --color=auto'
    alias egrep='egrep --color=auto'
fi

# colored GCC warnings and errors
#export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01'

# some more ls aliases
alias ll='ls -alF'
alias la='ls -A'
alias l='ls -CF'

# Add an "alert" alias for long running commands.  Use like so:
#   sleep 10; alert
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"'

# Alias definitions.
# You may want to put all your additions into a separate file like
# ~/.bash_aliases, instead of adding them here directly.
# See /usr/share/doc/bash-doc/examples in the bash-doc package.

if [ -f ~/.bash_aliases ]; then
    . ~/.bash_aliases
fi

# enable programmable completion features (you don't need to enable
# this, if it's already enabled in /etc/bash.bashrc and /etc/profile
# sources /etc/bash.bashrc).
if ! shopt -oq posix; then
  if [ -f /usr/share/bash-completion/bash_completion ]; then
    . /usr/share/bash-completion/bash_completion
  elif [ -f /etc/bash_completion ]; then
    . /etc/bash_completion
  fi
fi

source ~/.bash_profile

sio commented 5 years ago

Thank you, now I understand what's happening there. Your setup uses dynamic loading of completions with _completion_loader. I haven't encountered that before, and of course did not add a wrapper for that to my script.

I have an idea on how to make both completions to play along nicely. I'll edit my script and get back to you for testing.

As for the order of inclusion of startup scripts - my personal opinion is that the default documented behavior should've never been modified by distro maintainers "for users' convenience". Now we have this tangled mess of almost circular imports that looks too scary to debug :)

tiguchi commented 5 years ago

Awesome, looking forward to your update!

sio commented 5 years ago

This version should work fine with your dynamic completion loader. Please test it and I'll merge it to master if it's okay.

Partial path completion will work for most, but not all commands unfortunately. Some completion scripts within bash-completion project do not rely on global _filedir* functions but instead implement their own logic for path completion. For such scripts there is no simple way for me to inject my path completion without disabling the rest of their functionality.

If for any particular command you'll prefer my completion over its provided completion script, you can run complete -F _bcpp_complete_file YOURCOMMANDNAME (or add that to ~/.bashrc)

tiguchi commented 5 years ago

If for any particular command you'll prefer my completion over its provided completion script, you can run complete -F _bcpp_complete_file YOURCOMMANDNAME (or add that to ~/.bashrc)

This will come in handy!

I just tested the fixed version and it works. Thank you so much! I'm closing this issue now :-)

sio commented 5 years ago

Great! I've merged the changes into master

sio / bash-complete-partial-path