Request: also read completions from `/usr/local/...` even when installed with prefix `/usr/`

[MestreLion]

After reading the FAQ and a few related issues, I'm aware bash-completion uses traditional GNU Autotools to choose its installation prefix, be it /usr used by distros, the default /usr/local, or something else, and then use this prefix to set the dynamically loaded completions dir to <prefix>/share/bash-completion/completions.

Would it be possible to, besides loading the completions from the install preffix, say /usr, also load them from /usr/local?

Consider a system using bash-completion from distribution package, as virtually all distros already ship it. Now an admin installs some package, using the same GNU Autotools you guys use, in /usr/local, which is the default prefix (and the recommended one, being a local install and not an OS-provided via package manager/repositories)

Now if this package contains a bash-completion file, it will be installed at /usr/local/share/bash-completion/completions, and as such it will not be loaded by the system, as it is only reading from /usr/share/bash-completion/completions!

Can this be implemented?

[MestreLion]

Also, it would be really awkward for such package to install its own files to /usr/local/share and binaries to /usr/local/bin but their bash-completion files to /usr/share/...

[scop]

Possible, sure. And I get the point, but don't have a too good feeling about this, gut feeling being that I'd rather not open that can of worms. Should we also do it the other way around (if installed to /usr/local, also load from /usr)? What if we're installing to /opt or somewhere else entirely?

By the current status and docs (see 4th item in README.md's FAQ), the dir where to install completions should be grabbed from our pkgconfig file -- and we have autotools and cmake helpers for that -- no matter where the rest of the package lands. I agree it's not squeaky clean, but can't say I quite agree with "really awkward".

How do other packages of similar nature deal with this?

[MestreLion]

Thanks for taking the time to reply and consider this request!

Just a small clarification: by "really awkward" I was not referring to how the packages retrieve the bash completion dir: $(pkg-config bash-completion --variable=completionsdir) is quite clean and elegant for me. By awkward I mean the situation such package faces: they're installing all their files to /usr/local prefix, but pkg-config points bash-completion to /usr. Is it OK, or even allowed, for a locally-installed package to drop files in /usr tree?

[MestreLion]

About your question on the real issue, what other packages do?

Most packages avoid this dilemma by using the /etc tree for such files. Since /etc is "shared" among /usr, /usr/local (and even /opt, there's a single place to read "system" config. And that was the approach used by bash-completion in the static/eager loading era. Granted, /etc is meant for configuration files, and completion files are not exactly "config" files, but it is a common solution.

As for software that use "drop-in" dirs in the /usr//usr/local tree, I can think of a few: /usr/share/applications for .desktop files, /usr/share/icons for desktop icons (Gnome, KDE, etc), site-packages/dist-packages for Python modules. And in such cases all packages I'm aware of reads both /usr and /usr/local (and also the user tree in ~/.local, like bash completion already does as well). Taking Python as an example, even if you run it from source in a cloned repository dir, it will still have the 3 locations (system, local and user) in its default $PYTHONPATH/sys.path. And all desktop environments, themselves system-installed in /usr, also read desktop and icon files from /usr/local

[MestreLion]

I truly understand your "gut feeling" about reading from a tree other than the one you're installed in, but the alternative is to require packages to write to a different tree than the one they're installed. And if someone must step out of their install tree, I believe reading is more harmless/conformant than writing.

[scop]

Sure, I understood correctly what you meant by "really awkward" the first time.

The other software examples are useful, thanks. FWIW it seems e.g. Python behaves (well at least the Ubuntu system one and my personal pyenv ones) so that the system one in /usr peeks in both /usr and /usr/local, and the pyenv one in neither.

/etc BTW is out of the question.

Guess pkgconfig would actually be a rather interesting example. Its man page is a bit vague about this (would have to check under what conditions the mentioned "On most systems" would not apply): https://gitlab.freedesktop.org/pkg-config/pkg-config/-/blob/eb866ade77d933c3237a5c9e2114ca65c3baa537/pkg-config.1#L42 -- anyway on "those" systems it checks both /usr and /usr/local.

Then there's the question what would we do about our pkgconfig file. We could conceivably add a new variable, e.g. completiondirs (in plural) there, which would list all those dirs, colon separated. Or a separate localcompletiondir and perhaps systemcompletiondir (with current completiondir pointing to the latter). Choosing between those two in 3rd party packages would be kind of icky though, e.g. "if installing to /usr, install to bash-completion's systemcompletiondir, otherwise localcompletiondir". And if using the colon separated option, how does one choose between those? If just grab the first, it'll be wrong for either system or local installed packages and we haven't actually accomplished much at all.

The system installed pkg-config itself does this on my Ubuntu 18.04:

$ pkg-config --variable pc_path pkg-config
/usr/local/lib/x86_64-linux-gnu/pkgconfig:/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig

If system installed 3rd party packages would grab their .pc install dir from here, they'd all be wrong unless there are some /usr/local avoidance hacks somewhere. Guess it would be useful to research some more into this, but I've no time for that ATM.

We could add a simple hack in the code to look from /usr/local too if our own prefix is /usr, but that would not help 3rd party packages' install dirs.

See the can of worms already? ;)

[MestreLion]

Yes, I can see it now, lol.

Still, you could simply leave pkg-config out of this and keep your .pc it as it is. It only has your "main" completion dir, but nothing prevents you from reading other places too.

I mean, most software works this way already, no? They read their shared data from a predefined set of paths, and client packages are assumed/required to know the path they'll write to. pkg-config is a convenience, not a requirement. There's no pkg-config for /etc/apt/sources.list.d, or /etc/apache2/sites-available, or /usr/local/share/applications. Just like there's no pkg-config for the user directory ~/.local/share/bash-completion that bash-completion also reads from.

[gadefox]

The system installed pkg-config itself does this on my Ubuntu 18.04:

$ pkg-config --variable pc_path pkg-config
/usr/local/lib/x86_64-linux-gnu/pkgconfig:/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/lib/x86_64-linux-gnu/pkgconfig:/usr/lib/pkgconfig:/usr/share/pkgconfig

Please note this is the hardcoded path list which means you can't change the value. Of course you can build `pkg-config` from the source in case of your custom folder list..

> See the can of worms already? ;)

It's the responsibility of the admin (or the ordinary user) to update `PKG_CONFIG_PATH` when (s)he installs new package(s) to custom folders e.g. `/opt`, `~/.local/lib/pkgconfig` etc... Then `pkg-config` would use the hardcoded list and the list used by this env variable and it'd find the `pc` file..

[gadefox]

I'm little bit confused. What is the purpose of bash-completion.pc(?), cause I can see the hardcoded paths in the script:

for dir in ${XDG_DATA_DIRS:-/usr/local/share:/usr/share}; do
  dirs+=($dir/bash-completion/completions)
done

It would be useful to use one variable dirs (foreach $dir/completions, $dir/helpers in dirs) instead of completionsdir/helpersdir in the pkg-config file which could be used by the script..