better packaging & easier setup of hledger shell completions

[simonmichael]

[2013-09-12 earlier issue: bash, zsh, *sh auto-completion support #136]

[2019-03-09 this issue, original title: built-in command line completions] Here's an example where the command line (bash) completions are built in, for convenience. Something like this would make hledger's command line completions available to more folks.

You can as well use autocomplete of your shell with following command:

source <(chronos --bash-completion-script `which chronos`)

[2019-10-19 renamed: can we provide shell completions as part of hledger, for easier setup ?]

[2024-09-13 renamed: better packaging & easier setup of hledger shell completions]

[schoettl]

I think we only need to package the existing script into the distro packages.

People who install hledger using the source code can as well add source ~/software/hledger/shell-completion/hledger-completion.bash to their .bashrc.

Other people install it with their package manager and in future hopefully have the completion for free without doing anything.

How is the release cycle with hledeger? Do you know when package maintainers create a new version for hledger?

Last time, I asked at some distros, they closed/deferred the issue because the completion script was not yet in the sources they used for the package...

Status:

• Ubuntu told me to submit this issue to debian repos: https://answers.launchpad.net/ubuntu/+source/haskell-hledger/+question/678127
• Debian told me that the completion script must first be in the sources they use for the release.
• AUR told me nothing - I don't know who is the package maintainer: https://aur.archlinux.org/packages/hledger-git/#news
• Arch deferred the task, I guess for the same reason Debian did: https://bugs.archlinux.org/task/61546

[simonmichael]

On Mar 10, 2019, at 6:20 AM, Jakob Schöttl notifications@github.com wrote: I think we only need to package the existing script into the distro packages.

People who install hledger using the source code can as well add source ~/software/hledger/shell-completion/hledger-completion.bash to their .bashrc.

Other people install it with their package manager and in future hopefully have the completion for free without doing anything.

That would be ideal, but consider what a small proportion of hledger users that applies to, at least right now. Currently no package does this. Many people build from source since packages are too old, or use hledger-install because they want something easy, or download binaries for windows.

How is the release cycle with hledeger? Do you know when package maintainers create a new version for hledger?

hledger's usual release window is the the first day or two of each month. We don't yet have many active packagers; some updates come quickly, some come when you prod them, others come very rarely. I've been working to make the situtation clearer on the download page. Last time, I asked at some distros, they closed/deferred the issue because the completion script was not yet in the sources they used for the package...

That's a good point.. packagers work from the package tarball on Hackage. We should add the completion scripts to that, by adding the files to hledger/package.yaml (extra-source-files or data-files or something else, I can never remember).

Status:

Ubuntu told me to submit this issue to debian repos: https://answers.launchpad.net/ubuntu/+source/haskell-hledger/+question/678127 https://answers.launchpad.net/ubuntu/+source/haskell-hledger/+question/678127 Debian told me that the completion script must first be in the sources they use for the release. AUR told me nothing - I don't know who is the package maintainer: https://aur.archlinux.org/packages/hledger-git/#news https://aur.archlinux.org/packages/hledger-git/#news Arch deferred the task, I guess for the same reason Debian did: https://bugs.archlinux.org/task/61546 https://bugs.archlinux.org/task/61546Great, thanks for pursuing that.

I think maybe we should have packager names/contact links/issue tracker links on the download page.

[ony]

Looks like duplicate of #136

[simonmichael]

This topic is a bit confusing, but I think this issue is about having the existing shell completions built in to the hledger executable, for easier setup. I've edited to make that clearer.

[hseg]

Either option (adding the completion to the bindist or embedding it into the binary) would be OK, but please do one of them. I'm currently pushing back on the AUR packager for hledger-bin, but that wouldn't be necessary if the file were in the bindist to begin with.

[simonmichael]

Is there a user of shell completions out there who’d like to work on this ?

[simonmichael]

PS, since I never use shell completions (I use emacs shells, where they don't work), could y'all motivate them a little, by showing some examples of how they help you ? In my hledger usage, I'm not typing long commands or options and I'm not sure how helpful shell completions can be here.

[hseg]

Partially to discover options -- this is my primary way of discovering subcommands for a new program before cracking open the manual. Partially to make sure I make syntactically-correct invocations (and thus avoid typos). But mostly my motivation here is that given that it's there, it's a shame not to use it fully.

[simonmichael]

Thanks for the response. I don't think we should spend effort on this if it's not actually useful. I'm too lazy to figure out how to set this up again and try it for myself, but discovering options does sound like a good use case. That's instead of cracking open --help, I guess.

[simonmichael]

(Or --tldr).

[hseg]

Do note I'm barely proficient with hledger, so my experience is probably not indicative.

[schoettl]

I still use completions a lot

1. to (re)discover options and commands,
2. to complete long options and commands,
3. and most importantly to discover and complete account and tag names.

I use it every second when I use hledger – just because it feels so natural – the same way I use filename completion in bash.

But I also have other priorities and can't work on it. I have my working setup with the completions sourced in my .bashrc for years.

[simonmichael]

discover and complete account and tag names

That sounds cool too, I didn't know shell completions could do that. But how does it work, when shell completions are a static file saved somewhere ?

[schoettl]

That sounds cool too, I didn't know shell completions could do that. But how does it work, when shell completions are a static file saved somewhere ?

There are callback functions that are registered for hledger in hledger-completion.bash which scan the selected journal file and then generate the completions.

[simonmichael]

I'm struggling to get shell completions working, so here are some notes.

@schoettl I re-read your clever scripts (https://github.com/simonmichael/hledger/tree/master/hledger/shell-completion), and the README. I regenerated and installed the script with make install. I started a new bash shell, then I type hledger bal<TAB> and no completion happens. I'm on macos sonoma. I saw there's a bash-completion package in homebrew, but it's already installed.

I have both Apple's /bin/bash (GNU bash, version 3.2.57) and /opt/homebrew/bin/bash (GNU bash, version 5.2.21).

In both of them this command recommended in the README fails:

~/src/hledger/hledger/shell-completion$ type __load_completion
bash: type: __load_completion: not found

and after running the completions script manually, completing gives this error:

~/src/hledger/hledger/shell-completion$ source hledger-completion.bash
~/src/hledger/hledger/shell-completion$ hledger balbash: _init_completion: command not found

Any help welcome. It would be great to document a reliable install process so more people might use these.

[simonmichael]

Got it working on mac for the first time, by brew uninstall bash-completion; brew install bash-completion@2. Progress!

homebrew's default bash-completion formula is 1.3, intended to work with Apple's default bash 3 (but it doesn't; when installed, /bin/bash gives bash: complete: nosort: invalid option name errors). With newer bash, like homebrew's bash 5, the bash-completion@2 formula should be used instead.

I'm working on some process and install docs updates.

[schoettl]

Nice to that you got it working! I know that MacOS by default has a very old version of Bash. BTW, @zhelezov did great contributions on top of my initial draft.

[simonmichael]

You both did great work, the completions are amazing!

I have documented setup and usage at https://hledger.org/install.html#shell-completions . Any testing and improvements are welcome.

[simonmichael]

Shell completions are quite a deep rabbit hole. Some notes:

I did a bunch more testing on mac.

I updated the Install > Shell completions doc again.

I updated the bash completions generating scripts.

I updated the bash completions on the 1.40-branch, and added this step to RELEASING.

I merged #2223, so the bash completions will be included in future release bindists on Github.

I noted some issues:

• The hledger-ui and hledger-web symlinks seem unnecessary since the hledger script also registers itself for hledger-ui and hledger-web.
• Quoting of completions doesn't always work right. Eg completion of amt:<. Also completion of cur:$ (backslash is inserted in wrong place, and it should insert two of them).
• inacct: doesn't suggest account names. That's probably fine, it's intended for internal use by hledger-web.

and improvement ideas:

• not: has no completions; it could suggest another query prefix.
• aregister could suggest an account name rather than a query prefix for its first argument.
• In hledger queries, real: is equivalent to real:1 - is this inconsistent/surprising ?
• When completing the command name, we could search PATH or ask hledger to search PATH for add-on command names
• We could attempt to provide flag completions for known add-ons (hledger-iadd, hledger-edit, etc). Eg by parsing their --help when generating the completions script (or dynamically when completing a flag after a known add-on command name).

Here are some more tasks that could be done:

• Make the adding of files to the release bindists more systematic and consistent. Eg, automatically add the same extra-source-files listed in hledger/package.yaml.
• [x] Review and update shell-completion/README
• [x] Add some mention of shell completions setup/usage in the manuals
• Mention how completions won't work in non-standard environments like emacs, or figure out if that can be made to work.
• Add a hledger commands command to make listing command names easier
• Make the release bindists extract into a directory ? update install instructions in ghrelnotes accordingly; test.
• Review/update/document the status of completions across distros/packaging systems, and help get those updated.
• Add completions for other popular shells, if contributors provide them.
• (Re)announce completions more prominently in future release notes, preferably in a feature release if possible.

[hseg]

• The hledger-ui and hledger-web symlinks seem unnecessary since the hledger script also registers itself for hledger-ui and hledger-web.

If you're referring to the instructions inside the completion file, note that bash-completion loads completions on-demand. In particular, the registration for hledger-{ui,web} will only occur once a completion for hledger has been requested. You can check this by typing hledger-ui<TAB> and then checking complete | grep hledger -- it will only list the default completer for unknown commands for hledger-ui. To load the subcommand completion without the symlinks, you need to do hledger <TAB> (note the space -- otherwise bash tries to complete the command and offers hledger, hledger-ui and hledger-web).

[simonmichael]

Ah, that makes sense. Not all packagers are doing this, eg the mac homebrew package currently installs just the hledger script. So I guess I'll add to the readme that packagers should also set up these symlinks.

I guess we could also include such symlinks in the unix and mac release bindists, as a hint ? I'm not sure how helpful that is.

[hseg]

Readme should be enough, imo. Though I have experimented with using relative symlinks, and they seem to work -- see my prod at the AUR maintainer https://aur.archlinux.org/pkgbase/hledger-bin#comment-991191

[simonmichael]

README updated. I have also added a section for tracking packaging status.