Support for generating specs from man pages?

[heinwol]

Request

Sorry if I'm getting it all wrong, I didn't spend much time researching the topic, but it seems there's no way to generate completions (specs) from man pages. Maybe it's already there, then correct me, please.

I love this feature in fish. That's what make me feel comfortable using this shell in the first place. On the other hand, I would love to use nushell, but it seem to rely heavily on carapace, that's why I'm here~

Proposed solution

Like in fish, we can have a special command to generate completions from man pages installed on the system. Feels like it's not way to hard. I could've engaged in this myself maybe, but I don't plan on learning golang, unfortunately.

Anything else?

I feel like this feature could do a lot of good to both shells that don't have their own full-featured completion system (like nushell and maybe others) and carapace. The latter -- correct me if I'm wrong -- is implied by the fact that carapace doesn't provide many completions out of the box, so if one wants to have completions to most commands they have to fallback to native shell completion.

Also, having completions from parsing man pages can heavily reduce amount of effort people are putting into writing carapace specs by hand.

[rsteube]

Yeah, parsing manpages comes up quite often in discussions regarding completions. I had a look at it in the past and it is possible to a degree, but the results are quite bad.

Best way so far has been to prepare them by hand for parsing with carapace-parse. Or leveraging existing frameworks.

[heinwol]

Hmm, maybe it could be done a bit differently. If we already have completions generated by e.g. fish, we might be able to use them as a base. The source will be more structured at least.

The first thing I need to do in this regard is to look at format and structure of fish completions files (never done that btw). If it's convenient enough we could just turn them into carapace yaml. This might not be quite convenient for the end user.

The other thing we could do is to look into how fish_update_completions work and maybe reuse this code in a separate tool to produce carapace specs. The tool then could be written in python -- at least I know this language -- as fish is written in python.

[heinwol]

I thought it was an easy matter (i.e. if fish can do this automatically we can do it too). But after looking at this list of fish completions I understand I was too far from truth. Seems like most completions aren't generated from man pages; alas, most of them are written by hand and it often includes some nontrivial logic which cannot be easily transliterated.

So, it's hard for me to imagine how we can make use of these results... Maybe fish can be run as a background process for the completer to communicate with. But this 1) requires the user to have fish installed and set up in their environment and 2) it mostly kills the purpose of carapace, because, as I understand, carapace aims to become a replacement to traditional completions and not an addition to them. It might be a good idea to bring this topic up on the nushell side -- at least using fish as a completion fallback seems less out of place there.

On the other hand, fish seems to generate some basic completions from man pages quite well. Do you think it's not worth it?

[rsteube]

Fish has complete --do-complete so bridging is already possible: https://github.com/rsteube/carapace-bridge.

There are manpage completions additionally to the handwritten ones which look like this:

# git-checkout
# Autogenerated from man page /usr/share/man/man1/git-checkout.1.gz
complete -c git-checkout -s q -l quiet -d 'Quiet, suppress feedback messages'
complete -c git-checkout -l progress -l no-progress -d 'Progress status is reported on the standard error stream by default when it i…'
complete -c git-checkout -s f -l force -d 'When switching branches, proceed even if the index or the working tree differ…'
complete -c git-checkout -l ours -l theirs -d 'When checking out paths from the index, check out stage #2 (ours) or #3 (thei…'
complete -c git-checkout -s b -d 'Create a new branch named <new-branch>, start it at <start-point>, and check …'
complete -c git-checkout -s B -d 'Creates the branch <new-branch>, start it at <start-point>; if it already exi…'
complete -c git-checkout -s t -l track -d 'When creating a new branch, set up "upstream" configuration'
complete -c git-checkout -l no-track -d 'Do not set up "upstream" configuration, even if the branch'
complete -c git-checkout -l guess -l no-guess -d 'If <branch> is not found but there does exist a tracking branch in exactly on…'
complete -c git-checkout -s l -d 'Create the new branch\'s reflog; see git-branch(1) for details'
complete -c git-checkout -s d -l detach -d 'Rather than checking out a branch to work on it, check out a commit for inspe…'
complete -c git-checkout -l orphan -d 'Create a new orphan branch, named <new-branch>, started from <start-point> an…'
complete -c git-checkout -l ignore-skip-worktree-bits -d 'In sparse checkout mode, git checkout -- <paths> would update only entries ma…'
complete -c git-checkout -s m -l merge -d 'When switching branches, if you have local modifications to one or more files…'
complete -c git-checkout -l conflict -d 'The same as --merge option above, but changes the way the conflicting hunks a…'
complete -c git-checkout -s p -l patch -d 'Interactively select hunks in the difference between the <tree-ish> (or the i…'
complete -c git-checkout -l ignore-other-worktrees -d 'git checkout refuses when the wanted ref is already checked out by another wo…'
complete -c git-checkout -l overwrite-ignore -l no-overwrite-ignore -d 'Silently overwrite ignored files when switching branches'
complete -c git-checkout -l recurse-submodules -l no-recurse-submodules -d 'Using --recurse-submodules will update the content of all active submodules a…'
complete -c git-checkout -l overlay -l no-overlay -d 'In the default overlay mode, git checkout never removes files from the index …'
complete -c git-checkout -l pathspec-from-file -d 'Pathspec is passed in <file> instead of commandline args'
complete -c git-checkout -l pathspec-file-nul -d 'Only meaningful with --pathspec-from-file'
complete -c git-checkout -l literal-pathspecs

Which is fine compared to having nothing, but there are so many issues with this. like:

• Where to cut the description? This often spans over multiple lines
• Does the flag take an argument? Is it optional (--flag=arg)?
• Is it repeatable?

Manpages are so inconsistent that the results contain errors. In the end a lot has to be done by hand anyway.

[rsteube]

related https://github.com/nushell/nushell/issues/6075

[heinwol]

Thanks a lot, @rsteube, gonna check it out! Peace~