Smart completion packages

[plexus]

Currently Corgi uses Ivy, Counsel, Smex (although I just realize we are not actually binding any keys to use Smex), and Swiper. There have been a few questions about replacing these (#5, #14). Alternatives that have been proposed

by raxod502:

• Selectrum
• prescient

by oantolin:

• Icomplete Vertical
• orderless
• embark

by minad:

• vertico
• consul
• corfu

What we currently have: Ivy/Counsel/Swiper by abo-abo (monorepo).

I'm still from the time when it was Ido or Ivy so I find the current explosion of offers in this space a little overwhelming. We briefly switched to Selectrum and Prescient last year, but I quickly reverted the change. I remember being not satisfied with them at the time, but I don't remember what the issue was, see commit 776c4009048a9160cb0ec2f77cac4cacfdbf5888.

As a first step I think we need to make it easier for people to choose their own preferred packages here, that means splitting out the Ivy/Counsel/Smex/Swiper setup into a separate meta-package (corgi-completion? corgi-search?) so that people can load their own alternative.

We should also bind the commands these packages provide via signals. Some of them already are, e.g.

            :file/open counsel-find-file
            :file/open-recent counsel-recentf
            :jump/identifier counsel-imenu
            :buffer/incremental-search swiper

That way people can easily override this in their user-signals.el (see M-x corgi/open-user-signals-file).

Beyond that I would love it if someone could help do some meta-analysis, or point me at any good write-ups. Of all the packages listed above, which is an alternative to which? Which ones are bound to a specific underlying lib? What does each provide? etc. I've looked over some of these READMEs and feel none the wiser.

[jcmkk3]

I'm going to do my best to try to explain these packages, but they cover a lot of functionality so it is hard to do briefly. I personally use just a few of these, but I tend to keep things very minimal and might have more simplistic needs. I use Vertico, Corfu, Orderless, and Marginalia.

Minibuffer Completion UI

The main difference between Ivy (and Helm) and all of the other completion UIs listed here is that Ivy implements its own completion API instead of reusing the built-in completing-read.

The actual completion UIs represented in this list are Selectrum, Icomplete Vertical, and Vertico. The other items are supplements that can be used with any of these UIs since they all integrate with completing-read.

Of these, I would currently recommend Vertico. Vertico is a minimal take on a completion UI, written by a main contributor to Selectrum. He used that experience to reduce complexity and focus the package even further. Icomplete Vertical is mostly superceded by the new icomplete-vertical-mode in Emacs 28.

In-buffer Completion UI

Corfu is the option listed above. This package is a take on an in-buffer completion UI with a similar philosophy as Vertico. This would primarily replace Company Mode. Similar to Ivy, Company Mode implements its own completion API whereas corfu uses the built-in completion-in-region API, which is mostly powered by completion-at-point-functions implemented by various built-in and 3rd party packages.

You would want to add the corfu-popup package which provides a popup in the terminal. By default, corfu only provides popups in the graphical UI.

Extras

These other packages (and more) provide additional functionality by hooking into the underlying Emacs APIs that the above completion UIs use. These could actually be used independently with the built-in completion UIs.

• prescient: Provides some ranked sorting based on previous used commands. You can get a long way with just enabling the built-in savehist-mode. I don't find this package necessary.
• orderless: This is a completion style that can be used with any of the UIs from above. It provides fuzzy-ish completion kind of like fzf or other common fuzzy finders. This can be used in combination with the built-in completion styles. This is a must have for me.
• marginalia: This provides extra information to results in the minibuffer. I think this probably takes care of a lot of the benefit from smex, but I've never used smex myself.
• consult: This is a replacement for counsel that works with the above UIs. It provides additional completion commands like minibuffer powered line search, grep, etc. These will show the candidates for these completions in the minibuffer.
• cape: Not on your list, but this is basically the same idea as consult, but for in-buffer completions. For instance, there is an in-buffer file completion, one for ispell, etc.
• embark: This is probably the hardest one on this list to explain. I don't personally use it, but I know a lot of folks really find it integral to their workflows. This post by karthink is probably one of the better demonstrations of some things that it can do.

[plexus]

Thanks a lot @jcmkk3 , this is tremendously helpful!

[theophilusx]

@jcmkk3 Good write up! I use pretty much the same setup (vertico, corfu, orderlees, marginalia and consult) plus I use embark. I've only been using embark for a short time, but it is quite interesting because of how it flips completion on its head. WIht normal completion approaches, you start by selecting an action (like find file) and then add a target. Embark can still work that way, but it also allows you to start with the target and then it can tell you what actions you can do on that target. I'm not yet convinced about how well the practice fits with the theory, but am enjoying experimenting with it.

What I really like about these sets of tools is the fact they are built on the standard Emacs completion API and not one specific to the framework (like ivy and helm). I find this makes them smaller, faster and more stable, plus much easier to integrate with existing packages.

I think it would be a good first step to break out the completion related packages and key bindings into their own package and then add an alternative package based on vertico, corfu, consult and maybe embark. This would have two benefits. It would give people a simple choice and it would demonstrate how anyone could create their own and just drop it in to replace the default.

Even if it is decided to just break completions off into its own meta package, that would be sufficient. Because corgi is using straight, anyone can just put up their own completions package which also includes corkey key/signal definitions and all someone would need to do to use it is replace the use-package call to corgi-completion with a call to alternative-completion with a :straight recipe and restart.

[theophilusx]

Key bindings and modules question.

If we want to make things 'modular' we run into a problem with key bindings. The current configuration has all key bindings and all signals in two files, loaded with use-package corgi-bindings. This is very convenient as you can see all your bindings in one place and you can manage all the signals in one place.

However, if we also want something (such as completions) to be a module, we run into problems. If someone decides not to load the module, they now have key bindings which don't work. Likewise, if we wanted to have an alternative completions module, we need to be able to load new key bindings suitable for the alternative module.

If we are going to break completions off into a separate module, should we be including in that a module keys and signals files which are specific to that module? This would allow users to easily choose the module they want and know key bindings stay consistent. The downside is not all key bindings and signals will be together in their respective corgi files.

[plexus]

All our baseline bindings go into corgi-keys/corgi-bindings, including optional ones. Corkey ignores bindings if the command doesn't exist for this reason. We aim to avoid having "alternatives" or "choices/options" within Corgi itself, we provide a single opinionated set of packages as a baseline, and so all of those bindings can go into corgi-keys/corgi-bindings.

A third party package (e.g. one that is provided as an alternative to a corgi meta-package) can include their own keys/bindings files, which you can layer over the base config.

(corkey/watch-and-load
  (corgi-keys third-party-keys user-keys)
  (corgi-bindings third-party-bindings user-bindings))

We already make use of this to include bindings for packages that are not part of Corgi, e.g. we have some geiser bindings in scheme buffers, these will only become active if geiser is installed.

[theophilusx]

Thanks. Exactly the direction I was hoping for and in fact, very close to what I expected. I looked through the corkey code and suspected your solution for third party packages was the direction to take.

Once you have finished evaluating my tweaks on completion and any further refinements, I will create a 3rd party completions package based on vertico, corfu, orderlees, marinalia and maybe embark. I'll make it available on my github account initially so that others can give it a spin. We can then decide what (if anything) we want to do.