How to control completion styles used by consult-line?

[tarsius]

I was trying to get consult-line to match fewer lines despite (setq completion-styles '(basic partial-completion substring flex initials)). For example input magit should not match:

;; Homepage: https://github.com/minad/consult
     -  --            --

I hoped that I could do that with:

(setf (alist-get 'consult-location completion-category-defaults)
      '((styles basic)))

But it appears that is actively being discarded in consult--split-setup and I did not spot another mechanism for configuring this.

[minad]

There are many concepts at play here, and they are interacting in complicated ways.

The consult--split-setup function is only used for the async functionality. Even then the split style defers to the usual completion styles, but ignores the category overrides as you have noticed. Its only purpose is to extract the input string which is passed to the asynchronous process. The remaining string is passed to the completion styles and handled as usual. Example: #string-passed-to-async-process#string-passed-to-completion-styles. So to make it clear - async/split-style has nothing to do with this.

For consult-line and all the other synchronous completion functions, the usual completion styles is picked up. This is just the standard Emacs mechanism. If you want to enforce a local completion style you can write a wrapper.

(defun my-consult-line ()
   (let ((completion-styles ...))
       (consult-line)))

Your attempt to set the completion-category-defaults/overrides failed, since Emacs seems to append the override/defaults to the main completion-styles. This means that if your overrides/default style does not have a match, it defers to the main completion-styles. This is certainly a questionable design decision. Therefore if you really want to force this, you have to do it in a wrapper. See the Emacs code:

(defun completion--category-override (category tag)
  (or (assq tag (cdr (assq category completion-category-overrides)))
      (assq tag (cdr (assq category completion-category-defaults)))))

(defun completion--styles (metadata)
  (let* ((cat (completion-metadata-get metadata 'category))
         (over (completion--category-override cat 'styles)))
    (if over
        (delete-dups (append (cdr over) (copy-sequence completion-styles)))
       completion-styles)))

You tried using the basic completion style which only does prefix matching - unfortunately this style does never work with certain consult commands for technical reasons. I am sorry for that, but there is no other way since I have to use invisible disambiguation prefix strings. I could provide a modified basic style which fixes this problem. But I don't really see a reason to use basic and this has not been requested before. Most people seem to be happy with Orderless or Prescient.

My recommendation is to use orderless and for completion in region, I recommend combining substring+orderless since this allows TAB completion. I do not recommend Prescient since it is not actually a completion style but a separate filter mechanism which hooks directly into the completion system. This will hopefully be fixed at some point (https://github.com/raxod502/prescient.el/issues/89). But since neither @clemera nor I use the prescient filtering, I don't see it being fixed soon.

I furthermore use an orderless dispatcher character in order to select flex style on a case by case basis. Flex is simply too slow, since it does not scale well for long candidate strings, even worse if you have 10ks of candidates. With the given configuration I can enter a filtering string where each of the parts is interpreted specially: regexp flex~ literal= !without-literal

  (defun my--orderless-dispatch (pattern _index _total)
    (cond
     ((string= "!" pattern) `(orderless-literal . "")) ;; ignore empty !
     ((string-prefix-p "!" pattern) `(orderless-without-literal . ,(substring pattern 1)))
     ((string-suffix-p "=" pattern) `(orderless-literal . ,(substring pattern 0 -1)))
     ((string-suffix-p "~" pattern) `(orderless-flex . ,(substring pattern 0 -1)))))
  (setq completion-styles '(orderless)
                completion-category-defaults nil
                completion-category-overrides nil
                orderless-matching-styles '(orderless-regexp orderless-initialism)
                orderless-style-dispatchers '(my--orderless-dispatch)

I hope this answers your question.

[minad]

Without bothering about all the details - there are these possible working configurations:

1. Do not use basic

(setq completion-styles '(partial-completion substring flex initials))

2. Use a Consult-compatible basic, which does not exist yet

(setq completion-styles '(consult-basic partial-completion substring flex initials))

If you really do care about basic, it shouldn't be too difficult to make a Consult-compatible variant. It is certainly ugly and I would not support it officially, but it could be documented in the Consult wiki.

3. Just use Orderless, which is better ;)

(setq completion-styles '(orderless))
;; plus some tweaks as mentioned above

[minad]

Regarding option 2, here is the Consult-compatible basic style. It does less than the built-in basic style which contains some magic, I don't understand. Therefore I would strongly advise using it as a general replacement for basic. But if you use consult-basic only for Consult commands it should not make problems.

(defun consult-basic-all-completions (string table pred _point)
  ;; Ignore the disambiguation prefix
  (let ((completion-regexp-list (list (concat "\\`[\x100000-\x10FFFD]*" (regexp-quote string)))))
    (all-completions "" table pred)))

(defun consult-basic-try-completion (string table pred _point)
  ;; Ignore the disambiguation prefix
  (let ((completion-regexp-list (list (concat "\\`[\x100000-\x10FFFD]*" (regexp-quote string)))))
    (try-completion "" table pred)))

(add-to-list 'completion-styles-alist
             '(consult-basic
               consult-basic-try-completion consult-basic-all-completions
               "Consult-compatible basic completion."))

(setf (alist-get 'consult-location completion-category-defaults) '((styles consult-basic)))
(setf (alist-get 'consult-isearch completion-category-defaults) '((styles consult-basic)))
(setf (alist-get 'consult-multi completion-category-defaults) '((styles consult-basic)))

[tarsius]

Your attempt to set the completion-category-defaults/overrides failed, since Emacs seems to append the override/defaults to the main completion-styles. This means that if your overrides/default style does not have a match, it defers to the main completion-styles. This is certainly a questionable design decision. Therefore if you really want to force this, you have to do it in a wrapper.

That's indeed unfortunate but now that I know what is going I'll try to use it as is for a while. People who are coming from built-in completion are used to this, but I was coming from ivy.

Therefore if you really want to force this, you have to do it in a wrapper. See the Emacs code:

Or I might just override these functions. We'll see. Thanks for the hint!

You tried using the basic completion style which only does prefix matching - unfortunately this style does never work with certain consult commands for technical reasons. I am sorry for that, but there is no other way since I have to use invisible disambiguation prefix strings. I could provide a modified basic style which fixes this problem. But I don't really see a reason to use basic and this has not been requested before. Most people seem to be happy with Orderless or Prescient.

I think the problem isn't so much that the basic style is broken but that users cannot know that. (Or maybe they can, but if so, then I missed the memo.) I was actually using just the basic style for debugging purposes, which didn't work so well because it caused things to become more unexpected, instead of... you know: basic. :grin:

I don't know why you need to "use invisible disambiguation prefix strings". That sounds like it will cause a lot of issues down the line. If you could at all remove the need for that, then that would probably be a good idea.

If you don't mind, could you please briefly explain why you need to do this?

My recommendation is to use orderless and for completion in region, I recommend combining substring+orderless since this allows TAB completion.

I've just started doing that. I was not sure whether I could combine orderless with other styles and intended to ask about that. Any caveat I should be aware of?

[oantolin]

I've just started doing that. I was not sure whether I could combine orderless with other styles and intended to ask about that. Any caveat I should be aware of?

I don't think there are any caveats. As long as you understand that completion-styles are tried in turn and the first one to produce any matches is the only one used (until the next keystroke when the process starts again) you shouldn't be surprised.

[minad]

I don't know why you need to "use invisible disambiguation prefix strings". That sounds like it will cause a lot of issues down the line. If you could at all remove the need for that, then that would probably be a good idea.

Yes, this certainly leads to issues. I am well aware of that. But there is no better way since for consult-line we have to distinguish different lines with the same text. The problem is that string equality does not take string properties into account and that completing-read strips text properties. So this will not be changed any time soon.

Hopefully completing-read can be improved in upstream Emacs, but it will be years before we could profit from that. There are also a few other small related proposals I have to make. But the string disambiguation I am doing in Consult is certainly the most severe hack I am doing and I think it is also the only one. Besides that the completing-read API is good enough.

Note that if you use a more powerful completion style, there are no problems. And you really should use a more powerful completion style.

I've just started doing that. I was not sure whether I could combine orderless with other styles and intended to ask about that. Any caveat I should be aware of?

You can, for example I don't think it generally works well to combine substring+orderless, since substring is basically a subset of orderless. However in general I think the philosophy of Orderless is different, with Emacs completion styles each style is tried in order until one of the styles matches. With Orderless one uses just this one completion style and controls the matching style per token.

I am only using Orderless and I am pretty happy with that. One caveat is that TAB completion is not provided, but this is more an issue of Selectrum providing the corresponding command. But I usually don't miss this except for completion-in-region, where I am using a cycle threshold and I find it natural to press TAB to complete when I also have to press TAB to initiate the completion-in-region. See the long discussion in https://github.com/raxod502/selectrum/issues/272.

[tarsius]

Thanks for all the useful hints!

[minad]

See also https://github.com/raxod502/selectrum/pull/479#issuecomment-789675524. Maybe we will propose some improvements.