[Question] How can I use `consult-line` together with `visual-line-mode`?

[apc]

Is there a way to have consult-line play nice with visual-line-mode?

This is what I get using consult-line with global-visual-line turned on:

Compare to the result of using swiper, where the previews in the minibuffer don't show the beginning of the line, but rather the context where the match occurs:

Perhaps there's an alternative to counsel-line that works much like swiper does?

[minad]

This is actually a feature request. Consult does not handle visual lines as of now. There are two possibilities.

1. We could handle this on the side of Consult by splitting logical lines into visual lines. I am not sure how this works internally - if it is easily possible to move across all visual lines of a buffer.

• consult-line (split logical lines in visual lines)
• consult-mark (show only visual line around the mark)
• consult-global-mark (show only visual line around the mark)
• consult-outline (show only visual line of the heading which appears at the beginning of the logical line)

2. Another way to handle the issue would be to ensure on the side of the completion UI that the current match position is visible. Selectrum for example already does something like this for multi-line candidate strings, but not (yet?) for long single lines.

Disadvantage of approach 1 is that the candidates are less meaningful, we are not showing something "logical" anymore. The same issue applies to consult-yank-pop for example which also uses very long candidate strings. In that case one could also split up the kill ring candidates for a better display in substrings of the actual string. Disadvantage of the second approach is that every completion UI must do such special casing for long candidates as Selectrum does for multi-line candidates.

[minad]

I just checked swiper and it uses either a flag or a predicate to check if visual line movement should be used. As I suspected movement over visual lines is slow, so it is not actually easily possible to use option 1 unless one accepts that the command will be unacceptably slow for large buffers. One could use a predicate like swiper but this is not a particularly compelling solution.

(defvar swiper-use-visual-line-p
  (lambda (n-lines)
    (and visual-line-mode
         ;; super-slow otherwise
         (< (buffer-size) 20000)
         (< n-lines 400)))
  "A predicate that decides whether `line-move' or `forward-line' is used.
Note that `line-move' can be very slow.")

[minad]

I decided to close this as wontfix. consult-line is a line-based command operating on logical lines, similar to grep. There is also no such thing as a visual-grep. If there is interest in this feel free to experiment, maybe the solutions I proposed are indeed feasible. And well, there is always swiper you can use as an alternative. So there is no need to replicate every feature exactly here :)

[apc]

Makes sense! I suspect it’s a bit of a fringe case anyway. I just wanted to check in case it was already possible to do this and I was missing something.

Thanks for your work on this.

[minad]

I suspect it’s a bit of a fringe case anyway.

I am unsure how widely visual-line-mode is used - I rather use auto-fill-mode. Emacs itself has problems with files with long lines due to the data structures it uses internally for buffers. Until now the feature has not been requested, so it may indeed be fringe. But Consult itself is a fringe library in comparison with Counsel. The library is not supposed to copy over all the features of Counsel and they differ in many details. While Consult is still a kitchen sink library I try to be a little more restrictive - avoiding duplicating existing functionality, avoiding features which are technically problematic (like this one) or which are special purpose. For example there are the libraries consult-notmuch/recoll which are better maintained separately.

[hmelman]

FWIW I believe visual-line-mode is widely used as it mirrors the behaviors for many other apps. Anyone who cuts and pastes into other apps would be interested in it.

[minad]

FWIW I believe visual-line-mode is widely used as it mirrors the behaviors for many other apps.

This may be true. But all in all Emacs is still fundamentally a line based application, and consult-line is a line based command, like grep. Do you see a reasonable way to support the feature here?

[apc]

I'm too much of a rookie here to have views on how best to do this. But out of curiosity, why is the solution analog to the one implemented in swiper not compelling? (This is a genuine question, I'm wondering whether it's worth even trying to implement something like that for myself...)

[hmelman]

I haven't looked at how swiper handles things other than what's described here. Perhaps there is a halfway compromise. I think I'd be fine if the line in the candidate (in the minibuffer for selectrum) was the full actual line, but if preview is enabled, I'd like just the visual line of the current match to be highlighted. That may not be practical if there is more than one match for a string in a line, but maybe this suggestion sparks an idea.

FWIW I now enable visual-line-mode in text-mode buffers by default. I tend not to have problems with performance because it's rare I have large enough buffers to get there. So I'd be fine with a solution that worked up to some size limit, particularly if that was detected automatically.

[minad]

but if preview is enabled, I'd like just the visual line of the current match to be highlighted.

Okay, it is not a problem to highlight the part of the wrapped line which contains the match. But note that consult-line won't even work well with multiple matches on a line. It will jump to the first match then. This is another reason not to implement any support for visual line mode, since then it becomes more likely to have multiple matches. Again, consult-line is a line based command, like grep. Redefining it to work on something else than logical lines is not something that will fall out for free.

So I'd be fine with a solution that worked up to some size limit, particularly if that was detected automatically.

I don't like such a solution and wouldn't want to see it here. It is okay for fontification which is non critical eye-candy, but introducing thresholds for degraded functionality is something I dislike.

[oantolin]

@hmelman's suggested compromise of searching logical lines but having the preview highlight the visual line containing the start of the match sounds pretty reasonable to me. I don't think there is any way to support searching visual lines without taking a big speed hit, and consult-line already takes a while to start up in, for example, the *Packages* buffer. So I'd say marking this as wontfix is quite reasonable.

Although I find consult-line pretty usable with visual-line-mode as is. In most of my files I use auto-fill-mode, but when writing collaboratively I always let my coauthors pick the conventions (because Emacs is more adaptable that whatever they use) and some people like this "one line per paragraph" mode. In those cases I find that Consult's preview is useful enough: yes, it does highlight the whole paragraph, but it also puts point at the start of the match and that's usually fairly quick to spot (@hmelman's idea would make the previews even better in this case). I also like using for those cases a completion UI that displays multi-line candidates in full, like the completions buffer, icomplete(-vertical), or embark collect. I find the Consult previews and non-shortening completion UIs to be an acceptable workaround.

[minad]

@apc

But out of curiosity, why is the solution analog to the one implemented in swiper not compelling? (This is a genuine question, I'm wondering whether it's worth even trying to implement something like that for myself...)

Mainly because it is slow and will lead to a command which works only well up to a certain small buffer size limit. consult-line already becomes slow for large files. A consult-visual-line command would work only for smaller buffers. Then there is the multiple match issue in consult-line, which I didn't even realize before. One could modify consult-line such that it works with multiple matches, but it is yet another missing piece here. The more I think about it, a consult-visual-line command is really something distinct, which will differ significantly from the simple line-based consult-line command. It is not without reason that Swiper/Ivy/Counsel are large packages.

Note that there is also isearch, which you can also use. Then if you like the design of Swiper, by all means, use Swiper. Swiper has many variants (using grep for larger files, turning off visual mode for larger files and so on). There are these dwim commands which automatically switch between the variants. I am not fond of such a behavior.

Overall I don't feel the need to have the perfect search command in Consult. It works well for the use cases I consider most important. For quick jumps to logical lines, for listing all matching logical lines. It is like a grep inside Emacs enhanced with preview. But that's it. I understand that this is not exactly a satisfying answer for someone with different requirements.

[minad]

@oantolin I agree with your points mostly, highlighting the current visual line would be a fair compromise. However there is the issue with multiple matches.

[oantolin]

I say ignore multiple matches, highlight the visual line containing what is currently computed to be "the start of the match" (where preview puts point).

[oantolin]

I think ignoring multiple matches is what swiper does anyway, there is a separate swiper-isearch command that is not line-oriented and does list multiple matches in the same line as distinct candidates.

[apc]

Overall I don't feel the need to have the perfect search command in Consult. It works well for the use cases I consider most important. For quick jumps to logical lines, for listing all matching logical lines. It is like a grep inside Emacs enhanced with preview. But that's it. I understand that this is not exactly a satisfying answer for someone with different requirements.

Fair enough! Thanks for taking the time to explain.

[minad]

@oantolin

Okay, sure. If your search pattern is unique enough it will work well. I am happy to accept a patch which changes the jump preview highlighting to highlight the visual line instead of the current line. I agree that this is better than what we have now. swiper-isearch is a totally different command I think, I think you enter the search expression first? But I have never used it.

[minad]

Probably the patch is trivial, just use beginning-of-visual-line instead of beginning-of-line? I will give it a quick check.

[oantolin]

Probably the patch is trivial, just use beginning-of-visual-line instead of beginning-of-line? I will give it a quick check.

I think that's right. At least it's what I would have tried.

[minad]

this works:

diff --git a/consult.el b/consult.el
index 2f4d847..4392cc1 100644
--- a/consult.el
+++ b/consult.el
@@ -1002,2 +1002,7 @@ FACE is the cursor face."
-              (list (consult--overlay (line-beginning-position)
-                                      (1+ (line-end-position))
+              (list (consult--overlay (save-excursion
+                                        (beginning-of-visual-line)
+                                        (point))
+                                      (save-excursion
+                                        (end-of-visual-line)
+                                        (let ((end (line-end-position)))
+                                          (if (= (point) end) (1+ end) (point))))

But now I am running into https://github.com/oantolin/orderless/issues/39.

[minad]

@oantolin Are you seriously using this on tex files with paragraphs on a single line? Filtering is super slow. (EDIT: Ah, I forgot, you are not using a continuously updating UI. Makes sense.)

[oantolin]

I also mostly use it with literal regexps, for which matching is quite fast. I have regexp as the only default matching style and all the expensive styles require explicit syntax in my setup.

[hmelman]

I agree it's fine for a command to have a "sweet spot" and function within limitations. It's best of course if it documents them and offers alternatives (within reason).

Thinking about the multiple matches on a line point. ripgrep has a switch --vimgrep that outputs a separate line for every match. consult-ripgrep configured to use that could work well. occur mode has command occur-next on M-n that goes to the next match, not just the next line. If a user really needs that, that's probably their best bet.

I'm not a huge user of consult-line. I mostly use isearch, symbol-overlay, and the ripgrep package rg. I do use consult-line sometimes, but it's usually to visually inspect what lines match something (and then often use embark collect) and consult-focus-line or occur would work just as well for me. I do keep playing it with to see if I find something particularly compelling about it. I haven't had the need to use it with particularly advanced match patterns.

[minad]

@oantolin Okay I have only flex explicitly since this is just over the top. The rest is mostly fine if lines are of acceptable length :smile:

[oantolin]

I don't use explicit syntax for, say, initialism because it is slow, but mostly because if I want something to match as initials I don't also want the matches as a literal. Avoiding slow matches is just a nice side effect.

[minad]

@hmelman

Thinking about the multiple matches on a line point. ripgrep has a switch --vimgrep that outputs a separate line for every match. consult-ripgrep configured to use that could work well. occur mode has command occur-next on M-n that goes to the next match, not just the next line. If a user really needs that, that's probably their best bet.

Yes, this could work well. It is configurable via the consult-ripgrep-command. I am unsure if further adjustments are needed.

I'm not a huge user of consult-line. I mostly use isearch, symbol-overlay, and the ripgrep package rg. I do use consult-line sometimes, but it's usually to visually inspect what lines match something (and then often use embark collect) and consult-focus-line or occur would work just as well for me. I do keep playing it with to see if I find something particularly compelling about it. I haven't had the need to use it with particularly advanced match patterns.

I mostly use consult-line to get a list of matching lines and maybe export the matches. I think the strong point is actually that you can use flexible filter strings if you use orderless instead of only literals or regexps. This is what I like the most about it. For quick jumps I often use isearch. I almost never use symbol-overlay. Emacs 28 has support for ripgrep I believe, so one does not need the rg package in order to get a buffer view. Probably rg/deadgrep has other advantages? I like consult-ripgrep, but this may be the ikea effect.

[hmelman]

I mostly use consult-line to get a list of matching lines and maybe export the matches. I think the strong point is actually that you can use flexible filter strings if you use orderless instead of only literals or regexps. This is what I like the most about it.

I agree that's nice. I still have to give orderless a try. I'm close to doing so.

For quick jumps I often use isearch. I almost never use symbol-overlay.

I've come to really love symbol-overlay. I have a transient for it on s-. and really like highlighting a symbol and having the instances jump out in the buffer. That < usually takes me to the definition is very convenient.

Emacs 28 has support for ripgrep I believe, so one does not need the rg package in order to get a buffer view.

Yeah? Is it easy? Configuring the various grep and find-grep options to handle switches is I think one of the worst parts of emacs config.

Probably rg/deadgrep has other advantages? I like consult-ripgrep, but this may be the ikea effect.

I like that rg's output is grouped by file. Also it binds single letter keys to rerun the search changing either the search string or the type of search (literal or regexp). Lastly I have a couple of preconfigured searches (rg-define-search in rg.el terms) to search emacs source, my config and a couple of other projects quickly. This really helped for searching emacs sources as they are gzip'ed and getting those switches correct vs using .ripgreprc was surprisingly non-trivial (this became an example in rg.el docs).

[minad]

I like that rg's output is grouped by file.

Hmm, maybe consult-ripgrep should add that via its grouping functionality. The other points you mention seem less significant and can be achieved using consult-ripgrep, but I am under the influence of the ikea effect, so don't take me serious.

[hmelman]

but I am under the influence of the ikea effect, so don't take me serious.

I wouldn't expect you to use anything else 😉

Glad I inspired a feature :)

[minad]

I wouldn't expect you to use anything else

Well, and I also use it in conjunction with Embark, otherwise it wouldn't be worth half as much.

Glad I inspired a feature :)

Yes, but I am not yet 100% sure if I should keep it since we have the redundant filenames/titles then and I cannot rely on the completion UI to support x-group-function. I think I have to revise a few things.

[oantolin]

I cannot rely on the completion UI to support x-group-function.

If that's a (well-deserved) dig at Embark collect, I will get around to implementing it... 😛 Now we have Vertico, Selectrum and Icomplete-vertical with support for x-group-function, and a function on a wiki for the completions buffer. That seems like adequate selection.

We would have to change the exporter if the candidates stop carrying the file name (or was that in a text property too?, maybe the exporter wouldn't need to change, but I can't remember).

[minad]

@oantolin I have better ideas https://github.com/minad/consult/issues/283

[jdtsmith]

Looks like this discussion recurred in part a year and a half later (#748). For anyone interested, I threw together a very small package which left-truncates to bring long/visual-line matches from consult-line, consult-grep, and friends into view. I do not notice a slowdown for files with (occasional) lines up to 1500 chars or so (with orderless matching).