[Question] How to achieve behaviour similar to `swiper-isearch`?

[rafauke]

Thank you for this package, I really enjoy using it as an alternative swiper :pray:

I want to ask how can I achieve a similar result to swiper-isearch, i.e. searching for element matching phrase, then cycling with .e.g C-s and C-r:

I understand that consult-line should be used for that, but I don't know how can jump to separate occurrences that happen on the same line:

I would be grateful for any hints :)

[xavierr]

I am really looking for this same feature. I use forward-isearch (more precisely its swiper version) all the time to move in a buffer and then it is very helpful (if not essential) to be able to jump from one ocurence to the other also when they are on the same line.

Thanks!

[minad]

A swiper-isearch variant does not exist as part of consult. I consider it an awkward hybrid between swiper and isearch. consult-line does not aim to replace the good old Emacs isearch, which is powerful. consult-line is a simple line-based filter command similar to grep acting on a single file. Therefore the functionality to move between matches is not provided. I use both consult-line and isearch and don't see the need for a hybrid.

Nevertheless, at some point I experimented with a swiper-isearch like command. If there is interest, I can dig it up. But I don't have plans to include it here and the command was also not in a user consumable form. It needed more polishing and work.

Note that while consult generally tries to fill the niche of counsel (if you are not using ivy), it does not aim to replicate ivy/counsel precisely. There are certainly some commands missing in consult. And the same applies to counsel.

[minad]

For reference here is a toy implementation which behaves similarly to swiper-isearch:

;; -*- lexical-binding: t -*-

(defun consult-line-delayed-async (async)
  (lambda (action)
    (if (and (stringp action) (not (equal action "")))
        (let ((lines))
          (with-current-buffer (window-buffer (minibuffer-selected-window))
            (save-excursion
              (goto-char (point-min))
              (while (search-forward-regexp action nil t 1)
                (push (consult--location-candidate
                       (consult--buffer-substring (line-beginning-position)
                                                  (line-end-position)
                                                  'fontify)
                       (point-marker) (line-number-at-pos))
                      lines))))
          (funcall async 'flush)
          (funcall async (nreverse lines)))
      (funcall async action))))

(defun consult-line-delayed ()
  (interactive)
  (consult--read
   (thread-first (consult--async-sink)
     (consult--async-refresh-immediate)
     (consult-line-delayed-async)
     (consult--async-split)
     (consult--async-throttle 0.01 0.01))
   :sort nil
   :lookup #'consult--lookup-location
   :state (consult--jump-state)))

[xavierr]

It is very kind of you to spend time on that ! I am going to try it. It shows also that you have developed a setup that can very quickly accommodate for new features!

For me, it is much easier to relate to one command for doing all incremental search which I use all the time for navigation, and line search with minibuffer preview makes it also significantly more comfortable.

[minad]

It shows also that you have developed a setup that can very quickly accommodate for new features!

Yes, the internal consult APIs can be reused to implement this command. In the above snippet the C-s and C-r bindings are missing, but you can add a :keymap to the consult--read call, which binds them to commands which move to the next or previous candidate.

For me, it is much easier to relate to one command for doing all incremental search which I use all the time for navigation, and line search with minibuffer preview makes it also significantly more comfortable.

This is understandable. My perspective is a bit different. I am not using consult-line for navigation - for this I use standard Isearch. I use consult-line to get an overview, which one can also export via embark-collect and keep around.

I like to have mostly orthogonal commands and features. Given that Isearch is versatile and powerful, I decided to not try to replicate it inside of Consult. I think this makes it overall easier to understand the whole system (Emacs + the set of packages I've chosen to install). Furthermore it leads to a coherent more minimal package. In contrast, Swiper really aims to replace Isearch and even supports multiple different search commands swiper (like consult-line), swiper-isearch (like the command from the snippet consult-line-delayed), counsel-swiper-or-grep, ... The reason for this is also that Swiper evolved over a longer time and the different variants got added one after the other.

[oantolin]

I also think isearch-like behavior is fundamentally at odds with the idea of consult commands, which are about leveraging the Emacs completion system. You can ask a completion style things like "does this user input match this string?" or "among the strings in this list, which match the user's input?". Completion styles aren't really meant to answer questions like "which locations inside this long string or buffer does this user input match?".

So consult-line is perfectly in tune with completion, it asks "which lines match this user input?". On the hand something like isearch which is more about "at which locations in the buffer does this user input match", and more precisely something like "what's the leftmost maximal non-overlapping set of occurrences", which completion isn't really suited for.

Of course, something could be done:

• A hypothetical consult isearch-alike could abandon the completion system, and instead implement some sort of search directly. This is what @minad's function above does. I don't mean to sound too negative by calling this an abandonment of the completion system, all I mean is that what is considered a match and what isn't, is determined by search-forward-regexp rather than by the user's configured completion styles.

• Alternatively you could do something like uses as candidates for completion all the final segments of lines in the buffer and other than that behave like consult-line. If you only use completion styles that match anchored to the beginning of strings, this probably behaves the way you'd expect; and if you have completion styles that can match in the middle this will lead to a bunch of repetitions. Of course building the set of candidates would probaby be pretty slow with this approach (I believe Emacs copies substrings as soon as the substring is requested).

But I'd say that neither of those strategies comfortably fits in to the completion paradigm which most consult commands use. Now, the async consult commands with their splitting of the input into two parts only one of which is handed to the compeltion system, already leave the pure completion paradigm, so this doesn't definitively kill the idea, but I think it does point in the direction of this not being a great fit for Consult.

On a more personal note, like @minad I use both isearch and consult-line for different purposes. I use consult-line to get a quick overview of a buffer, but also for "coarse-grained navigation" where i just want to get to a particular line, or even paragraph or function, and don't particularly care where on the line I land. I use isearch when I want to put point at a specific character, specially if it's not on screen at the time, because for that I tend to use avy.

[minad]

@oantolin

I also think isearch-like behavior is fundamentally at odds with the idea of consult commands, which are about leveraging the Emacs completion system.

Yeah, therefore I called swiper-isearch an awkward hybrid ;) However we can abuse Consult async tables or dynamic completion tables to achieve something like this. I agree that consult-line fits completion and Consult naturally - if one looks at the code it is also pretty trivial.

A hypothetical consult isearch-alike could abandon the completion system, and instead implement some sort of search directly.

True. However the completion UI is reused/abused. So there is some value in going through the completion API at least for the users of incrementally updating UIs. The same applies to consult-ripgrep etc. As you mentioned completion also comes into play thanks to the two stage filtering of async commands. But arguably this feature got less important with the recent addition of the orderless-style regexp transformation.

so this doesn't definitively kill the idea, but I think it does point in the direction of this not being a great fit for Consult.

What kills the idea for me is mostly the lack of orthogonality to the other commands. I don't want to maintain an isearch clone - isearch has so many features which I cannot reasonably support. Furthermore I think the feature is not that widely used or needed since it hasn't been requested often. When Doom adopted Consult, @iyefrat observed that a swiper-isearch equivalent is missing but it wasn't a blocker for them. The main advantage of swiper-isearch is that it avoids the startup overhead (The startup overhead would be horrible for your final segments idea). If you are working with such large files, using consult-ripgrep is another alternative.

The abuse of completion is certainly a hint too. But with async completion tables and dynamic completion tables (see also completion-dynamic-table) the boundaries of completion are quite broad. Also Helm, Ivy and selectrum--read support some dynamic completion tables in the style of completion-table-dynamic. Selectrum supports this in order to work around its lack of completion boundary support.

The way you use avy, isearch, consult-line pretty much matches how I use the commands. For coarse navigation, jumping to headlines etc, there is also consult-imenu and consult-outline.

Another alternative could be to add C-s, C-r bindings to consult-line which use the minibuffer input and jump around with search-forward/backward on the current line or select the next/previous candidate. But such commands are probably wiki-quality and should be maintained in user configs, in particular since they would not work for arbitrary inputs, only single words.

[oantolin]

(The startup overhead would be horrible for your final segments idea)

Yes, it would! :)

Another alternative could be to add C-s, C-r bindings to consult-line which use the minibuffer input and jump around with search-forward/backward on the current line or select the next/previous candidate. But such commands are probably wiki-quality and should be maintained in user configs, in particular since they would not work for arbitrary inputs, only single words.

I guess those could use the same bisection method you use to decide where the match starts to look for the next or previous match.

[minad]

Closing this for now. If someone is willing to provide a PR which adds such C-s/C-r jump commands I may reconsider. But I thought about this for a while and if one uses the bisection method, the effort is considerable and will still not lead to a perfect UI. For example with Orderless multiple matches on a single line are not highlighted.

[amosbird]

swiper-isearch also acts as a performance booster. For huge buffers, it's significantly faster than consult-line and swiper.

[minad]

@amosbird Of course, but I see no reason to add yet another search command. As search/jump commands one can use avy, isearch, consult-line, consult-line-multi, consult-focus-lines, consult-grep and also the builtins occur, multi-occur and grep. There are probably many more alternatives provided by other packages. If you really want swiper-isearch you can continue to use swiper. Alternatively you can create your own command based on the prototype given in https://github.com/minad/consult/issues/417#issuecomment-922480825 and add it to your configuration or even publish it as your own package.

[amosbird]

but I see no reason to add yet another search command.

My motivation is different from OP. isearch is fast but lacks decent UI to collect all matches inside one buffer. Actually I don't use swiper at all. I use swiper-isearch as a complete replacement.

Alternatively you can create your own command based on the prototype

Cool. Lemme check if it works as fast as swiper-isearch

[minad]

I've implemented a dynamic consult-line-multi command, see https://github.com/minad/consult/commit/1247248ff023c970591ec2a99655132e2a81ee45 and #644. It should behave mostly like consult-grep but on buffers. The candidates are dynamically computed after entering some input, like swiper-isearch.

[gsingh93]

@minad I tried your example implementation in https://github.com/minad/consult/issues/417#issuecomment-922480825, but I get an error:

Debugger entered--Lisp error: (void-variable async)
  (funcall async action)
  (if (and (stringp action) (not (equal action ""))) (let ((lines)) (save-current-buffer (set-buffer (window-buffer (minibuffer-selected-window))) (save-excursion (goto-char (point-min)) (while (search-forward-regexp action nil t 1) (setq lines (cons (consult--location-candidate ... ... ...) lines))))) (funcall async 'flush) (funcall async (nreverse lines))) (funcall async action))
  (lambda (action) (if (and (stringp action) (not (equal action ""))) (let ((lines)) (save-current-buffer (set-buffer (window-buffer (minibuffer-selected-window))) (save-excursion (goto-char (point-min)) (while (search-forward-regexp action nil t 1) (setq lines ...)))) (funcall async 'flush) (funcall async (nreverse lines))) (funcall async action)))(setup)
  #f(compiled-function (action) #<bytecode 0x7750094ba8c46d1>)(setup)
  #f(compiled-function (action) #<bytecode -0x13018df83c5a23a8>)(setup)
  consult--minibuffer-setup-hook()
  #<subr completing-read-default>("Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil)
  apply((#<subr completing-read-default> "Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil))
  vertico--advice(#<subr completing-read-default> "Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil)
  apply(vertico--advice #<subr completing-read-default> ("Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil))
  completing-read-default("Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil)
  completing-read("Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil)
  #f(compiled-function () #<bytecode -0x7a6204e57411c85>)()
  consult--with-preview-1(any #f(compiled-function (action cand) #<bytecode 0x1e870b7d96306e41>) #f(compiled-function (narrow input cand) #<bytecode 0x999083ac25c9560>) #f(compiled-function (&rest args2) #<bytecode 0xfceaa82fb803ec6>) #f(compiled-function () #<bytecode -0x7a6204e57411c85>))
  consult--read-1(#f(compiled-function (action) #<bytecode -0x13018df83c5a23a8>) :sort nil :lookup consult--lookup-location :state #f(compiled-function (action cand) #<bytecode 0x1e870b7d96306e41>) :prompt "Select: " :preview-key any :sort t :lookup #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_162>)
  consult--read(#f(compiled-function (action) #<bytecode -0x13018df83c5a23a8>) :sort nil :lookup consult--lookup-location :state #f(compiled-function (action cand) #<bytecode 0x1e870b7d96306e41>))
  consult-line-delayed()
  funcall-interactively(consult-line-delayed)
  command-execute(consult-line-delayed record)
  execute-extended-command(nil "consult-line-delayed" "consult-")
  funcall-interactively(execute-extended-command nil "consult-line-delayed" "consult-")
  command-execute(execute-extended-command)

I know it's been a while since that comment so something probably changed, but I don't understand these async functions well enough to know what. This is on emacs 29.

[gsingh93]

Sorry, the above error was because I forgot to add ;; -*- lexical-binding: t -*- to the top of the file. But there is still another error after typing three characters:

Debugger entered--Lisp error: (wrong-number-of-arguments (4 . 4) 3)
  consult--location-candidate(#("(defun consult-line-delayed-async (async)" 1 6 (face font-lock-keyword-face) 7 33 (face font-lock-function-name-face)) #<marker at 42 in tmp.el> 3)
  (cons (consult--location-candidate (consult--buffer-substring (line-beginning-position) (line-end-position) 'fontify) (point-marker) (line-number-at-pos)) lines)
  (setq lines (cons (consult--location-candidate (consult--buffer-substring (line-beginning-position) (line-end-position) 'fontify) (point-marker) (line-number-at-pos)) lines))
  (while (search-forward-regexp action nil t 1) (setq lines (cons (consult--location-candidate (consult--buffer-substring (line-beginning-position) (line-end-position) 'fontify) (point-marker) (line-number-at-pos)) lines)))
  (save-excursion (goto-char (point-min)) (while (search-forward-regexp action nil t 1) (setq lines (cons (consult--location-candidate (consult--buffer-substring (line-beginning-position) (line-end-position) 'fontify) (point-marker) (line-number-at-pos)) lines))))
  (save-current-buffer (set-buffer (window-buffer (minibuffer-selected-window))) (save-excursion (goto-char (point-min)) (while (search-forward-regexp action nil t 1) (setq lines (cons (consult--location-candidate (consult--buffer-substring (line-beginning-position) (line-end-position) 'fontify) (point-marker) (line-number-at-pos)) lines)))))
  (let ((lines)) (save-current-buffer (set-buffer (window-buffer (minibuffer-selected-window))) (save-excursion (goto-char (point-min)) (while (search-forward-regexp action nil t 1) (setq lines (cons (consult--location-candidate (consult--buffer-substring ... ... ...) (point-marker) (line-number-at-pos)) lines))))) (funcall async 'flush) (funcall async (nreverse lines)))
  (if (and (stringp action) (not (equal action ""))) (let ((lines)) (save-current-buffer (set-buffer (window-buffer (minibuffer-selected-window))) (save-excursion (goto-char (point-min)) (while (search-forward-regexp action nil t 1) (setq lines (cons (consult--location-candidate ... ... ...) lines))))) (funcall async 'flush) (funcall async (nreverse lines))) (funcall async action))
  (closure ((async . #f(compiled-function (action) #<bytecode 0xe7961b4fd7dde81>))) (action) (if (and (stringp action) (not (equal action ""))) (let ((lines)) (save-current-buffer (set-buffer (window-buffer (minibuffer-selected-window))) (save-excursion (goto-char (point-min)) (while (search-forward-regexp action nil t 1) (setq lines ...)))) (funcall async 'flush) (funcall async (nreverse lines))) (funcall async action)))("con")
  #f(compiled-function (action) #<bytecode 0x716fe59590bc6d2>)("con")
  #f(compiled-function () #<bytecode 0x5b42f1d407d813f>)()
  apply(#f(compiled-function () #<bytecode 0x5b42f1d407d813f>) nil)
  timer-event-handler([t 25653 57639 791477 nil #f(compiled-function () #<bytecode 0x5b42f1d407d813f>) nil nil 0 nil])
  #<subr completing-read-default>("Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil)
  apply((#<subr completing-read-default> "Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil))
  vertico--advice(#<subr completing-read-default> "Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil)
  apply(vertico--advice #<subr completing-read-default> ("Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil))
  completing-read-default("Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil)
  completing-read("Select: " #f(compiled-function (str pred action) #<bytecode -0x1a19c5aea15d502>) nil nil nil nil nil nil)
  #f(compiled-function () #<bytecode -0x7a6204e57411c85>)()
  consult--with-preview-1(any #f(compiled-function (action cand) #<bytecode 0x1e81833b2fbcee41>) #f(compiled-function (narrow input cand) #<bytecode 0x99f686189c82d60>) #f(compiled-function (&rest args2) #<bytecode 0xc8f2487d0153ec6>) #f(compiled-function () #<bytecode -0x7a6204e57411c85>))
  consult--read-1(#f(compiled-function (action) #<bytecode 0x193a7111e339884>) :sort nil :lookup consult--lookup-location :state #f(compiled-function (action cand) #<bytecode 0x1e81833b2fbcee41>) :prompt "Select: " :preview-key any :sort t :lookup #<subr F616e6f6e796d6f75732d6c616d626461_anonymous_lambda_162>)
  consult--read(#f(compiled-function (action) #<bytecode 0x193a7111e339884>) :sort nil :lookup consult--lookup-location :state #f(compiled-function (action cand) #<bytecode 0x1e81833b2fbcee41>))
  consult-line-delayed()
  funcall-interactively(consult-line-delayed)
  command-execute(consult-line-delayed record)
  execute-extended-command(nil "consult-line-delayed" "consult-line")
  funcall-interactively(execute-extended-command nil "consult-line-delayed" "consult-line")
  command-execute(execute-extended-command)