Only present the result when the rg process has finished

[geza-herman]

A possible enhancement, sorry if it is already available in some way.

When I execute an rg command, the results are immediately displayed in emacs, on-the-fly. This makes rg very slow. On my machine, the speed advantages of rg are completely lost.

For example, here are the timing results of a search for PM_ in the 4.18 linux kernel (~60,000 results):

• rg in emacs: 39 seconds (this is with (setq read-process-output-max (* 1024 1024)))
• rg in command line: 0.8 sec (in comparison, pipeing the result into /dev/null, it is ~0.1 sec)
• grep in command line: 1.3 sec (pipeing to /dev/null, it is 1.0 sec)

So, it would be a good speed enhancement, if we had the option that results are not shown until rg is finished (I suppose that rg in emacs is slow, because of the constant buffer processing).

[dajva]

What is the use case for this? I think displaying 60.000 results is going to be slow either way so I doubt the end result is going to be much faster, i.e. the total time of searching and displaying the result.

Personally I would think 60.000 results would be too much to dig through manually so the package is not at all optimized for such cases. There are opportunities for improving the speed here. I think one could be to set rg-group-result to nil. Another would be to add the possibility to disable highlighting the matches which is not possible today.

[geza-herman]

This was an exaggerated example, but I regularly execute greps where there are thousands of results, and then execute a search and replace afterwards. And as the command line's rg is instant in this case, it actually unnecessary to process the result on-the-fly (even for 60,000 results, rg's time is only 0.1 sec, it is not really worthy to process on-the-fly).

I've checked:

• I executed emacs rg as my previous example (60,000 results)
• copied the result into scratch buffer
• turned on rg-mode on scratch buffer. It is instantaneous.
• If I do the same, when the cursor is at the bottom of the scratch buffer, turning on rg-mode takes 15 seconds. But only 2 seconds, if the search was executed with rg-group-result as nil (but executing the search still takes ~40 seconds in this case. So, for someone, who has grouping turned off, disabling on-the-fly processing gives 20x speed boost).

[dajva]

Thanks for the clarification. Just turning on rg-mode isn't enough to simulate the result processing, you need something like:

(defun rg-simulate-result ()
  (interactive)
  (goto-char (point-max))
  (let ((compilation-filter-start 0)
        (start (current-time)))
    (rg-filter)
    (rg-mode)
    (message "Time: %f" (float-time (time-since start)))))

The time does not include the actual display processing so even more time until the results is showing properly on screen. But anyway, the processing of the ripgrep output is handled by compilation mode with rg.el adding the rg-filter as a hook to do the rg.el specific processing. So this package is relying on upstream behaviour there. Maybe it's possible to further tweak how compilation-mode or its upstream modes behave in this area. Nothing available directly in this package at least.

[geza-herman]

I checked, and it prints Time: 0.123753 for my example. And yes, all the other time goes into font-lock presumably.

Thanks for the explanation of the internals. It seems that feature would need a lot of work, and admittedly doesn't give too much benefit.

Thanks for your time!

[dajva]

I suspect your example isn't really what happens though. Seems too fast. :) You need the correct flags to get all the color codes that is used internally in rg-filter. I think something like this should do it:

rg --color always --colors match:fg:red --colors path:fg:magenta --colors line:fg:green --colors column:none -n --column -i --heading --no-config -e PM_

[geza-herman]

Here's the exact example: rg.txt.gz.

If I gunzip it, and load it into emacs, and switch to fundamental mode, then (rg-simulate-result) prints times around 0.12. And the whole process lasts about 7 seconds (I don't know why I measured ~15 seconds before). It seems that this search does have some colors.

Maybe I measure an unexpectedly low number, because I use a newer emacs (I built it about a week ago from master)?

My whole configuration related torg is this (I used the custom rg-my-default search to create rg.txt.gz):

(use-package rg
  :defer t
  :init
  ;; (setq rg-group-result nil)
  :config

  (transient-append-suffix 'rg-menu "-a" '(3 "-b" "Search binary files" "--binary"))

  (rg-define-search rg-my-default
    "Search everything, binary, zipped."
    :files "everything"
    :flags '("--search-zip" "--binary")
    :confirm prefix)
)