Bug: rg backend empty pattern will crash emacs

[zozowell]

• Emacs version: 28.1
• Operating system: MacOS Catalina
• Search command(ag, pt, ack etc) and version: ripgrep 13.0.0
• Last value of helm-ag--last-command : ("rg" "--no-heading" "--ignore=node_modules" "--ignore=workspaces" "|he")

Actual behavior

Some weird error messages are showed from minibuffer, emacs hangs there. C-g has no effect, or just suspend the process and asks for dumping core. It was triggered by the empty pattern before or symbol |, this will cause rg returns every line of every files, basically a mage output.

The hanging sometimes ends with following error message, but most of the time, it very very slowly recovered with the error message in minibuffer disappeared one by one... feel like there were some long run processes finally gradually finished. However, from sys monitor, I couldn't see massive processes, or even couldn't see "rg" process ran there.

progn: End of buffer [203 times]
while: Variable binding depth exceeds max-specpdl-size
and: Variable binding depth exceeds max-specpdl-size
let: Variable binding depth exceeds max-specpdl-size
Error during redisplay: (#[128 "\300\301^B\"\210\300\302^B\"\207" [apply redisplay--pre-redisplay-functions ignore nil] 4 nil nil] nil) signaled$
sit-for: Variable binding depth exceeds max-specpdl-size

I tried to debug, but couldn't capture the real stuck point, debug-on-message can't capture the error message. debug-on-quit did work because c-g didn't resolve the hang actually.

Profiling shows following weird call stack, I couldn't see where the function minibuffer-error-function is called at all...

510  85% - command-execute
         510  85%  - funcall-interactively
         424  71%   - helm-do-ag-project-root
         424  71%    - let
         424  71%     - helm-do-ag
         424  71%      - let*
         424  71%       - if
         424  71%        - helm-do-ag--helm
         424  71%         - let
         423  70%          - helm
         423  70%           - helm
         423  70%            - helm-internal
         422  70%             - helm-read-from-minibuffer
         413  69%              - minibuffer-error-function
         413  69%               - minibuffer-message
         413  69%                - sit-for
         412  69%                 - minibuffer-error-function
         412  69%                  - minibuffer-message
         412  69%                   - sit-for
         410  68%                    - minibuffer-error-function
         410  68%                     - minibuffer-message
         409  68%                      - sit-for
         409  68%                       - minibuffer-error-function
         409  68%                        - minibuffer-message
         409  68%                         - sit-for
         408  68%                          - minibuffer-error-function
         408  68%                           - minibuffer-message
         407  68%                            - sit-for
         401  67%                             - minibuffer-error-function
         400  67%                              - minibuffer-message
         395  66%                               - sit-for
           2   0%                                - minibuffer-error-function
           1   0%                                   substitute-command-keys
           1   0%                                - redisplay_internal (C function)
           1   0%                                   tty-color-desc
           1   0%                              + substitute-command-keys
           4   0%                             + helm-output-filter

helm-log didn't reveal too much, the biggest time cost was here:

** 12:19:28.366453 (helm-output-filter-1)
 output-string = ".
init.el:564:;;                          (global-flycheck-mode) ;; enable flycheck manually.
init.el:565:;;                          (require 'lsp-pyright) ;; use lsp-pyright lsp
init.el:566:;;                          (lsp-deferred)
init.el:567:;; )))
"
** 12:19:28.366677 (helm-output-filter--collect-candidates)
 incomplete-line-info = "init.el:563:;;                          (setq flycheck-checker 'python-pyright) ;; use 'python-pyright instead as python checker"
** 12:24:23.646604 (helm-check-new-input)
 helm-pattern = "|S"
** 12:24:23.646781 (helm-check-new-input)
 helm-input = "|S"
** 12:24:23.646916 (helm-check-new-input)
 Executing helm-before-update-hook with value = nil
** 12:24:23.647035 (helm-check-new-input)
 Executing helm-before-update-hook with global value = nil
** 12:24:23.647148 (helm-check-new-input)
 executed helm-before-update-hook
** 12:24:23.647260 (helm-update)
 Start updating
** 12:24:23.647389 (helm--collect-matches)
 Using here `helm-while-no-input'
** 12:24:23.647503 (helm-update)
 Matches: nil
** 12:24:23.647773 (helm-update)
 Executing helm-after-update-hook with value = (helm--maybe-update-keymap helm-match-line-cleanup-maybe helm-revive-visible-mark helm-confirm-and-exit-hook)
** 12:24:23.647898 (helm-update)
 Executing helm-after-update-hook with global value = (helm--maybe-update-keymap helm-match-line-cleanup-maybe helm-revive-visible-mark helm-confirm-and-exit-hook)

Expected behavior

• It shows the search result, upto candidate number limit.
• C-g can stop the searching process.

Steps to reproduce

1. using helm-ag 20210702.845
2. Configuration

   (use-package helm-ag
   :ensure t
   :config
   (setq helm-ag-base-command "rg --no-heading")
   (setq helm-ag-success-exit-status '(0 2))
   (setq helm-ag-fuzzy-match t)
   (setq helm-ag-insert-at-point 'symbol)
   (setq helm-ag-ignore-patterns '("node_modules" "workspaces"))
   (setq helm-ag-ignore-buffer-patterns '("\\.txt\\'" "\\.mkd\\'")))

3. Using helm-ag-do-project-root command do the project search, search pattern |helm

[jcs090218]

@zozowell Can you test the patch #398?

[zozowell]

hi @jcs090218, @twlz0ne I verified, it works as expected now, the behavior is identical with helm-do-grep. Thanks! And I'm very curious about how you experts debug and identified this root cause, may you share some tips please?