[Bug] citre-global-dbpath causes Emacs hung indefinitely when opening any org files

[lambertlulala]

Hello @AmaiKinono @masatake

Thank you for the maintenance of such a great tool.

When I open any org (org mode) files, Emacs hangs, seems it keep running inside some loop indefinitely. I found citre-global-dbpath may cause Emacs hung. The reason is that I noticed when any org file opened, Emacs ran global --print-dbpath. And it seems that citre will automatically run that routine.

My citre configuration:

(when (require 'citre nil t)
  (require 'citre-config)
  (global-set-key (kbd "C-x c j") 'citre-jump)
  (global-set-key (kbd "C-x c J") 'citre-jump-back)
  (global-set-key (kbd "C-x c p") 'citre-ace-peek)
  (global-set-key (kbd "C-x c u") 'citre-update-this-tags-file))

Running on Windows. But I think it has nothing to do with OS.

Any suggestion to resolve the issue? Thanks in advance.

[lambertlulala]

The temporary workaround solution for me is add the code in my configuration:

(defun citre-global-dbpath-workaround (&optional dir)
    "Workaround solution for the citre-global-dbpath issue.")

(advice-add 'citre-global-dbpath :around #'citre-global-dbpath-workaround)

Yes, it makes citre-global-dbpath do nothing, I can't figure out other solution for the moment.

[Update] Another workaround I found is add the code in the configuration to turn off global features:

(custom-set-variables
 '(citre-auto-enable-citre-mode-backends '(tags))
 '(citre-completion-backends '(tags))
 '(citre-find-definition-backends '(tags))
 '(citre-tags-in-buffer-backends '(tags)))

[lambertlulala]

Another similar issue I encountered also resulted in Emacs hung.

The steps to reproduce:

1. Use citre-create-tags-file to create the tag file.
2. Wait 1. finish and use citre-create-tags-file again to overwrite the original tag file.
3. Wait 2. finish and use citre-create-tags-file again to overwrite the original tag file. But in this step, Emacs hangs again.

But this scenario I can't find other workaround solution. I just incidentally found this issue.

[AmaiKinono]

It is actually a reported issue: https://github.com/universal-ctags/citre/issues/150. Strangely, it never happens to me and most of the users, and our CI never hangs (so it may have something to do with the OS). That's why it's hard for me to come up with a fix.

I would like you to try the develop branch, or the solution here: https://github.com/universal-ctags/citre/issues/150#issuecomment-1872979244, just paste it, eval it and see if the problem is gone.

[lambertlulala]

It is actually a reported issue: #150. Strangely, it never happens to me and most of the users, and our CI never hangs (so it may have something to do with the OS). That's why it's hard for me to come up with a fix.

I would like you to try the develop branch, or the solution here: #150 (comment), just paste it, eval it and see if the problem is gone.

Thank you for the help, I tried it but unfortunately it didn't work. I may turn off global features first.

[AmaiKinono]

Could you help me to see on which line is Emacs stucked? The steps are:

• Eval the solution in https://github.com/universal-ctags/citre/issues/150#issuecomment-1872979244.
• Turn on "debug on quit" by M-x toggle-debug-on-quit
• Eval (citre-global-dbpath), which should cause Emacs to stuck.
• Press C-g, and a backtrace should show. Paste it here.

Thanks.

[lambertlulala]

The result

Debugger entered--Lisp error: (quit)
  (while (null finish-status) (accept-process-output proc nil 50))
  (progn (while (eq (process-status proc) 'run) (accept-process-output)) (while (null finish-status) (accept-process-output proc nil 50)) (cond ((eq finish-status 'success) (let nil (accept-process-output proc))) ((eq finish-status 'fail) (let nil (error err-msg))) (t (let nil (error "Invalid FINISH-STATUS %s" finish-status)))) result)
  (unwind-protect (progn (while (eq (process-status proc) 'run) (accept-process-output)) (while (null finish-status) (accept-process-output proc nil 50)) (cond ((eq finish-status 'success) (let nil (accept-process-output proc))) ((eq finish-status 'fail) (let nil (error err-msg))) (t (let nil (error "Invalid FINISH-STATUS %s" finish-status)))) result) (citre-destruct-process proc-data))
  (let* ((result nil) (err-msg nil) (finish-status nil) (callback #'(lambda (status msg) (cond ((eq status ...) (let nil ...)) ((eql status 0) (let nil ...)) ((integerp status) (let ... ...)) ((eq status ...) (let nil ...)) (t (let ... ...))))) (proc-data (citre-make-async-process cmd callback)) (proc (progn (or (progn (and (memq ... cl-struct-citre-process-tags) t)) (signal 'wrong-type-argument (list 'citre-process proc-data))) (aref proc-data 1)))) (unwind-protect (progn (while (eq (process-status proc) 'run) (accept-process-output)) (while (null finish-status) (accept-process-output proc nil 50)) (cond ((eq finish-status 'success) (let nil (accept-process-output proc))) ((eq finish-status 'fail) (let nil (error err-msg))) (t (let nil (error "Invalid FINISH-STATUS %s" finish-status)))) result) (citre-destruct-process proc-data)))
  citre-get-output-lines(("global" "--print-dbpath"))
  citre-global--get-output-lines(("--print-dbpath"))
  #f(compiled-function (dir) #<bytecode 0x16bac458d3f15d1b>)(nil)
  citre-global-dbpath()
  citre-backend-usable-p(global)
  citre-auto-enable-citre-mode()
  run-hooks(find-file-hook)
  after-find-file(nil t)
  find-file-noselect-1(#<buffer test.org> "d:/test.org" nil nil "d:/test.org" (12103423998601720 2091930523))
  find-file-noselect("d:/test.org" nil nil t)
  find-file("d:/test.org" t)
  funcall-interactively(find-file "d:/test.org" t)
  command-execute(find-file)

[AmaiKinono]

Thanks. It seems that finish-status is never set to non-nil, which may because finish-status is a local variable, and if the process callback function never captures it, it may try to change finish-status as a global variable. This shouldn't happen when using lexical binding, but maybe there's a bug on windows or so...

I've tried to come up with another fix (on the develop branch), could you try it out? A simple way is to copy https://raw.githubusercontent.com/universal-ctags/citre/develop/citre-common-util.el into a buffer and eval the buffer.

[lambertlulala]

Thanks. It seems that finish-status is never set to non-nil, which may because finish-status is a local variable, and if the process callback function never captures it, it may try to change finish-status as a global variable. This shouldn't happen when using lexical binding, but maybe there's a bug on windows or so...

I've tried to come up with another fix (on the develop branch), could you try it out? A simple way is to copy https://raw.githubusercontent.com/universal-ctags/citre/develop/citre-common-util.el into a buffer and eval the buffer.

The result

Debugger entered--Lisp error: (quit)
  (accept-process-output)
  (while (null (process-get proc 'citre-finish-status)) (accept-process-output))
  (progn (while (null (process-get proc 'citre-finish-status)) (accept-process-output)) (let* ((val (process-get proc 'citre-finish-status))) (cond ((eq val 'success) (let nil (accept-process-output proc))) ((eq val 'fail) (let nil (error err-msg))) (t (let ((s val)) (error "Invalid FINISH-STATUS %s" s))))) result)
  (unwind-protect (progn (while (null (process-get proc 'citre-finish-status)) (accept-process-output)) (let* ((val (process-get proc 'citre-finish-status))) (cond ((eq val 'success) (let nil (accept-process-output proc))) ((eq val 'fail) (let nil (error err-msg))) (t (let ((s val)) (error "Invalid FINISH-STATUS %s" s))))) result) (citre-destruct-process proc-data))
  (let* ((result nil) (err-msg nil) (callback #'(lambda (proc status msg) (cond ((eq status ...) (let nil ...)) ((eql status 0) (let nil ...)) ((integerp status) (let ... ... ...)) ((eq status ...) (let nil ...)) (t (let ... ... ...))))) (proc-data (citre-make-async-process cmd callback)) (proc (progn (or (progn (and (memq ... cl-struct-citre-process-tags) t)) (signal 'wrong-type-argument (list 'citre-process proc-data))) (aref proc-data 1)))) (unwind-protect (progn (while (null (process-get proc 'citre-finish-status)) (accept-process-output)) (let* ((val (process-get proc 'citre-finish-status))) (cond ((eq val 'success) (let nil (accept-process-output proc))) ((eq val 'fail) (let nil (error err-msg))) (t (let (...) (error "Invalid FINISH-STATUS %s" s))))) result) (citre-destruct-process proc-data)))
  citre-get-output-lines(("global" "--print-dbpath"))
  citre-global--get-output-lines(("--print-dbpath"))
  #f(compiled-function (dir) #<bytecode -0x1b1d3390d80ea2e6>)(nil)
  citre-global-dbpath()
  citre-backend-usable-p(global)
  citre-auto-enable-citre-mode()
  run-hooks(find-file-hook)
  after-find-file(nil t)
  find-file-noselect-1(#<buffer test.org> "d:/test.org" nil nil "d:/test.org" (12103423998601720 2091930523))
  find-file-noselect("d:/test.org" nil nil t)
  find-file("d:/test.org" t)
  funcall-interactively(find-file "d:/test.org" t)
  command-execute(find-file)

[AmaiKinono]

Thanks very much and I hope I didn't bother you too much.

I've fixed some other thing that I suspect may cause the problem, and the develop branch is updated. I'd like to ask you to try it again. This may either work or leave me having absolutely no idea about what causes the problem...

[lambertlulala]

Thanks very much and I hope I didn't bother you too much.

I've fixed some other thing that I suspect may cause the problem, and the develop branch is updated. I'd like to ask you to try it again. This may either work or leave me having absolutely no idea about what causes the problem...

No problem, it's OK. Appreciate the consistent maintenance.

The result of the latest commit https://raw.githubusercontent.com/universal-ctags/citre/develop/citre-common-util.el

Debugger entered--Lisp error: (quit)
  (accept-process-output)
  (while (null (process-get proc 'citre-finish-status)) (accept-process-output))
  (progn (while (null (process-get proc 'citre-finish-status)) (accept-process-output)) (let* ((val (process-get proc 'citre-finish-status))) (cond ((eq val 'success) (let nil (accept-process-output proc))) ((eq val 'fail) (let nil (error err-msg))) (t (let ((s val)) (error "Invalid FINISH-STATUS %s" s))))) result)
  (unwind-protect (progn (while (null (process-get proc 'citre-finish-status)) (accept-process-output)) (let* ((val (process-get proc 'citre-finish-status))) (cond ((eq val 'success) (let nil (accept-process-output proc))) ((eq val 'fail) (let nil (error err-msg))) (t (let ((s val)) (error "Invalid FINISH-STATUS %s" s))))) result) (citre-destruct-process proc-data))
  (let* ((result nil) (err-msg nil) (callback #'(lambda (proc status msg) (cond ((eq status ...) (let nil ...)) ((eql status 0) (let nil ...)) ((integerp status) (let ... ... ...)) ((eq status ...) (let nil ...)) (t (let ... ... ...))))) (proc-data (citre-make-async-process cmd callback)) (proc (progn (or (progn (and (memq ... cl-struct-citre-process-tags) t)) (signal 'wrong-type-argument (list 'citre-process proc-data))) (aref proc-data 1)))) (unwind-protect (progn (while (null (process-get proc 'citre-finish-status)) (accept-process-output)) (let* ((val (process-get proc 'citre-finish-status))) (cond ((eq val 'success) (let nil (accept-process-output proc))) ((eq val 'fail) (let nil (error err-msg))) (t (let (...) (error "Invalid FINISH-STATUS %s" s))))) result) (citre-destruct-process proc-data)))
  citre-get-output-lines(("global" "--print-dbpath"))
  citre-global--get-output-lines(("--print-dbpath"))
  #f(compiled-function (dir) #<bytecode -0xea0816d100ea2e2>)(nil)
  citre-global-dbpath()
  citre-backend-usable-p(global)
  citre-auto-enable-citre-mode()
  run-hooks(find-file-hook)
  after-find-file(nil t)
  find-file-noselect-1(#<buffer test.org> "d:/test.org" nil nil "d:/test.org" (12103423998601720 2091930523))
  find-file-noselect("d:/test.org" nil nil t)
  find-file("d:/test.org" t)
  funcall-interactively(find-file "d:/test.org" t)
  command-execute(find-file)

[lambertlulala]

Actually, I noticed when this situation occurred, Emacs kept hanging but the global process had terminated (no any running global process), no matter the version on master branch or develop branch (i.e. your latest commit). I'm not sure whether it helped you debug.

[AmaiKinono]

Thanks. Fortunately ( :rofl: ) I've managed to encounter some similar issue when wrapping a citre-get-output-lines call in a company backend. I've redesigned some of the things to fix things on my side, and I hope it works for you.

Please try the latest citre-common-util.el. Also, to make sure lexical binding is used, please do paste it in an empty buffer, making sure the first line containing -*- lexical-binding: t -*- is keeped, and run eval-buffer.

Emacs kept hanging but the global process had terminated (no any running global process)

This looks like the process sentinel (which is a callback Emacs calls when the status of an async process is changed) is never called, which really, really shouldn't happen. I've included some logging in the source code, so if it fails again, please look at the *message* buffer and see if there's a line says "Citre: Process sentinel is running.".

P.S. Sorry for posting the same comment earlier. I did some more fix and deleted that comment.

[lambertlulala]

This looks like the process sentinel (which is a callback Emacs calls when the status of an async process is changed) is never called, which really, really shouldn't happen. I've included some logging in the source code, so if it fails again, please look at the *message* buffer and see if there's a line says "Citre: Process sentinel is running.".

I just tested it, and the result

Debugger entered--Lisp error: (quit)
  accept-process-output()
  (while (eq (progn (or (progn (and (memq (type-of proc-data) cl-struct-citre-process-tags) t)) (signal 'wrong-type-argument (list 'citre-process proc-data))) (aref proc-data 5)) 'run) (accept-process-output))
  (progn (while (eq (progn (or (progn (and (memq ... cl-struct-citre-process-tags) t)) (signal 'wrong-type-argument (list 'citre-process proc-data))) (aref proc-data 5)) 'run) (accept-process-output)) (accept-process-output proc) (let* ((val (progn (or (progn (and ... t)) (signal 'wrong-type-argument (list ... proc-data))) (aref proc-data 5)))) (cond ((eq val 'success) 'nil) ((consp val) (let* ((x3 (car-safe val))) (cond ((integerp x3) (let* ... ...)) ((eq x3 ...) (let* ... ...)) (t (let ... ...))))) (t (let ((s val)) (error "Invalid PROCESS-STATUS %s" s))))) result)
  (unwind-protect (progn (while (eq (progn (or (progn (and ... t)) (signal 'wrong-type-argument (list ... proc-data))) (aref proc-data 5)) 'run) (accept-process-output)) (accept-process-output proc) (let* ((val (progn (or (progn ...) (signal ... ...)) (aref proc-data 5)))) (cond ((eq val 'success) 'nil) ((consp val) (let* ((x3 ...)) (cond (... ...) (... ...) (t ...)))) (t (let ((s val)) (error "Invalid PROCESS-STATUS %s" s))))) result) (citre-destruct-process proc-data))
  (let* ((result nil) (callback #'(lambda (status msg) (if (eq status 'output) (progn (setq result ...))))) (proc-data (citre-make-async-process cmd callback)) (proc (progn (or (progn (and (memq ... cl-struct-citre-process-tags) t)) (signal 'wrong-type-argument (list 'citre-process proc-data))) (aref proc-data 1)))) (unwind-protect (progn (while (eq (progn (or (progn ...) (signal ... ...)) (aref proc-data 5)) 'run) (accept-process-output)) (accept-process-output proc) (let* ((val (progn (or ... ...) (aref proc-data 5)))) (cond ((eq val 'success) 'nil) ((consp val) (let* (...) (cond ... ... ...))) (t (let (...) (error "Invalid PROCESS-STATUS %s" s))))) result) (citre-destruct-process proc-data)))
  citre-get-output-lines(("global" "--print-dbpath"))
  citre-global--get-output-lines(("--print-dbpath"))
  #f(compiled-function (dir) #<bytecode 0x1d2a9ca217f15d1f>)(nil)
  citre-global-dbpath()
  citre-backend-usable-p(global)
  citre-auto-enable-citre-mode()
  run-hooks(find-file-hook)
  after-find-file(nil t)
  find-file-noselect-1(#<buffer test.org> "d:/test.org" nil nil "d:/test.org" (12103423998601720 2091930523))
  find-file-noselect("d:/test.org" nil nil t)
  find-file("d:/test.org" t)
  funcall-interactively(find-file "d:/test.org" t)
  command-execute(find-file)

Yes, I did see the message Citre: Process sentinel is running. showing on *message*.

P.S. Sorry for posting the same comment earlier. I did some more fix and deleted that comment.

No problem, it' alright!

[AmaiKinono]

This is very, very weird. If you actually look into the sentinel, you'll see the process status is always modified, so the polling on the process status should finish.

The only thing I can suspect is the (accept-process-output) itself hangs, which I assume never happens (and it does not happen on my machine). But I've found a comment on reddit that says:

I've experimented with several forms of such a function (polling via accept-process-output and sit-for). They're all unreliable. It works some of the time, or it can hang indefinitely.

Before https://github.com/universal-ctags/citre/commit/97d4074795b4b2950bd845fb3902b8ccadde6658 I was using another hack to poll for the process status to change, but a lot of problems have been reported by Windows users (on some forum, not on GitHub). So what we may need to do is seeking for a reliable way to poll in Emacs, which may just be impossible.

[AmaiKinono]

I've added a 0.5s time limit on (accept-process-output), could you try the latest develop branch? And, if this fixes the problem for you, I'd like to know if you encounter a 0.5s delay when calling citre-peek, citre-jump or when auto-completing using Citre.

[lambertlulala]

Unfortunately it got the same result. :sweat_smile:

Debugger entered--Lisp error: (quit)
  cconv-fv(#'(lambda nil (throw 'timeout 'timeout)) (proc proc-data callback result cmd) (cl-struct-citre-process-tags t))
  cconv-make-interpreted-closure((lambda nil (throw 'timeout 'timeout)) ((proc . #<process global>) (proc-data . #s(citre-process :proc #<process global> :callback (closure ((result)) (status msg) (if (eq status 'output) (progn (setq result ...)))) :stderr-buffer #<buffer  *global-stderr*> :remote-p nil :status run :-stdout-str "")) (callback closure ((result)) (status msg) (if (eq status 'output) (progn (setq result (nconc result (split-string msg "\n" t)))))) (result) (cmd "global" "--print-dbpath") cl-struct-citre-process-tags t))
  #'(lambda nil (throw 'timeout 'timeout))
  (run-with-timer 0.5 nil #'(lambda nil (throw 'timeout 'timeout)))
  (let* ((-with-timeout-timer- (run-with-timer 0.5 nil #'(lambda nil (throw 'timeout 'timeout)))) (with-timeout-timers (cons -with-timeout-timer- with-timeout-timers))) (unwind-protect (progn (accept-process-output)) (cancel-timer -with-timeout-timer-)))
  (catch 'timeout (let* ((-with-timeout-timer- (run-with-timer 0.5 nil #'(lambda nil (throw ... ...)))) (with-timeout-timers (cons -with-timeout-timer- with-timeout-timers))) (unwind-protect (progn (accept-process-output)) (cancel-timer -with-timeout-timer-))))
  (let ((-with-timeout-value- (catch 'timeout (let* ((-with-timeout-timer- (run-with-timer 0.5 nil ...)) (with-timeout-timers (cons -with-timeout-timer- with-timeout-timers))) (unwind-protect (progn (accept-process-output)) (cancel-timer -with-timeout-timer-)))))) (if (eq -with-timeout-value- 'timeout) (progn nil) -with-timeout-value-))
  (while (eq (progn (or (progn (and (memq (type-of proc-data) cl-struct-citre-process-tags) t)) (signal 'wrong-type-argument (list 'citre-process proc-data))) (aref proc-data 5)) 'run) (let ((-with-timeout-value- (catch 'timeout (let* ((-with-timeout-timer- ...) (with-timeout-timers ...)) (unwind-protect (progn ...) (cancel-timer -with-timeout-timer-)))))) (if (eq -with-timeout-value- 'timeout) (progn nil) -with-timeout-value-)))
  (progn (while (eq (progn (or (progn (and (memq ... cl-struct-citre-process-tags) t)) (signal 'wrong-type-argument (list 'citre-process proc-data))) (aref proc-data 5)) 'run) (let ((-with-timeout-value- (catch 'timeout (let* (... ...) (unwind-protect ... ...))))) (if (eq -with-timeout-value- 'timeout) (progn nil) -with-timeout-value-))) (accept-process-output proc) (let* ((val (progn (or (progn (and ... t)) (signal 'wrong-type-argument (list ... proc-data))) (aref proc-data 5)))) (cond ((eq val 'success) 'nil) ((consp val) (let* ((x6 (car-safe val))) (cond ((integerp x6) (let* ... ...)) ((eq x6 ...) (let* ... ...)) (t (let ... ...))))) (t (let ((s val)) (error "Invalid PROCESS-STATUS: %s" s))))) result)
  (unwind-protect (progn (while (eq (progn (or (progn (and ... t)) (signal 'wrong-type-argument (list ... proc-data))) (aref proc-data 5)) 'run) (let ((-with-timeout-value- (catch 'timeout (let* ... ...)))) (if (eq -with-timeout-value- 'timeout) (progn nil) -with-timeout-value-))) (accept-process-output proc) (let* ((val (progn (or (progn ...) (signal ... ...)) (aref proc-data 5)))) (cond ((eq val 'success) 'nil) ((consp val) (let* ((x6 ...)) (cond (... ...) (... ...) (t ...)))) (t (let ((s val)) (error "Invalid PROCESS-STATUS: %s" s))))) result) (citre-destruct-process proc-data))
  (let* ((result nil) (callback #'(lambda (status msg) (if (eq status 'output) (progn (setq result ...))))) (proc-data (citre-make-async-process cmd callback)) (proc (progn (or (progn (and (memq ... cl-struct-citre-process-tags) t)) (signal 'wrong-type-argument (list 'citre-process proc-data))) (aref proc-data 1)))) (unwind-protect (progn (while (eq (progn (or (progn ...) (signal ... ...)) (aref proc-data 5)) 'run) (let ((-with-timeout-value- (catch ... ...))) (if (eq -with-timeout-value- 'timeout) (progn nil) -with-timeout-value-))) (accept-process-output proc) (let* ((val (progn (or ... ...) (aref proc-data 5)))) (cond ((eq val 'success) 'nil) ((consp val) (let* (...) (cond ... ... ...))) (t (let (...) (error "Invalid PROCESS-STATUS: %s" s))))) result) (citre-destruct-process proc-data)))
  citre-get-output-lines(("global" "--print-dbpath"))
  citre-global--get-output-lines(("--print-dbpath"))
  #f(compiled-function (dir) #<bytecode 0x1a6987750cf15d1f>)(nil)
  citre-global-dbpath()
  citre-backend-usable-p(global)
  citre-auto-enable-citre-mode()
  run-hooks(find-file-hook)
  after-find-file(nil t)
  find-file-noselect-1(#<buffer test.org> "d:/test.org" nil nil "d:/test.org" (17732923532814864 2091930523))
  find-file-noselect("d:/test.org" nil nil t)
  find-file("d:/test.org" t)
  funcall-interactively(find-file "d:/test.org" t)
  command-execute(find-file)

[AmaiKinono]

It looks like with-timeout doesn't work...

Could you try the latest citre-common-util.el on the develop branch? Please wait for several seconds when emacs is hanging, and after you press C-g, please see the *message* buffer and look for:

• "Citre: Process sentinel is running."
• "Citre: Finish a polling cycle." If this one is presented, please tell me about how many times is it repeated.

Also, I'd like to know:

• Does this happen only for org files or any other files?
• If you eval

  (let ((p (make-process :name "global" :command '("global" "--print-dbpath"))))
  (while (accept-process-output p)))

  manually in the opened org file, does it hang or return nil immediately? And what happens if you eval it in some other file?

• If you run $ global --print-dbpath in the command line in D:/, what happens?

[lambertlulala]

It looks like with-timeout doesn't work...

Could you try the latest citre-common-util.el on the develop branch? Please wait for several seconds when emacs is hanging, and after you press C-g, please see the *message* buffer and look for:

• "Citre: Process sentinel is running."
• "Citre: Finish a polling cycle." If this one is presented, please tell me about how many times is it repeated.

I waited for about 10 seconds. The message on *message*:

Debug on Quit enabled globally
Citre: Finish a polling cycle. [172128 times]
Entering debugger...
Citre: Process sentinel is running.

Also, I'd like to know:

• Does this happen only for org files or any other files?

I've tested .cpp and .txt files, they didn't hang. So yes, only org files so far.

• If you eval

  (let ((p (make-process :name "global" :command '("global" "--print-dbpath"))))
  (while (accept-process-output p)))

  manually in the opened org file, does it hang or return nil immediately? And what happens if you eval it in some other file?

Tested on org, cpp and text files, they all returned nil immediately

If the org is opened first time (i.e. after closing Emacs and restart it and then open the org file), it hangs. If I press C-g, however, it didn't hang. This situation existed at the first time I reported the issue.

Therefore, the steps to reproduce include:

1. Close Emacs.
2. Run Emacs and eval-buffer for that https://[raw.githubusercontent.com/universal-ctags/citre/develop/citre-common-util.el](https://raw.githubusercontent.com/universal-ctags/citre/develop/citre-common-util.el)
3. Open the org file first time -> Emacs hangs.
4. Press C-g to stop the routine.
5. Re-open the org file, and Emacs doesn't hang. And I evaluate the code above, it returned nil. Neither does Emacs hang after subsequent opening of org files.

• If you run $ global --print-dbpath in the command line in D:/, what happens?

It has two cases, with GTAGS or not.

1. With GTAGS, it will return D:/.
2. Without GTAGS, it will return global: GTAGS not found..

[AmaiKinono]

Thanks. The message you got indicates that the global process does run forever:

Debug on Quit enabled globally Citre: Finish a polling cycle. [172128 times] // Citre is waiting for the process to finish, and the polling mechanism works as expected. Entering debugger... // Here you pressed C-g. Citre: Process sentinel is running. // The process is terminated by C-g, and sentinel is called.

Citre does what it should do. Based on your experience, it seems running global in the find-file-hook when an org file is opened for the first time calls global to run forever, and at other times, it doesn't. Yeah that's very weird. I've tried your steps and didn't reproduce that on my side.

A simple experiment will check if my assumption is right, if you are interested. In your config file, remove global from all citre-*-backends, and add:

(defun run-global ()
  (let ((p (make-process :name "global" :command '("global" "--print-dbpath"))))
    (while (accept-process-output p))))

(add-hook 'find-file-hook #'run-global)

Close Emacs, and open an org file again. If it does hang, then it will be a bug in Emacs, org-mode or your config. You may try $ emacs -Q to know if it's a problem of your config. If it isn't, you may want to fill a bug report to GNU Emacs.

On Citre's side, I'll fix things by set a time limit for citre-global-dbpath.

[lambertlulala]

I've simplified my config without citre as follows.

(setq debug-on-error t)

(require 'package)
(package-initialize)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)

(add-to-list 'load-path (expand-file-name "lisp" user-emacs-directory))
(add-to-list 'load-path (expand-file-name "site-lisp" user-emacs-directory))

(defun run-global ()
  (let ((p (make-process :name "global" :command '("global" "--print-dbpath"))))
    (while (accept-process-output p))))

(add-hook 'find-file-hook #'run-global)

When open any file including org file, Emacs never hangs.

The config with citre:

(setq debug-on-error t)

(require 'package)
(package-initialize)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)

(add-to-list 'load-path (expand-file-name "lisp" user-emacs-directory))
(add-to-list 'load-path (expand-file-name "site-lisp" user-emacs-directory))

(when (require 'citre nil t)
  (require 'citre-config)
  (global-set-key (kbd "C-x c j") 'citre-jump)
  (global-set-key (kbd "C-x c J") 'citre-jump-back)
  (global-set-key (kbd "C-x c p") 'citre-ace-peek)
  (global-set-key (kbd "C-x c u") 'citre-update-this-tags-file))

(defun run-global ()
  (let ((p (make-process :name "global" :command '("global" "--print-dbpath"))))
    (while (accept-process-output p))))

(add-hook 'find-file-hook #'run-global)

(custom-set-variables
 '(citre-auto-enable-citre-mode-backends '(tags))
 '(citre-completion-backends '(tags))
 '(citre-find-definition-backends '(tags))
 '(citre-tags-in-buffer-backends '(tags)))

Emacs doesn't hang as well. The latter config uses the version on the citre master branch.

Yes, weird.

Anyway, before finding out the culprit, I'd like to remove all global backends. Thanks for the help.

[AmaiKinono]

I've added a 1-second timeout in citre-global-dbpath as a workaround (now on the master branch). This should make it possible for you to use the global backend. It will make opening an org file for the first time slower, but org-mode is itself slow to load.

[lambertlulala]

I've added a 1-second timeout in citre-global-dbpath as a workaround (now on the master branch). This should make it possible for you to use the global backend. It will make opening an org file for the first time slower, but org-mode is itself slow to load.

Thanks so much!

I just tested it several times. It seems Emacs haven't hung so far. The steps I tested:

1. eval-buffer the elisp file on https://[raw.githubusercontent.com/universal-ctags/citre/a900987221930e03975178d2b75b6dcd2366013d/citre-global.el](https://raw.githubusercontent.com/universal-ctags/citre/a900987221930e03975178d2b75b6dcd2366013d/citre-global.el)
2. Open any files.

After the version on the master branch published to melpa, I'll update then. Thank you so much! :grinning:

By the way, another issue I tested these 3 days is that when I repeat to use M-x citre-create-tags-file to create tag file (i.e. call it and wait it finished, generated the tag file and then call it again), the Emacs hangs as well. But I think this is an unusual scenario, most users may not use it like this. Instead, I think the normal use case would be:

1. Use M-x citre-create-tags-file to create the tag file.
2. If the user needs to update the tag file, use M-x citre-update-tags-file.
3. If something is wrong, delete the created tag file and then go to step 1 to create a new one.

[AmaiKinono]

In fact, citre-update-tags-file will create one if it doesn't find a tags file. That's the recommended way to do things so you only need one command. Maybe I should have call it citre-update-or-create-tags-file...

The logic of generating a tags file will change (see https://github.com/universal-ctags/citre/issues/162). Though I have now idea what causes Emacs to hang, the problem may disappear after then.

[lambertlulala]

I've updated the fixed version from melpa, it works great for me, thanks for your effort and maintenance again!

[lambertlulala]

Hello @AmaiKinono

I just want to share some findings about packages including citre on Windows. These days I've tested some Emacs packages on Windows platform with Anti-Virus software installed. I find Emacs or other programs get slower since those AV software may get their drivers filter some "suspicious" I/O operations. Of course, citre's actions are ones. Therefore, some feature's in citre may be inhibited by AV software and then users may report some weird issues, they may not perceive.

After I added those programs to the excluded program list, Emacs performed very well. Taking ctags for example, without AV software's intervention, it's faster almost 4 seconds on my machine, global and other programs also performs better.

What I want to say is, some issues reported by users may be caused by their AV software, if programming logic is correct, maybe the issue originates from some AV software. For instance, today I used citre during my working hours, creating tags file, however, failed and got Emacs hung. But the IT of our company installed AV software for every staffs. After I added Emacs and related tools including citre, global, to the excluded list, issues never occurs.

[AmaiKinono]

It is very kind of you to share your findings ;) I should add this as a tip in README.

Edit: I rethinked about it and found your explanation makes a lot of sense, as it answers why we need very specific conditions like

• opening org file for the first time, not after the first time, not on any other formats.
• reading global output using Citre, not only just run global.

to trigger the problem. If some ghost or fairy in your PC deliberately caused the hung, it can't be done inside Emacs.