Error messages with tag-follow-mode

minad commented 6 years ago

I have tag-follow-mode activated. If I am switching to a buffer without imenu support or with empty imenus, treemacs prints many error messages. I can deactivate those using treemacs--no-messages. Is this desired behavior or is possible to handle the "common errors" such that no error message is printed in those cases?

Alexander-Miller commented 6 years ago

Nope, that's not desired behaviour at all. Check whether https://github.com/Alexander-Miller/treemacs/commit/7b75a7db755b4b08fc66ef3b2a34fb2a49db9544 resolves this for you.

minad commented 6 years ago

Ok, that was quick! I will take a look.

minad commented 6 years ago

Unfortunately it does not seem to work for me. There are no errors anymore for text files, however some other prog-modes don't show the tags anymore (e.g. haskell). Imenu entries are available since they are shown in the lsp-ui-imenu sidebar, if I open that one instead.

Furthermore I got the unrelated issue that icons are shown with white background. Before I used the version from melpa which does not have that problem.

minad commented 6 years ago

Both in melpa and master, compare to lsp-ui-imenu treemacs behaves differently. Treemacs seems to trigger many rescans which makes everything feeling a bit more sluggish. Particularily annonying is the haskell mode which always prints a message (Sorting declarations...) when the imenu scanning takes place. I have to admit I turned down the delay, but I still wonder why rescanning takes place when I move the cursor?

Alexander-Miller commented 6 years ago

Now we're moving into elisp limitations and optimizations territory.

The follow mode is running on an idle timer, every time you move your cursor it is reset. So that's intended behaviour.
I need to make sure that movement is correct even when you just typed up or deleted a tag, so I need to force a rescan, even though imenu caches its results.
As an optimization I've removed all major mode hooks when fetching tags for a file that's not already open. That's why you get different behaviour than what lsp ui shows.

If everything is going as I think it should treemacs should still show the right lsp tags for open buffers, but fallback to the slow regex scan for files that are not open in emacs.

I suppose I'll need to do something like create a whitelist of modes whose mode hooks are allowed to run.

As for the icons I guess it depends on how you loaded your version, treemacs reads your theme's colors when it's first run. So long as a melpa install is working correctly we're in the clear.

minad commented 6 years ago

So you are saying that I don't get some of the entries anymore since you removed hooks in master compared to melpa? How can I activate those then, add-hook something to each mode which wants treemacs?

I didn't really get what you mean with closed vs open files. How does treemacs scans closed files?

Concerning rescanning, I agree that it is necessary to do that when something is edited. Basically my points are:

~~Would it be possible to avoid the delay when opening a file? Then I am happy with a long rescan delay.~~
Can rescanning be avoided for cursor movements when nothing is edited?

Edit: I got icon colors working when reordering as you suggested. After your fix I have to set imenu-generic-expression to some fake value, to get the imenus working again. Only lisp and c seemed to work after the fix, since you overwrite it for elisp and the c mode might do the same. However many modes don't set the generic expression, therefore I hope there is a better way to check for imenu availability?

 (setq imenu-generic-expression '(("fake" "fake" 0)))

Edit2: There is another unrelated problem with tag following. Haskell-mode with treemacs master works ok with this "fake expression" from before. However after a while errors start to occur and treemacs stops working

Error running timer ‘treemacs--follow-tag-at-point’: (wrong-type-argument integer-or-marker-p nil)

Alexander-Miller commented 6 years ago

I didn't really get what you mean with closed vs open files

I just meant files that are open in emacs. Of course on second thought that distinction does not really apply to your case. When you use tag-follow-mode you obviously open the files before tag-follow comes in. The difference would only be visible when you expand a file's tags with TAB. When that file is open in emacs imenu will have been set up to ask for tags from the language server. When the file is not open treemacs opens it anyway, does not run any mode hooks to prevent potentially slow setup code from running, so the tags it then gets are based on the builtin regex scan. That's the difference.

It's not always been like this, I'v received that part as a PR some months ago. Apparently without this optimization treemacs opening some type of files for their tags would lead to emacs trying to load them in a repl and cause errors.

So you are saying that I don't get some of the entries anymore since you removed hooks in master compared to melpa?

No, the comments on melpa were only referencing icons' colors. There's nothing you can do on your end, I'll have to patch this myself.

Would it be possible to avoid the delay when opening a file?

When opening, yes. That's just a simple find-file-hook. But there are too many different ways to select a file otherwise, some without providing a hook into them, like select-window. Best I can do is then add the same hook when you open a file via treemacs.

I got icon colors working when reordering as you suggested

To be clear: under what circumstances were your icon colors wrong. Did you pull and load master directly from github to try my fix (especially into an already running emacs instance) or does it also happen when you install from melpa?

Can rescanning be avoided for cursor movements when nothing is edited?

Perhaps, but it'd be non-trivial. I'd have to cache my results and keep track of the buffer being modified.

After your fix I have to set imenu-generic-expression to some fake value, to get the imenus working again

Shouldn't be needed anymore. With https://github.com/Alexander-Miller/treemacs/commit/40b341592f722147ecbb5f9f255c328a7db7387d the check should be done correctly if you use lsp, at least it works for me when I tried a simple python lsp setup.

However after a while errors start to occur and treemacs stops working

Assuming it's still wrong without the fake generic expression: how do you generate tags for haskell - laguage server or via haskell-mode?

minad commented 6 years ago

To be clear: under what circumstances were your icon colors wrong. Did you pull and load master directly from github to try my fix (especially into an already running emacs instance) or does it also happen when you install from melpa?

The version from melpa was loaded by use-package and worked fine with use-package in any order of use-package in the init.el.

For testing, I put the version from github on the load path. Then the icons were wrong. After moving the treemacs use-package to the end after the theme configuration, the icons where fine again.

Shouldn't be needed anymore. With 40b3415 the check should be done correctly if you use lsp, at least it works for me when I tried a simple python lsp setup.

Unfortunately it still does not work. It works for haskell-mode, lsp-modes, elisp and c mode. But the error messages come back for simple text files which don't have an imenu. I always tried these different modes for testing.

how do you generate tags for haskell - laguage server or via haskell-mode?

haskell-mode generates them itself I guess via regexes. I am also editing other files in the same emacs instance which rely on lsp-mode and other files which don't have an imenu. After treemacs tag following stops working, it stops for all the modes. It is not just not working for haskell mode but the whole system seems to be in a bad state. I am also using emacsclient, could that lead to problems?

Alexander-Miller commented 6 years ago

For testing, I put the version from github on the load path. Then the icons were wrong.

Are they still wrong if you load it like this, but also run (add-hook 'treemacs-mode-hook #'treemacs--setup-icon-background-colors)?

But the error messages come back for simple text files which don't have an imenu.

Looks like plain text buffers still have a default imenu function defined, even if they cannot use it. At this point we're better off just ignoring that error. https://github.com/Alexander-Miller/treemacs/commit/7d120ed5a30f9d450837b2959ba82d6098a9b130 take care of that.

After treemacs tag following stops working, it stops for all the modes.

There is no global state, just about everything is buffer local, so for a clean restart it should be enough to kill the treemacs buffer with Q. Next time run your tests with debug on error enabled, I won't be able to do much without a stack trace.

minad commented 6 years ago

Are they still wrong if you load it like this, but also run (add-hook 'treemacs-mode-hook #'treemacs--setup-icon-background-colors)?

I will test that.

Looks like plain text buffers still have a default imenu function defined, even if they cannot use it. At this point we're better off just ignoring that error. 7d120ed take care of that.

Yes, ignoring irrelevant errors is a good way to go :+1:

There is no global state, just about everything is buffer local, so for a clean restart it should be enough to kill the treemacs buffer with Q. Next time run your tests with debug on error enabled, I won't be able to do much without a stack trace.

Ok, I will try. While I am using emacs for quite a while, I am still kind of an elisp noob without much experience with the debugger...

Alexander-Miller commented 6 years ago

While I am using emacs for quite a while, I am still kind of an elisp noob without much experience with the debugger

That won't be really debugging. All debug-on-error does is make emacs barf up a stack trace when it encounters an error.

minad commented 6 years ago

Are they still wrong if you load it like this, but also run (add-hook 'treemacs-mode-hook #'treemacs--setup-icon-background-colors)?

Does not work. The colors are only good if the use-package treemacs happens after the use-package mytheme.

Looks like plain text buffers still have a default imenu function defined, even if they cannot use it. At this point we're better off just ignoring that error. 7d120ed take care of that.

Seems to work now! Thank you!

There is no global state, just about everything is buffer local, so for a clean restart it should be enough to kill the treemacs buffer with Q. Next time run your tests with debug on error enabled, I won't be able to do much without a stack trace.

checking that one now...

minad commented 6 years ago

The error didn't occur for a while. Maybe it is resolved now due and was coupled to the other issue afterall. I will report back if it might still occur during my regular work flow. Shall we still keep this issue open for a short while or so?

minad commented 6 years ago

A small note about avoiding the previously discussed messages during rescanning in haskell mode - I am using advice to suppress them.

(defun suppress-messages-advice (&rest args) (ignore))

(defun suppress-messages (old-fun &rest args)
    (advice-add 'message :around #'suppress-messages-advice)
    (unwind-protect
        (apply old-fun args)
      (advice-remove 'message #'suppress-messages-advice)))

(defun suppress-haskell-messages ()
  (advice-add 'haskell-ds-create-imenu-index :around #'suppress-messages))

(add-hook 'haskell-mode-hook 'suppress-haskell-messages)

With that it work really great :+1: I should also say thank you for this great emacs package!

minad commented 6 years ago

error occurred again in haskell mode

Debugger entered--Lisp error: (wrong-type-argument integer-or-marker-p nil)
  get-text-property(nil :path)
  button-get(nil :path)
  (string= (button-get (treemacs-current-button) :path) (button-get treemacs--previously-followed-tag-btn :path))
  (and (string= (button-get (treemacs-current-button) :path) (button-get treemacs--previously-followed-tag-btn :path)) (eq (quote file-node-open) (button-get treemacs--previously-followed-tag-btn :state)))
  (if (and (string= (button-get (treemacs-current-button) :path) (button-get treemacs--previously-followed-tag-btn :path)) (eq (quote file-node-open) (button-get treemacs--previously-followed-tag-btn :state))) (progn (treemacs--collapse-file-node treemacs--previously-followed-tag-btn)))
  (save-excursion (goto-char treemacs--previously-followed-tag-btn) (if (and (string= (button-get (treemacs-current-button) :path) (button-get treemacs--previously-followed-tag-btn :path)) (eq (quote file-node-open) (button-get treemacs--previously-followed-tag-btn :state))) (progn (treemacs--collapse-file-node treemacs--previously-followed-tag-btn))))
  (progn (save-excursion (goto-char treemacs--previously-followed-tag-btn) (if (and (string= (button-get (treemacs-current-button) :path) (button-get treemacs--previously-followed-tag-btn :path)) (eq (quote file-node-open) (button-get treemacs--previously-followed-tag-btn :state))) (progn (treemacs--collapse-file-node treemacs--previously-followed-tag-btn)))))
  (if (and treemacs--previously-followed-tag-btn (not (eq treemacs--previously-followed-tag-btn btn))) (progn (save-excursion (goto-char treemacs--previously-followed-tag-btn) (if (and (string= (button-get (treemacs-current-button) :path) (button-get treemacs--previously-followed-tag-btn :path)) (eq (quote file-node-open) (button-get treemacs--previously-followed-tag-btn :state))) (progn (treemacs--collapse-file-node treemacs--previously-followed-tag-btn))))))
  (progn (if (memq (button-get btn :state) (quote (tag-node-open tag-node-closed tag-node))) (progn (while (not (memq (button-get btn :state) file-states)) (setq btn (button-get btn :parent))))) (if (and treemacs--previously-followed-tag-btn (not (eq treemacs--previously-followed-tag-btn btn))) (progn (save-excursion (goto-char treemacs--previously-followed-tag-btn) (if (and (string= (button-get ... :path) (button-get treemacs--previously-followed-tag-btn :path)) (eq (quote file-node-open) (button-get treemacs--previously-followed-tag-btn :state))) (progn (treemacs--collapse-file-node treemacs--previously-followed-tag-btn)))))) (if (string-equal buffer-file (button-get btn :path)) nil (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button))))
  (if btn (progn (if (memq (button-get btn :state) (quote (tag-node-open tag-node-closed tag-node))) (progn (while (not (memq (button-get btn :state) file-states)) (setq btn (button-get btn :parent))))) (if (and treemacs--previously-followed-tag-btn (not (eq treemacs--previously-followed-tag-btn btn))) (progn (save-excursion (goto-char treemacs--previously-followed-tag-btn) (if (and (string= ... ...) (eq ... ...)) (progn (treemacs--collapse-file-node treemacs--previously-followed-tag-btn)))))) (if (string-equal buffer-file (button-get btn :path)) nil (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button)))) (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button)))
  (progn (select-window (car save-selected-window--state) (quote norecord)) (setq btn (treemacs-current-button)) (if btn (progn (if (memq (button-get btn :state) (quote (tag-node-open tag-node-closed tag-node))) (progn (while (not (memq ... file-states)) (setq btn (button-get btn :parent))))) (if (and treemacs--previously-followed-tag-btn (not (eq treemacs--previously-followed-tag-btn btn))) (progn (save-excursion (goto-char treemacs--previously-followed-tag-btn) (if (and ... ...) (progn ...))))) (if (string-equal buffer-file (button-get btn :path)) nil (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button)))) (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button))) (goto-char (button-start btn)) (setq treemacs--previously-followed-tag-btn btn) (let ((imenu-auto-rescan nil)) (setcar tag-path (car (car tag-path))) (setcdr tag-path (cons buffer-file (cdr tag-path))) (treemacs--goto-tag-button-at tag-path)) (hl-line-highlight) (treemacs--evade-image) (if treemacs-recenter-after-tag-follow (progn (treemacs--maybe-recenter))))
  (unwind-protect (progn (select-window (car save-selected-window--state) (quote norecord)) (setq btn (treemacs-current-button)) (if btn (progn (if (memq (button-get btn :state) (quote (tag-node-open tag-node-closed tag-node))) (progn (while (not ...) (setq btn ...)))) (if (and treemacs--previously-followed-tag-btn (not (eq treemacs--previously-followed-tag-btn btn))) (progn (save-excursion (goto-char treemacs--previously-followed-tag-btn) (if ... ...)))) (if (string-equal buffer-file (button-get btn :path)) nil (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button)))) (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button))) (goto-char (button-start btn)) (setq treemacs--previously-followed-tag-btn btn) (let ((imenu-auto-rescan nil)) (setcar tag-path (car (car tag-path))) (setcdr tag-path (cons buffer-file (cdr tag-path))) (treemacs--goto-tag-button-at tag-path)) (hl-line-highlight) (treemacs--evade-image) (if treemacs-recenter-after-tag-follow (progn (treemacs--maybe-recenter)))) (internal--after-with-selected-window save-selected-window--state))
  (save-current-buffer (unwind-protect (progn (select-window (car save-selected-window--state) (quote norecord)) (setq btn (treemacs-current-button)) (if btn (progn (if (memq (button-get btn :state) (quote ...)) (progn (while ... ...))) (if (and treemacs--previously-followed-tag-btn (not ...)) (progn (save-excursion ... ...))) (if (string-equal buffer-file (button-get btn :path)) nil (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button)))) (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button))) (goto-char (button-start btn)) (setq treemacs--previously-followed-tag-btn btn) (let ((imenu-auto-rescan nil)) (setcar tag-path (car (car tag-path))) (setcdr tag-path (cons buffer-file (cdr tag-path))) (treemacs--goto-tag-button-at tag-path)) (hl-line-highlight) (treemacs--evade-image) (if treemacs-recenter-after-tag-follow (progn (treemacs--maybe-recenter)))) (internal--after-with-selected-window save-selected-window--state)))
  (let ((save-selected-window--state (internal--before-with-selected-window treemacs-window))) (save-current-buffer (unwind-protect (progn (select-window (car save-selected-window--state) (quote norecord)) (setq btn (treemacs-current-button)) (if btn (progn (if (memq ... ...) (progn ...)) (if (and treemacs--previously-followed-tag-btn ...) (progn ...)) (if (string-equal buffer-file ...) nil (treemacs-goto-button buffer-file project) (setq btn ...))) (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button))) (goto-char (button-start btn)) (setq treemacs--previously-followed-tag-btn btn) (let ((imenu-auto-rescan nil)) (setcar tag-path (car (car tag-path))) (setcdr tag-path (cons buffer-file (cdr tag-path))) (treemacs--goto-tag-button-at tag-path)) (hl-line-highlight) (treemacs--evade-image) (if treemacs-recenter-after-tag-follow (progn (treemacs--maybe-recenter)))) (internal--after-with-selected-window save-selected-window--state))))
  (progn (let ((save-selected-window--state (internal--before-with-selected-window treemacs-window))) (save-current-buffer (unwind-protect (progn (select-window (car save-selected-window--state) (quote norecord)) (setq btn (treemacs-current-button)) (if btn (progn (if ... ...) (if ... ...) (if ... nil ... ...)) (treemacs-goto-button buffer-file project) (setq btn (treemacs-current-button))) (goto-char (button-start btn)) (setq treemacs--previously-followed-tag-btn btn) (let ((imenu-auto-rescan nil)) (setcar tag-path (car ...)) (setcdr tag-path (cons buffer-file ...)) (treemacs--goto-tag-button-at tag-path)) (hl-line-highlight) (treemacs--evade-image) (if treemacs-recenter-after-tag-follow (progn (treemacs--maybe-recenter)))) (internal--after-with-selected-window save-selected-window--state)))))
  (unwind-protect (progn (let ((save-selected-window--state (internal--before-with-selected-window treemacs-window))) (save-current-buffer (unwind-protect (progn (select-window (car save-selected-window--state) (quote norecord)) (setq btn (treemacs-current-button)) (if btn (progn ... ... ...) (treemacs-goto-button buffer-file project) (setq btn ...)) (goto-char (button-start btn)) (setq treemacs--previously-followed-tag-btn btn) (let (...) (setcar tag-path ...) (setcdr tag-path ...) (treemacs--goto-tag-button-at tag-path)) (hl-line-highlight) (treemacs--evade-image) (if treemacs-recenter-after-tag-follow (progn ...))) (internal--after-with-selected-window save-selected-window--state))))) (with-no-warnings (setq treemacs--ready-to-follow o)))
  (let ((o (with-no-warnings treemacs--ready-to-follow))) (with-no-warnings (setq treemacs--ready-to-follow nil)) (unwind-protect (progn (let ((save-selected-window--state (internal--before-with-selected-window treemacs-window))) (save-current-buffer (unwind-protect (progn (select-window ... ...) (setq btn ...) (if btn ... ... ...) (goto-char ...) (setq treemacs--previously-followed-tag-btn btn) (let ... ... ... ...) (hl-line-highlight) (treemacs--evade-image) (if treemacs-recenter-after-tag-follow ...)) (internal--after-with-selected-window save-selected-window--state))))) (with-no-warnings (setq treemacs--ready-to-follow o))))
  (progn (let ((o (with-no-warnings treemacs--ready-to-follow))) (with-no-warnings (setq treemacs--ready-to-follow nil)) (unwind-protect (progn (let ((save-selected-window--state (internal--before-with-selected-window treemacs-window))) (save-current-buffer (unwind-protect (progn ... ... ... ... ... ... ... ... ...) (internal--after-with-selected-window save-selected-window--state))))) (with-no-warnings (setq treemacs--ready-to-follow o)))))
  (if tag-path (progn (let ((o (with-no-warnings treemacs--ready-to-follow))) (with-no-warnings (setq treemacs--ready-to-follow nil)) (unwind-protect (progn (let ((save-selected-window--state ...)) (save-current-buffer (unwind-protect ... ...)))) (with-no-warnings (setq treemacs--ready-to-follow o))))))
  (let* ((tag-path (treemacs--find-index-pos (point) flat-index)) (file-states (quote (file-node-open file-node-closed root-node-open root-node-closed))) (btn)) (if tag-path (progn (let ((o (with-no-warnings treemacs--ready-to-follow))) (with-no-warnings (setq treemacs--ready-to-follow nil)) (unwind-protect (progn (let (...) (save-current-buffer ...))) (with-no-warnings (setq treemacs--ready-to-follow o)))))))
  treemacs--do-follow-tag(((("CCommon.Location" . #<marker (moves after insertion) at 41 in Repl.hs>) "Imports") (("CCommon.Name" . #<marker (moves after insertion) at 70 in Repl.hs>) "Imports") (("CEmit.Main" . #<marker (moves after insertion) at 95 in Repl.hs>) "Imports") (("CIntro" . #<marker (moves after insertion) at 118 in Repl.hs>) "Imports") (("CParser.Main" . #<marker (moves after insertion) at 137 in Repl.hs>) "Imports") (("CPipeline.Loader" . #<marker (moves after insertion) at 162 in Repl.hs>) "Imports") (("CPipeline.Monad" . #<marker (moves after insertion) at 191 in Repl.hs>) "Imports") (("CPipeline.Options" . #<marker (moves after insertion) at 219 in Repl.hs>) "Imports") (("CPipeline.TypedModule" . #<marker (moves after insertion) at 249 in Repl.hs>) "Imports") (("CPretty.Exp" . #<marker (moves after insertion) at 283 in Repl.hs>) "Imports") (("CPretty.Message" . #<marker (moves after insertion) at 307 in Repl.hs>) "Imports") (("CPretty.Options" . #<marker (moves after insertion) at 335 in Repl.hs>) "Imports") (("CSugar.Syntax" . #<marker (moves after insertion) at 363 in Repl.hs>) "Imports") (("Data.IORef" . #<marker (moves after insertion) at 389 in Repl.hs>) "Imports") (("Data.Text.IO (getLine)" . #<marker (moves after insertion) at 407 in Repl.hs>) "Imports") (("System.Environment" . #<marker (moves after insertion) at 437 in Repl.hs>) "Imports") (("System.FilePath" . #<marker (moves after insertion) at 463 in Repl.hs>) "Imports") (("System.IO (stderr, Handle, hPutStrLn, hFlush, hClose)" . #<marker (moves after insertion) at 486 in Repl.hs>) "Imports") (("System.Process" . #<marker (moves after insertion) at 547 in Repl.hs>) "Imports") (("Text.PariPari hiding (getLine)" . #<marker (moves after insertion) at 569 in Repl.hs>) "Imports") (("CCore.Core as Core" . #<marker (moves after insertion) at 607 in Repl.hs>) "Imports") (("CCore.Utils as Core" . #<marker (moves after insertion) at 648 in Repl.hs>) "Imports") (("Data.ByteString as B" . #<marker (moves after insertion) at 690 in Repl.hs>) "Imports") (("Data.ByteString.Builder as B" . #<marker (moves after insertion) at 728 in Repl.hs>) "Imports") (("Data.ByteString.Builder.Extra as B" . #<marker (moves after insertion) at 774 in Repl.hs>) "Imports") (("Data.Map as M" . #<marker (moves after insertion) at 826 in Repl.hs>) "Imports") (("Data.Text as T" . #<marker (moves after insertion) at 857 in Repl.hs>) "Imports") (("Data.Text.IO as T" . #<marker (moves after insertion) at 889 in Repl.hs>) "Imports") (("ReplEnv" . #<marker (moves after insertion) at 925 in Repl.hs>) "Datatypes") (("makeLenses" . #<marker (moves after insertion) at 1271 in Repl.hs>) "Functions") (("ReplHandler" . #<marker (moves after insertion) at 1293 in Repl.hs>) "Datatypes") (("PipelineHandler (ReplHandler m)" . #<marker (moves after insertion) at 1453 in Repl.hs>) "Instances") (("CacheHandler (ReplHandler m)" . #<marker (moves after insertion) at 1745 in Repl.hs>) "Instances") (("removeFromCache" . #<marker (moves after insertion) at 2021 in Repl.hs>) "Functions") (("readInput" . #<marker (moves after insertion) at 2187 in Repl.hs>) "Functions") (("formatImport" . #<marker (moves after insertion) at 2370 in Repl.hs>) "Functions") (("compileModule" . #<marker (moves after insertion) at 2582 in Repl.hs>) "Functions") (("processImport" . #<marker (moves after insertion) at 2750 in Repl.hs>) "Functions") (("processDecl" . #<marker (moves after insertion) at 3366 in Repl.hs>) "Functions") (("writeModule" . #<marker (moves after insertion) at 4210 in Repl.hs>) "Functions") (("processExp" . #<marker (moves after insertion) at 4453 in Repl.hs>) "Functions") (("cmdUnload" . #<marker (moves after insertion) at 5374 in Repl.hs>) "Functions") (("cmdHelp" . #<marker (moves after insertion) at 6032 in Repl.hs>) "Functions") (("cmdRestart" . #<marker (moves after insertion) at 6233 in Repl.hs>) "Functions") (("cmdType" . #<marker (moves after insertion) at 6422 in Repl.hs>) "Functions") (("cmdInfo" . #<marker (moves after insertion) at 6520 in Repl.hs>) "Functions") (("cmdUndef" . #<marker (moves after insertion) at 6618 in Repl.hs>) "Functions") (("cmdShowBindings" . #<marker (moves after insertion) at 6719 in Repl.hs>) "Functions") (("cmdShowImports" . #<marker (moves after insertion) at 6913 in Repl.hs>) "Functions") (("cmdExit" . #<marker (moves after insertion) at 7219 in Repl.hs>) "Functions") ...) #<window 6 on  *Treemacs-Framebuffer-1*> "/home/user/c/compiler/C/Command/Repl.hs" [cl-struct-treemacs-project "c" "/home/user/c"])
  (progn (treemacs--do-follow-tag index treemacs-window buffer-file project))
  (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project)))
  (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project))))
  (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project)))))
  (if (and treemacs-window buffer-file project) (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project))))))
  (let* ((treemacs-window (treemacs-get-local-window)) (buffer (current-buffer)) (buffer-file (if buffer (progn (buffer-file-name)))) (project (treemacs--find-project-for-buffer))) (if (and treemacs-window buffer-file project) (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project)))))))
  treemacs--follow-tag-at-point()
  apply(treemacs--follow-tag-at-point nil)
  timer-event-handler([t 0 0 100000 t treemacs--follow-tag-at-point nil idle 0])

minad commented 6 years ago

It happend during heavy editing I think.

Alexander-Miller commented 6 years ago

Check if https://github.com/Alexander-Miller/treemacs/commit/dbe670f78c1378fa7737ab731e414b7c354343ff takes care of that. I couldn't reproduce the issue on my end, so it might be just a symptom, not the cause.

minad commented 6 years ago

there is another one

https://gist.github.com/minad/16e929aa4ee5103dcbd5c42487eb4d62

Alexander-Miller commented 6 years ago

"/home/user/c/lib/*C Repl [~/c/lib"

Is that right. Does part of your path look like that?

minad commented 6 years ago

sorry, thats slightly obfuscated for confidentiality reasons

Alexander-Miller commented 6 years ago

Ok, can you put in an XYZ or something like that in there? I must be able to properly read the paths to figure this one out.

Just the stack traces are insufficient, so I've uploaded a debug branch for you to try, it'll spit out plenty of log info, so let's try that one next.

minad commented 6 years ago

Hmm, ok I understand that you need the paths for a tree viewer. I was hoping that the trace would do. But just to make it clear - the error discussed now happens much more rarely than the other one. So your first symptom fix definitely helped! I will try to debug it using the other branch.

One other thing which came to my mind - as discussed, treemacs triggers a lot of rescanning for cursor movement (this is flooding the lsp server with many textDocument/documentSymbol messages). In contrast to that the lsp-mode sends textDocument/didChange when the document really changed, maybe we can just copy the logic from there to avoid the superfluous rescans?

Alexander-Miller commented 6 years ago

Getting into lsp details should not be necessary (and would require putting things into its own treemacs-lsp package). It'd be nice if the lsp had a get symbol at point feature, but after some research I should be able to implement some simple caching on my own with just the hooks emacs offers.

minad commented 6 years ago

I think caching is not even necessary. It would be sufficient just to distinguish cursor movements from edit operations, since edit operations will in every case send change notifications to the language server. From that point on it is the responsibility of the language server to do the caching etc imho.

In principle the language servers should also handle those unnecessary documentSymbol requests well if no change occurred. However it would still be better if the frontend does its best not to produce unnecessary work for the underlying language server.

Alexander-Miller commented 6 years ago

I'm not dealing with a language server. Treemacs just asks imenu, and imenu can get its data from various sources - a language server, a regex scan, or something like ggtags. Filetypes like elisp, org or markdown, which are always scanned with regex, will definitely benefit from caching.

minad commented 6 years ago

This happens in adoc-mode:

Debugger entered--Lisp error: (wrong-type-argument markerp 441)
  marker-position(441)
  (< (marker-position (cdr (car p1))) (marker-position (cdr (car p2))))
  treemacs--compare-tag-paths(((#("Important links" 0 15 (fontified t face markup-title-1-face)) . 441) "Functions") ((#("Installation" 0 12 (fontified t face markup-title-1-face)) . 549) "Functions"))
  sort((((#("Debugging and profiling tools" 0 29 (fontified nil)) . 1533) "Functions")) treemacs--compare-tag-paths)
  (let* ((imenu-auto-rescan t) (org\? (eq major-mode (quote org-mode))) (index (treemacs--get-imenu-index (buffer-file-name))) (flat-index (if org\? (treemacs--flatten-org-mode-imenu-index index) (treemacs--flatten-imenu-index index))) (first (car (car flat-index))) (semantic\? (and (consp first) (overlayp (cdr first))))) (cond (semantic\? (let ((--dolist-tail-- flat-index)) (while --dolist-tail-- (let ((tag-path ...)) (let (... ...) (setcdr leaf ...)) (setq --dolist-tail-- (cdr --dolist-tail--)))))) (org\? (let ((--dolist-tail-- flat-index)) (while --dolist-tail-- (let ((tag-path ...)) (let (...) (if ... ...)) (setq --dolist-tail-- (cdr --dolist-tail--))))))) (sort flat-index (function treemacs--compare-tag-paths)))
  treemacs--flatten&sort-imenu-index()
  (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project))))
  (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project)))) (imenu-unavailable (ignore e)))
  (if (and treemacs-window buffer-file project) (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project)))) (imenu-unavailable (ignore e))))
  (let* ((treemacs-window (treemacs-get-local-window)) (buffer (current-buffer)) (buffer-file (if buffer (progn (buffer-file-name)))) (project (treemacs--find-project-for-buffer))) (if (and treemacs-window buffer-file project) (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project)))) (imenu-unavailable (ignore e)))))
  treemacs--follow-tag-at-point()
  apply(treemacs--follow-tag-at-point nil)
  timer-event-handler([t 0 0 500000 t treemacs--follow-tag-at-point nil idle 0])

minad commented 6 years ago

In c-mode:

Debugger entered--Lisp error: (wrong-type-argument arrayp nil)
  aref(nil 2)
  treemacs-workspace->projects(nil)
  (let ((list (treemacs-workspace->projects (treemacs-current-workspace))) (continue t) (it-index 0)) (while (and list continue) (let ((it (car list))) (if (not (not needle)) (setq continue nil) (if (treemacs--is-path-in-dir\? (buffer-file-name) (treemacs-project->path it)) (progn (setq needle it))))) (setq it-index (1+ it-index)) (setq list (cdr list))))
  (let (needle) (let ((list (treemacs-workspace->projects (treemacs-current-workspace))) (continue t) (it-index 0)) (while (and list continue) (let ((it (car list))) (if (not (not needle)) (setq continue nil) (if (treemacs--is-path-in-dir\? (buffer-file-name) (treemacs-project->path it)) (progn (setq needle it))))) (setq it-index (1+ it-index)) (setq list (cdr list)))) needle)
  (setq treemacs--project-of-buffer (let (needle) (let ((list (treemacs-workspace->projects (treemacs-current-workspace))) (continue t) (it-index 0)) (while (and list continue) (let ((it (car list))) (if (not (not needle)) (setq continue nil) (if (treemacs--is-path-in-dir\? ... ...) (progn ...)))) (setq it-index (1+ it-index)) (setq list (cdr list)))) needle))
  (progn (setq treemacs--project-of-buffer (let (needle) (let ((list (treemacs-workspace->projects (treemacs-current-workspace))) (continue t) (it-index 0)) (while (and list continue) (let ((it ...)) (if (not ...) (setq continue nil) (if ... ...))) (setq it-index (1+ it-index)) (setq list (cdr list)))) needle)))
  (if (buffer-file-name) (progn (setq treemacs--project-of-buffer (let (needle) (let ((list (treemacs-workspace->projects ...)) (continue t) (it-index 0)) (while (and list continue) (let (...) (if ... ... ...)) (setq it-index (1+ it-index)) (setq list (cdr list)))) needle))))
  (if treemacs--project-of-buffer nil (if (buffer-file-name) (progn (setq treemacs--project-of-buffer (let (needle) (let ((list ...) (continue t) (it-index 0)) (while (and list continue) (let ... ...) (setq it-index ...) (setq list ...))) needle)))))
  treemacs--find-project-for-buffer()
  (let* ((treemacs-window (treemacs-get-local-window)) (buffer (current-buffer)) (buffer-file (if buffer (progn (buffer-file-name)))) (project (treemacs--find-project-for-buffer))) (if (and treemacs-window buffer-file project) (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project)))) (imenu-unavailable (ignore e)))))
  treemacs--follow-tag-at-point()
  apply(treemacs--follow-tag-at-point nil)
  timer-event-handler([t 0 0 500000 t treemacs--follow-tag-at-point nil idle 0])

minad commented 6 years ago

perl-mode:

Debugger entered--Lisp error: (wrong-type-argument markerp nil)
  marker-position(nil)
  (< (marker-position (cdr (car p1))) (marker-position (cdr (car p2))))
  treemacs--compare-tag-paths((("$desc" . #<marker at 12513 in generate.pl>) "Functions" "Variables") (("Packages") "Functions"))
  sort(((("ctfField" . #<marker at 3236 in generate.pl>) "Functions")) treemacs--compare-tag-paths)
  (let* ((imenu-auto-rescan t) (org\? (eq major-mode (quote org-mode))) (index (treemacs--get-imenu-index (buffer-file-name))) (flat-index (if org\? (treemacs--flatten-org-mode-imenu-index index) (treemacs--flatten-imenu-index index))) (first (car (car flat-index))) (semantic\? (and (consp first) (overlayp (cdr first))))) (cond (semantic\? (let ((--dolist-tail-- flat-index)) (while --dolist-tail-- (let ((tag-path ...)) (let (... ...) (setcdr leaf ...)) (setq --dolist-tail-- (cdr --dolist-tail--)))))) (org\? (let ((--dolist-tail-- flat-index)) (while --dolist-tail-- (let ((tag-path ...)) (let (...) (if ... ...)) (setq --dolist-tail-- (cdr --dolist-tail--))))))) (sort flat-index (function treemacs--compare-tag-paths)))
  treemacs--flatten&sort-imenu-index()
  (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project))))
  (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project)))))
  (if (and treemacs-window buffer-file project) (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project))))))
  (let* ((treemacs-window (treemacs-get-local-window)) (buffer (current-buffer)) (buffer-file (if buffer (progn (buffer-file-name)))) (project (treemacs--find-project-for-buffer))) (if (and treemacs-window buffer-file project) (progn (let ((index (treemacs--flatten&sort-imenu-index))) (if index (progn (treemacs--do-follow-tag index treemacs-window buffer-file project)))))))
  treemacs--follow-tag-at-point()
  apply(treemacs--follow-tag-at-point nil)
  timer-event-handler([t 0 0 500000 t treemacs--follow-tag-at-point nil idle 0])

Alexander-Miller commented 6 years ago

Oh boy, just when I had a major breakthrough with my work on extensions you deliver these bomb shells.

First one looks like like an unexpected representation of tag positions, similar to markdown, so I guess I'll need to special-case this.

Second one looks like the current-workspace call returns nil. Shouldn't really happen, the current workspace is saved is a frame parameter. Where multiple frames involved here?

With the third one there's something wrong when comparison of tag positions.

I've tried some basic files for C and perl and it worked as expected on my end, so I'll need a recipe for these two, some small example file with an example project structure where these errors can be reproduced.

minad commented 6 years ago

Sorry :(

I have no idea about the internals, but is it not possible to check for the exact format that you require by some kind of pattern matching and ignoring all other events? Maybe printing a warning if an unexpected sexp appears?

Frame is the x11 window, right? It could be. Flycheck or the compilation buffer did open new frames and I might have confused things here. I also want to configure emacs to never open new frames, but I didn't find out yet how to do that.

Do you need some files to reproduce? For me these errors occur pretty often for when browsing multiple files and switching between filetypes. I don't even have to edit them, the errors occur when I move the cursor around. See for example the perl file https://gist.github.com/minad/ffc473c1b4a3eddd1c797d66c183e7fb

Which emacs version are you using? It happens on 25.2 and 26.1 for me.

Alexander-Miller commented 6 years ago

As expected adoc-mode is returning position information as numbers instead of as markers. That one should be easy enough to fix.

Frame is the x11 window, right?

Yes. Specifically the kind of window that pops up when you press C-x 5 2. It's very strange that flycheck or a compilation would pop up a whole frame. By default they should open a new window in your current emacs frame. You'd definitely have to go out of your way to have them pop-up frames by customizing display-buffer-alist, either directly, or with a package like shackle.

Even so, if tag-follow-mode activates it means you must have a treemacs window visible in the current frame, which means there must be a workspace.

is it not possible to check for the exact format that you require by some kind of pattern matching and ignoring all other events?

Imenu's format is not that strange it's just nested lists for a hierarchy with leaf elements having their positions attacked. You can see what that looks like by evaluating (treemacs--get-imenu-index (buffer-file-name)) in any buffer (evaluating elisp works regardless of major mode).

By all rights that format should be deterministic. What I then do is flatten the hierarchy so I can use it to navigate, sort it by position in your file, and then find the nearest tag to the position of your cursor with a simple binary search. Validation happens somewhat implicitly since I need to recursively navigate the given tag index based on some of imenu's builtin methods to flatten it.

Now here's what my thought process for the whole thing looks like. Take the error on perl:

(< (marker-position (cdr (car p1))) (marker-position (cdr (car p2)))) treemacs--compare-tag-paths((("$desc" . #<marker at 12513 in generate.pl>) "Functions" "Variables") (("Packages") "Functions"))

It is comparing the position of 2 tag paths. You can read (("$desc" . #<marker at 12513 in generate.pl>) "Functions" "Variables") that a semantic hierarchy that looks like (Functions (Variables ($desc . position))) in your file (imenu has a habit of throwing all top level elements into a section called Functions).

The second tag in this comparison has no position marker anywhere. It looks like some part is missing. This could be an error in my flattening process, but then it should always happen, especially without a follow-mode when just calling treemacs-find-tag.

But from what you said it seems to appear sporadically, when switching files, but not always. Or is is consistent? Because if it isn't we're looking at a bit of a debugging nightmare. Since I cannot reproduce the errors - I tried your perl file and switched between files in emacs 26 and the jit branch of emacs 27 - it looks like all that's left is for you to start with emacs -Q and then keep adding chunks of your config until we hopefully find something.

Alexander-Miller commented 6 years ago

Looks like all my rambling wasn't necessary after all, even if it did help me clear my head on this.

I've compiled emacs 25 now where I can reproduce the errors you're having. I'll start looking into that now.

Alexander-Miller commented 6 years ago

https://i.imgur.com/eOJOdp8.png

Left is emacs 26, right is emacs 25 - quite the difference. I don't know any perl, so I cannot judge which one is more accurate.

Either way you can see on the right that "Packages" is an empty list. My post processing probably sees it as a leaf node that should have a position attached. I should be able to filter cases like that one.

minad commented 6 years ago

Cool, lucky me. Debugging the config is horrible. This is a bit of the downside of emacs and I had that experience a few times before. You start adding stuff, more features and everything works nicely and at some point things start to fall apart due to bad interactions of the various packages. For sometime I tried spacemacs, but it just felt too bloated and too heavly modified for me. Then I used a mostly vanilla emacs for a while but thats not how I think it is supposed to be. Why have all this flexibility after all ;)

Concerning the frames, there are some functions open-other-frame etc, those might be triggered by some keyboard combinations. I try to configure it that no such functions are called for example in dired.
The errors appear both in 25.2 and 26.1. I am looking forward to this jit branch. Is there a noticeable difference and do you know if it will land in 27?
Concerning checking the imenu sexps. You said you are checking implicitly. I am more from the strongly typed land, so I think it makes some sense to write some contract checking stuff - this will pay of later hopefully. But I didn't dig into your code - so no offense!

minad commented 6 years ago

Hmm, I don't see a difference in the imenus for 25 and 26 if opening with lsp-ui imenu. I cannot tell why you get those different pictures.

Alexander-Miller commented 6 years ago

That's because you're using a language server. The lsp doesn't care about your emacs version. The differences I see are for the builtin regex scan variant.

minad commented 6 years ago

No, I am not using an lsp for perl. The lsp-ui imenu is just a gui element, which queries imenu similar to what you do. I am using lsp only for one custom mode.

Alexander-Miller commented 6 years ago

Strange. What about the output of (imenu--make-index-alist t)? That's what treemacs calls internally to fetch the tags.

minad commented 6 years ago

I checked with list-processes, there are no lsp servers besides the one I am expecting. All the old languages are imenued by their respective modes via regexes. imenu--make-index-alist gives the same result for me on 25.2 and 26.1. I don't think something was changed in the perl mode between those versions, given how deep frozen perl5 is.

Alexander-Miller commented 6 years ago

More patches pushed. The issues in adoc-mode, markdown-mode and perl should now be fixed. Let me know what's left.

minad commented 6 years ago

Great! Thank you! I quickly tried it and it worked perfectly. :tada:

I will let you know if there are further issues! But I think if a few different major modes work well it is fine in general!

I created #288 (which was discussed here) but should probably get its own issue. I hope you agree!

minad commented 6 years ago

There is still an error occurring, which happens in particular if I open a file directly via the command line when starting emacs.

[Treemacs] Encountered error while following tag at point: (end-of-buffer)

Unfortunately I cannot get a stacktrace this time. I tried to set debug-on-error and debug-on-signal. If I remove the condition-case in treemacs--follow-tag-at-point I get an error while running timer: (end-of-buffer).

Have you observed such an issue too?

Alexander-Miller commented 6 years ago

You can try calling treemacs-find-tag directly.

minad commented 6 years ago

I debugged a bit and the error occurs in (imenu--make-index-alist t). If I evaluate it using M-x it prints "End of buffer". What does that mean and what can treemacs do about that? Is it an internal imenu error?

minad commented 6 years ago

Hmm ok, treemacs-find-tag and imenu both throw this error. So this is not a treemacs error. It looks like I have some issues with the language server. Thank you for your help!

mcraveiro commented 4 years ago

Hi guys,

I also see a lot of errors such as these, for example at present I'm getting:

[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp configuration (Method))
[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp end_transform (Method))
[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp end_chain (Method))
LSP :: Connected to [clangd:21917].
[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp (Namespace))
[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp invalid_tracing_format (Variable))
[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp validate (Method))
[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp invalid_tracing_format (Variable))
[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp unexpected_empty (Variable))
[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp using namespace dogen::utility::log (Variable)) [4 times]
[Treemacs] Encountered error while following tag at point: (wrong-type-argument listp get_logging_impact (Function))
Mark set [2 times]

I'm running latest treemacs from elpa I believe (e.g. [Treemacs] v2.6-2019.11.07 @ 26.1). The errors are occurring on a c++ buffer with LSP mode, using clangd as the LSP server. I have follow mode enabled. I wonder if the fact that I am using C++ 17 has something to do with it. Interestingly, when I set toggle-debug-on-error, these errors do not seem to trigger the debugger...

Alexander-Miller commented 4 years ago

I recently changed quite a bit about tags, so I doubt it's C++17 at work. You should post your symptoms to https://github.com/Alexander-Miller/treemacs/issues/562

Alexander-Miller / treemacs

Error messages with tag-follow-mode #283