highlight-indent-guides-mode + racket-hash-lang-mode = "No hash-lang exists with ID <x>" errors

[sorawee]

Consider:

;; abc

#lang racket

1

This produces a message

racket-back-end-/ command exception:
hash-lang-bridge: No hash-lang exists with ID <x>

over and over again.

((alist-get 'racket-mode package-alist))
((emacs-version "29.1")
 (system-type darwin)
 (x-gtk-use-system-tooltips UNDEFINED)
 (major-mode help-mode)
 (racket--el-source-dir "/Users/sorawee/.config/emacs/.local/straight/build-29.1/racket-mode/")
 (racket--rkt-source-dir "/Users/sorawee/.config/emacs/.local/straight/build-29.1/racket-mode/racket/")
 (racket-program "racket")
 (racket-command-timeout 10)
 (racket-path-from-emacs-to-racket-function UNDEFINED)
 (racket-path-from-racket-to-emacs-function UNDEFINED)
 (racket-browse-url-function racket-browse-url-using-temporary-file)
 (racket-documentation-search-location "https://docs.racket-lang.org/search/index.html?q=%s")
 (racket-xp-after-change-refresh-delay 1)
 (racket-xp-mode-lighter
  (:eval
   (racket--xp-mode-lighter)))
 (racket-xp-highlight-unused-regexp "^[^_]")
 (racket-repl-buffer-name-function nil)
 (racket-submodules-to-run
  ((test)
   (main)))
 (racket-memory-limit 2048)
 (racket-error-context medium)
 (racket-repl-history-directory "~/.config/emacs/.local/cache/racket-mode/")
 (racket-history-filter-regexp "\\`\\s *\\'")
 (racket-images-inline t)
 (racket-imagemagick-props nil)
 (racket-images-keep-last 100)
 (racket-images-system-viewer "open")
 (racket-pretty-print t)
 (racket-use-repl-submit-predicate nil)
 (racket-pretty-print t)
 (racket-indent-curly-as-sequence t)
 (racket-indent-sequence-depth 0)
 (racket-pretty-lambda nil)
 (racket-smart-open-bracket-enable nil)
 (racket-module-forms "\\s(\\(?:module[*+]?\\|library\\)")
 (racket-logger-config
  ((cm-accomplice . warning)
   (GC . info)
   (module-prefetch . warning)
   (optimizer . info)
   (racket/contract . error)
   (racket-mode-debugger . info)
   (sequence-specialization . info)
   (* . fatal)))
 (racket-show-functions
  (racket-show-pseudo-tooltip)))
(enabled-minor-modes
 (+popup-mode)
 (anzu-mode)
 (auto-composition-mode)
 (auto-compression-mode)
 (auto-encryption-mode)
 (auto-fill-mode)
 (auto-save-mode)
 (better-jumper-local-mode)
 (better-jumper-mode)
 (buffer-read-only)
 (centaur-tabs-mode)
 (column-number-mode)
 (delete-selection-mode)
 (doom-modeline-mode)
 (electric-indent-mode)
 (eros-mode)
 (evil-escape-mode)
 (evil-goggles-mode)
 (evil-local-mode)
 (evil-mode)
 (evil-snipe-local-mode)
 (evil-snipe-mode)
 (evil-snipe-override-local-mode)
 (evil-snipe-override-mode)
 (evil-surround-mode)
 (file-name-shadow-mode)
 (flyspell-lazy-mode)
 (font-lock-mode)
 (gcmh-mode)
 (general-override-mode)
 (global-anzu-mode)
 (global-company-mode)
 (global-eldoc-mode)
 (global-evil-surround-mode)
 (global-flycheck-mode)
 (global-font-lock-mode)
 (global-git-commit-mode)
 (global-hl-line-mode)
 (global-so-long-mode)
 (global-visual-line-mode)
 (hl-line-mode)
 (isearch-fold-quotes-mode)
 (line-number-mode)
 (mac-mouse-wheel-mode)
 (marginalia-mode)
 (ns-auto-titlebar-mode)
 (override-global-mode)
 (persp-mode)
 (projectile-mode)
 (recentf-mode)
 (save-place-mode)
 (savehist-mode)
 (semantic-minor-modes-format)
 (server-mode)
 (shell-dirtrack-mode)
 (show-paren-mode)
 (show-smartparens-global-mode)
 (size-indication-mode)
 (smartparens-global-mode)
 (solaire-global-mode)
 (solaire-mode)
 (transient-mark-mode)
 (undo-fu-mode)
 (undo-fu-session-global-mode)
 (vertico-mode)
 (visual-line-mode)
 (which-key-mode)
 (window-divider-mode)
 (winner-mode)
 (ws-butler-global-mode)
 (yas-global-mode)
 (yas-minor-mode))
(disabled-minor-modes
 (+emacs-lisp-ert-mode)
 (+emacs-lisp-non-package-mode)
 (+javascript-gulp-mode)
 (+javascript-npm-mode)
 (+lsp-optimization-mode)
 (+org-pretty-mode)
 (+popup-buffer-mode)
 (+web-angularjs-mode)
 (+web-django-mode)
 (+web-jekyll-mode)
 (+web-phaser-mode)
 (+web-react-mode)
 (+web-wordpress-mode)
 (2C-mode)
 (abbrev-mode)
 (auto-fill-function)
 (auto-revert-mode)
 (auto-revert-tail-mode)
 (auto-save-visited-mode)
 (blink-cursor-mode)
 (buffer-face-mode)
 (bug-reference-mode)
 (bug-reference-prog-mode)
 (button-mode)
 (centaur-tabs-local-mode)
 (cl-old-struct-compat-mode)
 (comint-fontify-input-mode)
 (company-box-mode)
 (company-mode)
 (company-search-mode)
 (compilation-minor-mode)
 (compilation-shell-minor-mode)
 (completion-in-region-mode)
 (consult-preview-at-point-mode)
 (context-menu-mode)
 (copilot-mode)
 (cursor-face-highlight-mode)
 (cursor-intangible-mode)
 (cursor-sensor-mode)
 (dash-fontify-mode)
 (defining-kbd-macro)
 (diff-auto-refine-mode)
 (diff-minor-mode)
 (dired-hide-details-mode)
 (dired-isearch-filenames-mode)
 (dired-omit-mode)
 (diredfl-global-mode)
 (diredfl-mode)
 (display-fill-column-indicator-mode)
 (display-line-numbers-mode)
 (dtrt-indent-global-mode)
 (dtrt-indent-mode)
 (edebug-backtrace-mode)
 (edebug-mode)
 (edit-indirect--overlay)
 (editorconfig-mode)
 (eldoc-mode)
 (electric-layout-mode)
 (electric-quote-mode)
 (elisp-def-mode)
 (embark-collect-direct-action-minor-mode)
 (ert--current-run-stats)
 (evil-collection-magit-toggle-text-minor-mode)
 (evil-markdown-mode)
 (evil-mc-mode)
 (flycheck-mode)
 (flycheck-popup-tip-mode)
 (flyspell-mode)
 (general-override-local-mode)
 (git-commit-mode)
 (git-gutter-mode)
 (global-auto-revert-mode)
 (global-copilot-mode)
 (global-dash-fontify-mode)
 (global-display-fill-column-indicator-mode)
 (global-display-line-numbers-mode)
 (global-evil-mc-mode)
 (global-git-gutter-mode)
 (global-goto-address-mode)
 (global-hide-mode-line-mode)
 (global-hl-todo-mode)
 (global-prettify-symbols-mode)
 (global-reveal-mode)
 (global-semantic-highlight-edits-mode)
 (global-semantic-highlight-func-mode)
 (global-semantic-show-parser-state-mode)
 (global-semantic-show-unmatched-syntax-mode)
 (global-semantic-stickyfunc-mode)
 (global-vi-tilde-fringe-mode)
 (global-whitespace-mode)
 (global-whitespace-newline-mode)
 (goto-address-mode)
 (goto-address-prog-mode)
 (header-line-indent-mode)
 (hide-mode-line-mode)
 (highlight-indent-guides-mode)
 (highlight-numbers-mode)
 (highlight-quoted-mode)
 (hl-todo-mode)
 (horizontal-scroll-bar-mode)
 (hs-minor-mode)
 (ibuffer-auto-mode)
 (indent-tabs-mode)
 (isearch-mode)
 (ispell-minor-mode)
 (jit-lock-debug-mode)
 (lock-file-mode)
 (lost-selection-mode)
 (mac-auto-ascii-mode)
 (mac-auto-operator-composition-mode)
 (mac-font-panel-mode)
 (magit-auto-revert-mode)
 (magit-blame-mode)
 (magit-blame-read-only-mode)
 (magit-blob-mode)
 (magit-todos-mode)
 (magit-wip-after-apply-mode)
 (magit-wip-after-save-local-mode)
 (magit-wip-after-save-mode)
 (magit-wip-before-change-mode)
 (magit-wip-initial-backup-mode)
 (magit-wip-mode)
 (mail-abbrevs-mode)
 (markdown-live-preview-mode)
 (menu-bar-mode)
 (mml-mode)
 (mouse-wheel-mode)
 (next-error-follow-minor-mode)
 (org-capture-mode)
 (org-cdlatex-mode)
 (org-indent-mode)
 (org-list-checkbox-radio-mode)
 (org-src-mode)
 (org-table-follow-field-mode)
 (org-table-header-line-mode)
 (orgtbl-mode)
 (outline-minor-mode)
 (overseer-mode)
 (overwrite-mode)
 (paragraph-indent-minor-mode)
 (pcre-mode)
 (prettify-symbols-mode)
 (racket-smart-open-bracket-mode)
 (racket-xp-mode)
 (rainbow-delimiters-mode)
 (read-extended-command-mode)
 (rectangle-mark-mode)
 (reveal-mode)
 (rxt--read-pcre-mode)
 (rxt-global-mode)
 (rxt-mode)
 (semantic-highlight-edits-mode)
 (semantic-highlight-func-mode)
 (semantic-mode)
 (semantic-show-parser-state-mode)
 (semantic-show-unmatched-syntax-mode)
 (semantic-stickyfunc-mode)
 (sh-electric-here-document-mode)
 (shell-command-with-editor-mode)
 (shell-highlight-undef-mode)
 (show-smartparens-mode)
 (smartparens-global-strict-mode)
 (smartparens-mode)
 (smartparens-strict-mode)
 (smerge-mode)
 (so-long-minor-mode)
 (tab-bar-history-mode)
 (tab-bar-mode)
 (temp-buffer-resize-mode)
 (text-scale-mode)
 (tool-bar-mode)
 (tooltip-mode)
 (transient-resume-mode)
 (treesit-explore-mode)
 (treesit-inspect-mode)
 (undelete-frame-mode)
 (undo-fu-session-mode)
 (url-handler-mode)
 (use-hard-newlines)
 (vc-dir-git-mode)
 (vc-parent-buffer)
 (vi-tilde-fringe-mode)
 (view-mode)
 (visible-mode)
 (which-function-mode)
 (whitespace-mode)
 (whitespace-newline-mode)
 (with-editor-mode)
 (ws-butler-mode)
 (xref-etags-mode))

[greghendershott]

Thanks for the report!

So far I can't reproduce this.

I tried creating the buffer "live" by typing it in... but no error.

Then I tried saving to a file, and visiting the file... but no error.

Are there some other steps I should try?

---

p.s. One scenario where I know you definitely will get such a message, is when you do something like M-x racket-stop-back-end to kill the back end process, or, M-x racket-start-back-end which kills then restarts it.

In that case the hash-lang object created by the old back end racket process disappeared along with that process. The new back end process knows nothing about it. The hash-lang object for the buffer would need to be re-created. The only way to do that now is to re-create each such racket-hash-lang-mode buffer in Emacs (e.g. by visiting its file again), to force the re-creation.

I need to figure out some mechanism to do this automatically for folks. Meanwhile fortunately restarting the whole back end is something I tend to do a lot while developing it, as opposed to "normal" users. But it does need a general solution.

[greghendershott]

I pushed commit d00194b which addresses the back end vanishing.

(I also changed how the back end hash-lang object is initially created. Although I still can't reproduce the problem you reported, it's possible the commit may have fixed it -- or at least changed the symptoms in some way that might be useful to know.)

[sorawee]

Thanks! I will try it and report back. It could also be that my setup is wrong -- I'm using Doom Emacs which uses straight.el for package building + use-package for package setup, and I have been struggling with it. I think :mode ("\\.rkt\\'" . racket-hash-lang-mode) is the right thing to do under (use-package! racket-mode ...), but I'm not sure.

Notably, I now see that the above file actually initially works OK. I can even run racket-xp-mode. But after editing for a while, I will start getting either this error message ("No hash-lang exists") or the one that says < in #<<EOF is unrecognized by the reader. When that happens, I either have to revert the buffer, or restart Emacs.

Sorry for the unorganized information dumping. I posted it in case you can recognize anything off the top of your head, but I know this is probably not enough information for debugging.

[greghendershott]

Sorry for the unorganized information dumping. I posted it in case you can recognize anything off the top of your head, but I know this is probably not enough information for debugging.

No worries. From your example maybe you guessed that the comment before the #lang line was the problem -- which would have been a reasonable thing to guess. (I suppose it could even be a necessary ingredient; it's just not sufficient based on me trying.)

Thanks! I will try it and report back. It could also be that my setup is wrong -- I'm using Doom Emacs which uses straight.el for package building + use-package for package setup, and I have been struggling with it. I think :mode ("\\.rkt\\'" . racket-hash-lang-mode) is the right thing to do under (use-package! racket-mode ...), but I'm not sure.

Although :mode may be correct (or even the preferred way in modern use-package, idk), FWIW just I directly set auto-mode-alist in the :config section:

(use-package racket-mode
  :load-path "~/src/elisp/racket-mode"
...
  :config
...
  (require 'racket-hash-lang)
  (add-to-list 'auto-mode-alist '("\\.rkt\\'" . racket-hash-lang-mode))
  (add-to-list 'auto-mode-alist '("\\.scrbl\\'" . racket-hash-lang-mode))
  (add-to-list 'auto-mode-alist '("\\.rhm\\'" . racket-hash-lang-mode))
...

Notably, I now see that the above file actually initially works OK. I can even run racket-xp-mode. But after editing for a while, I will start getting either this error message ("No hash-lang exists") or the one that says < in #<<EOF is unrecognized by the reader. When that happens, I either have to revert the buffer, or restart Emacs.

The "< in #<<EOF" one is an interesting additional message. I don't think I've seen that one, exactly. Hmm....

If this still happens for you with the new commit, could you please:

1. Switch to your Emacs *Messages* buffer (where you'll find a copy of these things displayed in the mode line).
2. Starting maybe a couple dozen lines before the first of either of these hash-lang related error messages, copy.
3. Paste here.

Thanks!!

[greghendershott]

Also, I just pushed a commit to add more detail to these messages about a back end exception; this might help us figure out what's happening.

[sorawee]

I followed your use-package advice, and things are now somehow worse. They don't appear to work at all...

Here's a video recording of the issue, with both *Messages* and *Racket Logger* buffers shown

https://github.com/greghendershott/racket-mode/assets/9099577/6e9dbcf4-06a2-467a-86a9-0442feeee0cc

[greghendershott]

Thanks! One thing I notice is that you seem to have two back ends running at the same time. (A racket process started by Racket Mode, to run programs and to host functionality that can't be implemented in Emacs.)

Although that could be normal if intentional -- e.g. one back is on a remote machine, or uses a different version of Racket when source files are under a certain subdirectory -- maybe not in your case? Maybe somehow the one buffer is trying to interact variously with both back ends? Which shouldn't normally happen, but maybe there's a bug I don't know about there.

1. One hint there are two back ends: The *Racket Logger* buffer shows "Accepting TCP connections" twice, with different port numbers (the back end chooses an "ephemeral" port number, i.e. whatever's available). You might see that if you restart the same server (e.g. via racket-start-back-end), which is something I do a lot while developing Racket Mode but AFAIK most users wouldn't, and I didn't notice such a command in your video.

2. More specifically, the *Messages* buffer has some lines showing the names of each back end:

   • racket-back-end-/ (the default that gets created automatically)
   • racket-back-end-/-stderr ???

The second one seems... odd. Is this intentional -- do you have any back-end commands in your config? Or is this somehow getting created automagically?

Actually can you paste your whole use-package form for Racket Mode, here, please?

[sorawee]

Thanks for taking a look. I feel very bad because it looks like I'm the only one who experienced this problem. So please feel free to deprioritize this issue.

Here's my use-package:

(use-package racket-mode
  :load-path "~/projects/racket-mode"
  :config
  (require 'racket-hash-lang)
  (add-to-list 'auto-mode-alist '("\\.rkt\\'" . racket-hash-lang-mode))
  (add-to-list 'auto-mode-alist '("\\.scrbl\\'" . racket-hash-lang-mode))
  (add-to-list 'auto-mode-alist '("\\.rhm\\'" . racket-hash-lang-mode)))

In the screen recording above, after the *Racket Logger* buffer is shown on screen, every action I did is solely editing operations: entering a newline, inserting characters, o in evil mode (which inserts a newline and indent), explicit indent (via tab). I definitely didn't intentionally spawn any new backend. And I don't think I have any other Racket Mode configuration.

[greghendershott]

I know you're busy and I'm sorry you're spending so much time on this issue

I'm not sorry that I am; I feel like I'm going to learn something valuable, eventually.

---

I realize I might have misunderstood the "{racket-back-end-/-stderr}" in the *Messages* buffer. I think that is simply relaying some stderr output from the back end process. Maybe an additional back end is not being created.

When you're in this situation (like near the end of your video), with the test.rkt buffer selected, can you M-x describe-variable racket-back-end-configurations and paste the value here?

(Again I'm sorry this is turning out to be such a saga for you. :disappointed:)

[sorawee]

Mystery partially solved: it's due to the package highlight-indent-guides. If I disable this package, the problem goes away.

[greghendershott]

Mystery partially solved: it's due to the package highlight-indent-guides. If I disable this package, the problem goes away.

Thanks!! Installed highlight-indent-guides. Can reproduce. Investigating...

[sorawee]

Thanks, I really appreciate it!

[greghendershott]

The basic problem is that highlight-indent-guides-mode is using a similar mechanism as racket-hash-lang-mode -- updates driven by jit-font-lock-mode (wait until a span of the buffer becomes visible, before doing work) -- and they are conflicting.

This is exacerbated by me trying to avoid blocking Emacs in the situation where racket-hash-lang-mode starts, but the Racket Mode back end is not yet running. I have been trying to initialize the buffer in an intermediate state, until the back end becomes available. Although nice, this results in a situation where highlight-indent-guides-mode would go ahead and do some font-lock work -- causing some of my code to run earlier than expected. This is the source of the "no hash-lang exists" messages.

Even fixing that, the conflict means that racket-hash-lang-mode is preventing highlight-indent-guides-mode from having any visible effect, mostly.

---

After looking at this, the fix seems to be:

1. Abandon the fancy stuff. If the back end isn't running when the first racket-hash-lang-mode is started, too bad, Emacs will be blocked for N seconds (where N is around 5 on my not-new laptop). I can show a message explaining why, during the wait.

2. Instead of doing my work as a font-lock-fontify-region-function, do it as a jit-lock-register function, which runs sooner (and is how highlight-indent-guides-mode works, so they co-exist on equal footing). This allows both to work, in my testing so far.

---

I have a commit I'll push to an issue topic branch, soon. Because this change is not exactly trivial, and it's for (as far as I know, so far) just this one minor mode (where "don't use it" is a work-around), I'd like to sleep on this awhile before merging.

[greghendershott]

After sleeping on it, I believe I understand how to preserve the "fancy" part of not blocking Emacs while waiting for the back end to start. Still testing...

[greghendershott]

Still working on this. It bothered me I didn't know why you got the error about #< being unreadable.

It turns out this is because my fontify-region function was being given, in some cases, not an integer position but an Emacs marker -- which prints as #<marker ...>. Sending that in a command sexp to the back end would cause the back end to error -- and abend!

I've added a sanity check in Emacs for this generally (so it fails earlier, with a better error message, and without abending the back end). I also added a check/conversion specifically where this was happening.

---

Also: I'm not so sure I should switch to using a jit-lock-register function here. I think I should stick with a font-lock-fontify-region-function, and simply call the default fontify-region (plus my extra work) to allow things like highlight-indent-guides to font-lock, too.

So I'm exploring that.

Also everything I do in racket-hash-lang-mode, I need to test and make sure it works for font-locking input and current-printed values in racket-repl-mode.

So...

[sorawee]

Thank you again!

[greghendershott]

You're welcome and thanks for all your help!