Semicolon in at-exp braces

[sorawee]

Consider:

#lang at-exp racket

(values
 @~a{b;cdef}
     "def")

This is a valid Racket code in the at-exp notation. However, ; is treated as a comment marker, causing a bad syntax highlight and brace mismatches.

((alist-get 'racket-mode package-alist))
((emacs-version "28.1")
 (system-type darwin)
 (x-gtk-use-system-tooltips UNDEFINED)
 (major-mode racket-mode)
 (racket--el-source-dir "/Users/sorawee/.emacs.d/.local/straight/build-28.1/racket-mode/")
 (racket--rkt-source-dir "/Users/sorawee/.emacs.d/.local/straight/build-28.1/racket-mode/racket/")
 (racket-program "racket")
 (racket-command-timeout 10)
 (racket-path-from-emacs-to-racket-function UNDEFINED)
 (racket-path-from-racket-to-emacs-function UNDEFINED)
 (racket-browse-url-function racket-browse-url-using-temporary-file)
 (racket-documentation-search-location "https://docs.racket-lang.org/search/index.html?q=%s")
 (racket-xp-after-change-refresh-delay 1)
 (racket-xp-mode-lighter
  (:eval
   (racket--xp-mode-lighter)))
 (racket-xp-highlight-unused-regexp "^[^_]")
 (racket-repl-buffer-name-function nil)
 (racket-submodules-to-run
  ((test)
   (main)))
 (racket-memory-limit 2048)
 (racket-error-context medium)
 (racket-repl-history-directory "~/.emacs.d/.local/etc/racket-mode/")
 (racket-history-filter-regexp "\\`\\s *\\'")
 (racket-images-inline t)
 (racket-imagemagick-props nil)
 (racket-images-keep-last 100)
 (racket-images-system-viewer "open")
 (racket-pretty-print t)
 (racket-use-repl-submit-predicate nil)
 (racket-pretty-print t)
 (racket-indent-curly-as-sequence t)
 (racket-indent-sequence-depth 0)
 (racket-pretty-lambda nil)
 (racket-smart-open-bracket-enable nil)
 (racket-module-forms "\\s(\\(?:module[*+]?\\|library\\)")
 (racket-logger-config
  ((cm-accomplice . warning)
   (GC . info)
   (module-prefetch . warning)
   (optimizer . info)
   (racket/contract . error)
   (racket-mode-debugger . info)
   (sequence-specialization . info)
   (* . fatal)))
 (racket-show-functions
  (racket-show-pseudo-tooltip)))
(enabled-minor-modes
 (+popup-mode)
 (anzu-mode)
 (auto-composition-mode)
 (auto-compression-mode)
 (auto-encryption-mode)
 (auto-fill-mode)
 (auto-save-mode)
 (better-jumper-local-mode)
 (better-jumper-mode)
 (centaur-tabs-mode)
 (column-number-mode)
 (company-mode)
 (delete-selection-mode)
 (diff-hl-margin-mode)
 (display-fill-column-indicator-mode)
 (display-line-numbers-mode)
 (doom-modeline-mode)
 (dtrt-indent-mode)
 (editorconfig-mode)
 (electric-indent-mode)
 (evil-escape-mode)
 (evil-goggles-mode)
 (evil-local-mode)
 (evil-mode)
 (evil-snipe-local-mode)
 (evil-snipe-mode)
 (evil-snipe-override-local-mode)
 (evil-snipe-override-mode)
 (evil-surround-mode)
 (evil-traces-mode)
 (file-name-shadow-mode)
 (flycheck-mode)
 (flycheck-popup-tip-mode)
 (flyspell-lazy-mode)
 (font-lock-mode)
 (gcmh-mode)
 (general-override-mode)
 (global-anzu-mode)
 (global-company-mode)
 (global-eldoc-mode)
 (global-evil-surround-mode)
 (global-flycheck-mode)
 (global-font-lock-mode)
 (global-git-commit-mode)
 (global-hl-line-mode)
 (global-so-long-mode)
 (global-undo-fu-session-mode)
 (global-visual-line-mode)
 (highlight-indent-guides-mode)
 (highlight-numbers-mode)
 (highlight-quoted-mode)
 (hl-line-mode)
 (hl-todo-mode)
 (hs-minor-mode)
 (ivy-mode)
 (ivy-rich-mode)
 (ivy-rich-project-root-cache-mode)
 (line-number-mode)
 (lispy-mode)
 (lispyville-mode)
 (mouse-wheel-mode)
 (ns-auto-titlebar-mode)
 (persp-mode)
 (projectile-mode)
 (racket-xp-mode)
 (rainbow-delimiters-mode)
 (ranger-override-dired-mode)
 (recentf-mode)
 (save-place-mode)
 (savehist-mode)
 (semantic-minor-modes-format)
 (shell-dirtrack-mode)
 (show-paren-mode)
 (show-smartparens-global-mode)
 (show-smartparens-mode)
 (size-indication-mode)
 (smartparens-global-mode)
 (smartparens-mode)
 (solaire-global-mode)
 (transient-mark-mode)
 (undo-fu-mode)
 (undo-fu-session-mode)
 (vi-tilde-fringe-mode)
 (visual-line-mode)
 (which-key-mode)
 (whitespace-mode)
 (windmove-mode)
 (window-divider-mode)
 (winner-mode)
 (ws-butler-global-mode)
 (ws-butler-mode)
 (yas-global-mode)
 (yas-minor-mode))
(disabled-minor-modes
 (+emacs-lisp-ert-mode)
 (+javascript-gulp-mode)
 (+javascript-npm-mode)
 (+lsp-optimization-mode)
 (+org-pretty-mode)
 (+popup-buffer-mode)
 (+web-angularjs-mode)
 (+web-django-mode)
 (+web-jekyll-mode)
 (+web-phaser-mode)
 (+web-react-mode)
 (+web-wordpress-mode)
 (abbrev-mode)
 (amx-debug-mode)
 (amx-mode)
 (auto-fill-function)
 (auto-revert-mode)
 (auto-revert-tail-mode)
 (auto-save-visited-mode)
 (avy-linum-mode)
 (blink-cursor-mode)
 (buffer-face-mode)
 (buffer-read-only)
 (bug-reference-mode)
 (bug-reference-prog-mode)
 (button-mode)
 (centaur-tabs-local-mode)
 (cl-old-struct-compat-mode)
 (company-search-mode)
 (compilation-minor-mode)
 (compilation-shell-minor-mode)
 (completion-in-region-mode)
 (context-menu-mode)
 (counsel-mode)
 (counsel-projectile-mode)
 (cursor-intangible-mode)
 (cursor-sensor-mode)
 (dash-fontify-mode)
 (defining-kbd-macro)
 (diff-auto-refine-mode)
 (diff-hl-dir-mode)
 (diff-hl-dired-mode)
 (diff-hl-margin-local-mode)
 (diff-hl-mode)
 (diff-minor-mode)
 (dired-hide-details-mode)
 (dtrt-indent-global-mode)
 (dumb-jump-mode)
 (edebug-mode)
 (edit-indirect--overlay)
 (eldoc-mode)
 (electric-layout-mode)
 (electric-quote-mode)
 (evil-collection-magit-toggle-text-minor-mode)
 (evil-markdown-mode)
 (evil-mc-mode)
 (evil-multiedit-mode)
 (evil-visualstar-mode)
 (flyspell-mode)
 (general-override-local-mode)
 (git-commit-mode)
 (git-gutter-mode)
 (global-auto-revert-mode)
 (global-dash-fontify-mode)
 (global-diff-hl-mode)
 (global-display-fill-column-indicator-mode)
 (global-display-line-numbers-mode)
 (global-evil-mc-mode)
 (global-evil-visualstar-mode)
 (global-git-gutter-mode)
 (global-goto-address-mode)
 (global-hide-mode-line-mode)
 (global-hl-todo-mode)
 (global-prettify-symbols-mode)
 (global-reveal-mode)
 (global-semantic-highlight-edits-mode)
 (global-semantic-highlight-func-mode)
 (global-semantic-show-parser-state-mode)
 (global-semantic-show-unmatched-syntax-mode)
 (global-semantic-stickyfunc-mode)
 (global-vi-tilde-fringe-mode)
 (global-whitespace-mode)
 (global-whitespace-newline-mode)
 (goto-address-mode)
 (goto-address-prog-mode)
 (hide-mode-line-mode)
 (horizontal-scroll-bar-mode)
 (ibuffer-auto-mode)
 (iedit-mode)
 (indent-tabs-mode)
 (irony-mode)
 (isearch-mode)
 (ispell-minor-mode)
 (jit-lock-debug-mode)
 (lispy-goto-mode)
 (lispy-other-mode)
 (lock-file-mode)
 (macrostep-mode)
 (magit-auto-revert-mode)
 (magit-blame-mode)
 (magit-blame-read-only-mode)
 (magit-blob-mode)
 (magit-gitflow-mode)
 (magit-popup-help-mode)
 (magit-todos-mode)
 (magit-wip-after-apply-mode)
 (magit-wip-after-save-local-mode)
 (magit-wip-after-save-mode)
 (magit-wip-before-change-mode)
 (magit-wip-initial-backup-mode)
 (magit-wip-mode)
 (mail-abbrevs-mode)
 (markdown-live-preview-mode)
 (mc-hide-unmatched-lines-mode)
 (menu-bar-mode)
 (mml-mode)
 (multiple-cursors-mode)
 (next-error-follow-minor-mode)
 (nroff-electric-mode)
 (org-capture-mode)
 (org-cdlatex-mode)
 (org-list-checkbox-radio-mode)
 (org-src-mode)
 (org-table-follow-field-mode)
 (org-table-header-line-mode)
 (orgtbl-mode)
 (outline-minor-mode)
 (overwrite-mode)
 (paragraph-indent-minor-mode)
 (pcre-mode)
 (prettify-symbols-mode)
 (racket-smart-open-bracket-mode)
 (rectangle-mark-mode)
 (repeat-mode)
 (reveal-mode)
 (rxt--read-pcre-mode)
 (rxt-global-mode)
 (rxt-mode)
 (semantic-highlight-edits-mode)
 (semantic-highlight-func-mode)
 (semantic-mode)
 (semantic-show-parser-state-mode)
 (semantic-show-unmatched-syntax-mode)
 (semantic-stickyfunc-mode)
 (server-mode)
 (sh-electric-here-document-mode)
 (shell-command-with-editor-mode)
 (smartparens-global-strict-mode)
 (smartparens-strict-mode)
 (smerge-mode)
 (so-long-minor-mode)
 (solaire-mode)
 (tab-bar-history-mode)
 (tab-bar-mode)
 (temp-buffer-resize-mode)
 (text-scale-mode)
 (tool-bar-mode)
 (tooltip-mode)
 (transient-resume-mode)
 (unify-8859-on-decoding-mode)
 (unify-8859-on-encoding-mode)
 (url-handler-mode)
 (use-hard-newlines)
 (vc-parent-buffer)
 (view-mode)
 (visible-mode)
 (which-function-mode)
 (whitespace-newline-mode)
 (with-editor-mode)
 (xref-etags-mode))

[greghendershott]

Yes.

A (not very satisfactory) work-around is the usual "Escape to Racket using @", like how you'd enter a literal "@":

@~a{foo; bar}
@~a{foo@";" bar}

Note that the rule-based syntax highlighter here on GitHub shares the problem of the rule-based racket-mode.

The optional racket-hash-lang-mode minor mode available on the hash-lang branch does handle this. It replaces the "traditional" Racket major mode stuff with code provided by the lang (and in this case, meta-lang) to do syntax highlighting, indent, and expression navigation. Not only does this support entirely different langs, like rhombus, it handles various edge cases like this bug, for #lang racket, and knows if at-exp or scribble is actually used (as opposed to guessing based on a @ character textually).

So, my hunch is that it's not worth the effort (as well as the risk of introducing other bugs) to try to fix most of these remaining edge cases the traditional way. Instead, this would probably be closed by merging the hash-lang branch, and recommending people enable racket-hash-lang-mode.

[Someday maybe that should even be enabled by default. But I think there would still be a role for a traditional, Emacs racket-mode -- one that doesn't need a Racket back end process running, which some people might be unwilling or unable to do, "just to edit a text file".]

[greghendershott]

FWIW this is definitely fixed on the hash-lang branch. I've dog-fooding that heavily lately and trying to pin down enough details to marge it to main branch, as soon as I can.

[greghendershott]

trying to pin down enough details to marge it to main branch, as soon as I can.

To clarify: After Marge Simpson approves, I plan to also, you know, merge it.