Issue with racket-xp-documentation after switching from emacs 27.2 to native comp branch 28.0.5

bhrgunatha commented 3 years ago

I'm not sure if this is a true bug or just a heads-up for a potential future problem. (I did see you mention using emacs 28 in a different issue though) so I thought I'd post here as it's currently affecting racket-mode for me.

Issue - racket-xp-documentation behaves differently with the native comp branch of emacs 28 from emacs 27.2.
Description: Whenever I call racket-xp-documentation (whether keyboard shortcut or M-x racket-xp-documentation I see messages in the echo area and a new fundamental mode buffer appears in a window (split to the right). An example looking for for/list documentation opens a buffer named "list))" and is visiting "file:///usr/share/doc/racket/reference/for.html#(form._((lib._racket/private/base..rkt)._for/list))" - which is the correct file and the correct url slug!
What I expect. I expect the browser to open that document at the correct place according to the url + slug "...for.html#".

I see 2 messages "File exists, but cannot be read" then a longer message "File mode specification error: (file-error Setting current directory Not a directory file:///usr/share/doc/racket/reference/for.html#(form.((lib._racket/private/base..rkt)._for/)".

After repeatedly trying to open documentation the messages can change e.g. if I try documentation for for/list followed by for/or - the message says "file:///...for/html#..._for./or" and file file:///...for/html#..._for./list are the same file. I think the underlying problem of opening the OS file in a buffer instead of a browser is the same.

racket-bug-report

((alist-get 'racket-mode package-alist)
 #s(package-desc racket-mode
                 (20210328 2038)
                 "Racket editing, REPL, and more"
                 ((emacs
                   (25 1))
                  (faceup
                   (0 0 2))
                  (pos-tip
                   (20191127 1028)))
                 nil nil "/home/bhrgunatha/.emacs.d/elpa/racket-mode-20210328.2038"
                 ((:url . "https://www.racket-mode.com/")
                  (:maintainer "Greg Hendershott")
                  (:authors
                   ("Greg Hendershott"))
                  (:commit . "045a871d61e930c2eea8647822df39b8319018e1"))
                 nil))
((emacs-version "28.0.50")
 (system-type gnu/linux)
 (x-gtk-use-system-tooltips t)
 (major-mode racket-mode)
 (racket--el-source-dir "/home/bhrgunatha/.emacs.d/elpa/racket-mode-20210328.2038/")
 (racket--rkt-source-dir "/home/bhrgunatha/.emacs.d/elpa/racket-mode-20210328.2038/racket/")
 (racket-program "racket")
 (racket-command-timeout 10)
 (racket-xp-after-change-refresh-delay 1)
 (racket-xp-highlight-unused-regexp "^[^_]")
 (racket-repl-buffer-name-function nil)
 (racket-memory-limit 8192)
 (racket-error-context medium)
 (racket-history-filter-regexp "\\`\\s *\\'")
 (racket-images-inline t)
 (racket-images-keep-last 100)
 (racket-images-system-viewer "display")
 (racket-images-system-viewer "display")
 (racket-use-repl-submit-predicate nil)
 (racket-pretty-print t)
 (racket-indent-curly-as-sequence t)
 (racket-indent-sequence-depth 0)
 (racket-pretty-lambda nil)
 (racket-smart-open-bracket-enable nil)
 (racket-module-forms "\\s(\\(?:module[*+]?\\|library\\)")
 (racket-logger-config
  ((cm-accomplice . warning)
   (GC . info)
   (module-prefetch . warning)
   (optimizer . info)
   (racket/contract . error)
   (sequence-specialization . info)
   (* . fatal)))
 (racket-show-functions nil))
(enabled-minor-modes
 (auto-composition-mode)
 (auto-compression-mode)
 (auto-encryption-mode)
 (auto-fill-mode)
 (auto-save-mode)
 (blink-cursor-mode)
 (company-mode)
 (delete-selection-mode)
 (display-fill-column-indicator-mode)
 (display-line-numbers-mode)
 (electric-indent-mode)
 (file-name-shadow-mode)
 (font-lock-mode)
 (global-company-mode)
 (global-display-fill-column-indicator-mode)
 (global-display-line-numbers-mode)
 (global-eldoc-mode)
 (global-font-lock-mode)
 (global-hl-line-mode)
 (hs-minor-mode)
 (ido-vertical-mode)
 (ivy-mode)
 (line-number-mode)
 (menu-bar-mode)
 (minibuffer-depth-indicate-mode)
 (mouse-wheel-mode)
 (override-global-mode)
 (paredit-mode)
 (projectile-mode)
 (racket-xp-mode)
 (rainbow-delimiters-mode)
 (recentf-mode)
 (semantic-minor-modes-format)
 (show-paren-mode)
 (tooltip-mode)
 (transient-mark-mode)
 (which-key-mode)
 (window-divider-mode)
 (yas-global-mode)
 (yas-minor-mode))
(disabled-minor-modes
 (abbrev-mode)
 (auto-fill-function)
 (auto-save-visited-mode)
 (buffer-read-only)
 (button-mode)
 (cl-old-struct-compat-mode)
 (column-number-mode)
 (company-search-mode)
 (compilation-minor-mode)
 (compilation-shell-minor-mode)
 (completion-in-region-mode)
 (counsel-mode)
 (dash-fontify-mode)
 (defining-kbd-macro)
 (diff-auto-refine-mode)
 (diff-minor-mode)
 (dired-hide-details-mode)
 (eldoc-mode)
 (electric-layout-mode)
 (electric-quote-mode)
 (global-dash-fontify-mode)
 (global-prettify-symbols-mode)
 (global-semantic-highlight-edits-mode)
 (global-semantic-highlight-func-mode)
 (global-semantic-show-parser-state-mode)
 (global-semantic-show-unmatched-syntax-mode)
 (global-semantic-stickyfunc-mode)
 (global-visual-line-mode)
 (hl-line-mode)
 (horizontal-scroll-bar-mode)
 (html-autoview-mode)
 (ibuffer-auto-mode)
 (ido-everywhere)
 (isearch-mode)
 (jit-lock-debug-mode)
 (mail-abbrevs-mode)
 (mml-mode)
 (next-error-follow-minor-mode)
 (overwrite-mode)
 (paragraph-indent-minor-mode)
 (prettify-symbols-mode)
 (racket-smart-open-bracket-mode)
 (semantic-highlight-edits-mode)
 (semantic-highlight-func-mode)
 (semantic-mode)
 (semantic-show-parser-state-mode)
 (semantic-show-unmatched-syntax-mode)
 (semantic-stickyfunc-mode)
 (server-mode)
 (sgml-electric-tag-pair-mode)
 (sh-electric-here-document-mode)
 (show-smartparens-global-mode)
 (show-smartparens-mode)
 (size-indication-mode)
 (smartparens-global-mode)
 (smartparens-global-strict-mode)
 (smartparens-mode)
 (smartparens-strict-mode)
 (tab-bar-history-mode)
 (tab-bar-mode)
 (temp-buffer-resize-mode)
 (tool-bar-mode)
 (unify-8859-on-decoding-mode)
 (unify-8859-on-encoding-mode)
 (url-dired-minor-mode)
 (url-handler-mode)
 (use-hard-newlines)
 (view-mode)
 (visible-mode)
 (visual-line-mode)
 (xref-etags-mode))

Steps to reproduce:

Create a new file temp.rkt with contents
```
#lang racket/base
(for/list ([x 10]) x)
```

Open it with `racket -mode` and `racket-xp-mode`. Put cursor on `for/list` - hit `C-c C-d` or `M-x racket-xp-documentation`.

My racket-mode config has:

(use-package racket-mode
:diminish (hs-minor-mode racket-xp-mode)
:hook (racket-mode . racket-xp-mode)
:mode (("\\.rkt$" . racket-mode)
     ("\\.scm$" . racket-mode)
     ("\\.ss$"  . racket-mode))
:bind
(:map racket-mode-map
    ("C-M-\\" . racket-insert-lambda)
    ("C-c C-k" . racket-run-and-switch-to-repl)
    )
:config
(set-face-attribute 'racket-keyword-argument-face  nil :foreground "DeepSkyBlue3")
(set-face-attribute 'racket-selfeval-face  nil :foreground "gold3")
(setq racket-memory-limit 8192
    racket-show-functions nil))

I do use e.g. paredit and company as well as other too.

Using the Arch linux default emacs package - 27.2 works fine!

I compiled the native comp branch of 28.0.5 using the AUR package but basically all that does is compile the current state of the native comp branch. I understand that's different every time it's compiled but I see the same behaviour each time.

bhrgunatha commented 3 years ago

I've tried to find out the cause using message debugging to see what procedure are getting called with what arguments:

DIAG: (racket-xp-documentation nil)
DIAG: (get-text-property (point) ’racket-xp-doc) -> nil
DIAG: (racket--doc prefix: nil how: /home/bhrgunatha/scratch/code/test.rkt completions: nil)
DIAG: search-p nil
DIAG: (racket--symbol-at-point-or-prompt force-prompt-p: nil
                                          prompt: Documentation for: 
                                          completions: nil
                                          allow-blank-p: nil)
DIAG: (racket--doc-command repl-session-id: nil how: /home/bhrgunatha/scratch/code/test.rkt str: for/list)
DIAG: (racket--cmd/async repl-session-id: nil command-sexpr: (doc /home/bhrgunatha/scratch/code/test.rkt for/list) callback: (closure ((str . for/list) (how . /home/bhrgunatha/scratch/code/test.rkt) (repl-session-id) t) (maybe-url) (if maybe-url (racket-browse-url maybe-url) (racket--search-doc str))))
DIAG: (racket-browse-url (url: file:///usr/share/doc/racket/reference/for.html#(form._((lib._racket/private/base..rkt)._for/list))
args: nil)
DIAG: racket-browse-url-function: browse-url

In the scratch buffer BOTH

(browse-url "file:///usr/share/doc/racket/reference/for.html#(form._((lib._racket/private/base..rkt)._for/list))")
(browse-url "file:///usr/share/doc/racket/reference/for.html#(form._((lib._racket/private/base..rkt)._for/list))" nil)

open the browser at the correct page at the correct url-slug.

I can't understand the difference between what the code and my debugging show and the actual behaviour I see from emacs..

I'm totally stuck.

bhrgunatha commented 3 years ago

I've unstuck myself by learning how to use edebug!

If anyone else stumbles over this problem I fixed it by adding this in my config:

(setq browse-url-handlers '(("\\`file://.*\\.html?" . browse-url-default-browser)))

You should probably write something better to determine valid html documents though.

Explanation:

There is a change in how emacs' built-in browse-url works.

The reason emacs is opening the file (instead of a browser) is because browse-url calls browse-url-select-handler whose behaviour has changed. It checks these in order:

browse-url-browser-function
browse-url-handlers
browse-url-default-handlers

In my case because the url is a local file (file:///...) and the url doesn't end with "htm" or "html", the new behaviour falls through all the test cases to determine what to handle the url with and defaults to emacs itself as a last resort.

Note: None of this explains why evaluating (browse-url "file:///usr/share/doc/racket/reference/for.html#(form._((lib._racket/private/base..rkt)._for/list))") from the *scratch* buffer opens the file in a browser, but when racket-mode calls browse-url it fails. :(

greghendershott commented 3 years ago

I'm sorry I didn't comment on this issue before.

My excuse: I was focused on other things, and, this seemed like it might be less urgent due to it being for pre-release Emacs.

But I was really happy you reported this, and were even investigating -- thanks!!

I'm actually going to re-open this issue as a to-do for me to make Racket Mode work with 28.0.50 better "out-of-box". (Of course I'd understand if you want to mute this issue if you've had enough of it by now!)

Thanks again!

bhrgunatha commented 3 years ago

If you need anything from me just let me know here.

The only odd thing for me is the note in my comment above.

greghendershott commented 3 years ago

I've started to reload my brain with some of the history here, including #476.

Quick question: What happens if you change the customization variable racket-browse-url-function from its default value of browse-url to instead be racket-browse-url-using-temporary-file?

p.s. I notice I have an open PR #530 which I should review and merge, but I think that's orthogonal to the issue you reported.

bhrgunatha commented 3 years ago

Success!

Removing

(setq browse-url-handlers '(("\\`file://.*\\.html?" . browse-url-default-browser)))

and adding

(setq racket-browse-url-function 'racket-browse-url-using-temporary-file)

Opens the correct local racket html documentation file (including the # slug) at the correct anchor within the document in my browser.

greghendershott commented 3 years ago

Interesting. Probably I should just default that to racket-browse-url-using-temporary-file on Linux, as well.

I notice I also have recent-ish open PR #530 I need to review and merge.

I'll try to tackle both items today.

Thanks again for your help!

greghendershott / racket-mode