"Wrong major mode" when using treesit modes

[fast-90]

I get a Wrong major-mode error when using python-ts-mode (set through major-mode-remap-alist). This is caused by the fact that the Jupyter REPL starts in python-ts-mode, but the org-src edit buffer is python-mode.

A workaround would be to do something like:

(org-babel-jupyter-override-src-block "python")
;; To ensure python src blocks are opened in python-ts-mode
(setf (alist-get "python" org-src-lang-modes nil nil #'equal) 'python-ts)

(I haven't tried it without org-babel-jupyer-override-src-block, but I suspect it should work the same if you replace "python" with "jupyter-python")

I was wondering if there is any way to build this into emacs-jupyter, i.e. identify whether the user has mapped python-mode to python-ts-mode and ensure org-src-lang-modes is set accordingly?

Further information on my test environment:

System / package versions used:

• MacOS Ventura 13.4
• Emacs 29.0.91
• Org mode version 9.6.6
• Jupyter 2023-06-27 from MELPA

Backtrace:

Debugger entered--Lisp error: (error "Cannot associate buffer to REPL.  Wrong ‘major-mod...")
  error("Cannot associate buffer to REPL.  Wrong `major-mod...")
  jupyter-repl-associate-buffer(#<buffer *jupyter-repl[python 3.11.3]-py*>)
  org-babel-edit-prep:jupyter-python(("python" "a = 2\n\n\nmatch a:\n    case 1:\n        print(\"It's 1..." ((:results . "replace") (:exports . "code") (:session . "py") (:kernel . "numpy") (:async . "yes") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 79 "(ref:%s)"))
  apply(org-babel-edit-prep:jupyter-python ("python" "a = 2\n\n\nmatch a:\n    case 1:\n        print(\"It's 1..." ((:results . "replace") (:exports . "code") (:session . "py") (:kernel . "numpy") (:async . "yes") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 79 "(ref:%s)"))
  org-babel-edit-prep:python(("python" "a = 2\n\n\nmatch a:\n    case 1:\n        print(\"It's 1..." ((:results . "replace") (:exports . "code") (:session . "py") (:kernel . "numpy") (:async . "yes") (:tangle . "no") (:hlines . "no") (:noweb . "no") (:cache . "no")) "" nil 79 "(ref:%s)"))
  org-edit-src-code()
  org-babel-do-key-sequence-in-edit-buffer("\11")
  org-indent-line()
  indent-according-to-mode()
  electric-indent-post-self-insert-function()
  newline(nil 1)
  org--newline(nil nil 1)
  org-return(nil nil 1)
  funcall-interactively(org-return nil nil 1)
  command-execute(org-return)

Minimum config:

(setq native-comp-async-report-warnings-errors nil)

(use-package org
  :elpaca nil
  :init
  (org-babel-do-load-languages
   'org-babel-load-languages
   '((emacs-lisp . t)
     (python . t)))
  :custom
  (org-confirm-babel-evaluate nil)   
  (org-src-fontify-natively t))

(use-package jupyter
  :elpaca t
  :init
  (org-babel-do-load-languages
   'org-babel-load-languages
   (append org-babel-load-languages '((jupyter . t))))
  (org-babel-jupyter-override-src-block "python")
  (setq org-babel-default-header-args:jupyter-python '((:async . "yes")
                                                       (:session . "py")
                                                       (:kernel . "python3"))))

(use-package treesit
  :elpaca nil
  :init
  (setq treesit-language-source-alist
        '((python . ("git@github.com:tree-sitter/tree-sitter-python.git"))))
  :config
  (when (and (treesit-available-p) (executable-find "tree-sitter"))
    (treesit-install-language-grammar 'python)
    (add-to-list 'major-mode-remap-alist '(python-mode . python-ts-mode))))

Minimum notebook:

:PROPERTIES:
:header-args:python: :async yes :kernel python3 :session py
:END:

#+begin_src python
print("Hello world!")
#+end_src

[nnicandro]

It would be possible to consider major-mode-remap-alist somewhere, but it seems more of a user side issue that is already solved by modifying the value of org-src-lang-modes as you have done.

[ed9w2in6]

I am using major-mode-remap-alist as well and this issue manifests itself differently. Basically whenever I tried to indent (or insert newline with RET, which will try to indent) the src block will try to open in a dedicated buffer. I am not sure if this is org-mode's or emacs-jupyter's mechanism. (see #488) @nnicandro May I ask which is it?

It was quite painful to debug as it is not immediately clear where the issue originates from. Also slightly annoying as I must use the dedicated buffer to edit the src blocks.

FWIW, as a quick hack you can replace references to lang in the function org-babel-jupyter-make-language-alias with:

(substring (symbol-name
            (alist-get
             (intern (concat "python" "-mode")) ; can we do (concat lang "-mode") instead?
             major-mode-remap-alist))
           0 -5)

If it is not too troublesome I would like emacs-jupyter to respect major-mode-remap-alist, in which we can follow a similar logic as the above code.

BTW, a few packages out there (e.g. rustic, ob-sql-mode, etc.) also did not consider major-mode-remap-alist in which similar issue may appear. I think they should though on the assumption that treesit modes will find more adaptation in the future, and that major-mode-remap-alist is the only way to configure it.

[joostkremers]

It would be possible to consider major-mode-remap-alist somewhere, but it seems more of a user side issue that is already solved by modifying the value of org-src-lang-modes as you have done.

I ran into this issue as well and tried to remedy it by adding

(setf (alist-get "jupyter-python" org-src-lang-modes nil nil #'equal) 'python-ts)

to my .init.el. That appears to work well with new source blocks, but the error reappears as soon as I edit an existing source block. Examining org-src-lang-modes, I found that an additional entry for jupyter-python had been added:

Elisp> org-src-lang-modes
(("jupyter-python" . python)
 ("jupyter-python" . python-ts)
 ("C" . c)
 ("C++" . c++)
 ("asymptote" . asy)
 ("bash" . sh)
 ("beamer" . latex)
 ("calc" . fundamental)
 ("cpp" . c++)
 ("ditaa" . artist)
 ("desktop" . conf-desktop)
 ("dot" . fundamental)
 ("elisp" . emacs-lisp)
 ("ocaml" . tuareg)
 ("screen" . shell-script)
 ("shell" . sh)
 ("sqlite" . sql)
 ("toml" . conf-toml))

I don't know where or when this happens (haven't investigated), but it does suggest that a more principled solution is necessary.

[ed9w2in6]

@joostkremers Have you tried my hack? Try edit org-babel-jupyter-make-language-alias directly, which should be the function that added the "additional entry" in your case. Replace lang at the end of the function with:

(substring (symbol-name
            (alist-get
             (intern (concat "python" "-mode")) ; can we do (concat lang "-mode") instead?
             major-mode-remap-alist))
           0 -5)

The expression should just evals to python-ts if you have (python-mode . python-ts-mode) in your major-mode-remap-alist. You can just usepython-ts too if you wish.

[joostkremers]

@ed9w2in6 , no I haven't yet. I took a quick look at the function but it was too long to grasp in the little time I had... Is your suggestion to replace all occurrences of lang with your snippet of code (in which case I would just let-bind it around the entire body of the function), or just the last occurrence? And I guess it means it's no longer possible to use emacs-jupyter with languages other than Python, right? (That's not an issue for me, just asking.)

[ed9w2in6]

@joostkremers No, NOT all occurrences but just the VERY LAST ONE, followed by all the parenthesis as shown in the following code block:

https://github.com/emacs-jupyter/jupyter/blob/da306a6dbda6f1e285281765a311938a1d9db022/ob-jupyter.el#L726-L730

Actually, if you replace "python" in my hack with the param lang it might work for other languages as well. The caveat being that I haven't checked to make sure that this is true. I have edited my snippet the emphasise this fact.

Alternatively, you can manually add an entry to org-src-lang-modes as ("python" . python-ts) in your config, which according the the code block above, it will respect your config.

[joostkremers]

@ed9w2in6 Thanks, but I just realised your snippet is not going to work in my case, since I don't use major-mode-remap-alist, at least not for treesitter-enabled modes. Instead, I use treesit-auto to add entries to auto-mode-alist. So your snippet actually triggers an error, because alist-get returns nil.

Instead, I've added ("python" . python-ts) to org-src-lang-modes, which for now seems to do the trick. The reason it wasn't working before is because I used "jupyter-python" instead of "python". :facepalm:

[ed9w2in6]

@joostkremers I use treesit-auto too, and treesit-auto uses major-mode-remap-alist. (ref: https://github.com/renzmann/treesit-auto)

But anyways, any hack that works is a good hack.

Perhaps I should spend some time to read the code base to confirm that (concat lang "-mode") will always work. If it does a pull request will do the trick.