pyenv exited with status 127 when using pet-mode with emacs-jupyter

[fast-90]

Description

When using pet-mode with jupyter (emacs-jupyter), entering an org-edit-special buffer results in pyenv exited with status 127 [3 times] (as observed in the *Messages* buffer when turning on pet-debug).

This behaviour causes several issues for me when using jupyter notebooks in org-mode, in particular a few seconds of lag when I try to add a new line with RET or indent a line with TAB (as org-mode opens an org-edit-special buffer in the background with python-mode on to get the "correct" indentation behaviour).

Reproduction steps

1. Minimum config

init.el

```el (setq native-comp-async-report-warnings-errors nil) (use-package org :elpaca nil :init (org-babel-do-load-languages 'org-babel-load-languages '((emacs-lisp . t) (python . t))) :custom (org-confirm-babel-evaluate nil) (org-src-fontify-natively t) (org-edit-src-content-indentation 2)) (use-package jupyter :elpaca t :init (setq native-comp-jit-compilation-deny-list '("jupyter-channel.*.el")) ; Needed to ignore a recent bug on main (org-babel-do-load-languages 'org-babel-load-languages (append org-babel-load-languages '((jupyter . t)))) (setq org-babel-default-header-args:jupyter-python '((:async . "yes") (:session . "py") (:kernel . "python3")))) (use-package pet :elpaca t :hook (python-mode . python-pet-hook) :init (defun python-pet-hook () (pet-mode))) ```

2. Ensure you have Jupyter installed on your machine and have at least one basic working python3 kernel.
3. Open/create new notebook.

notebook.org (adjust kernel name in the notebook if needed)

```el :PROPERTIES: :header-args:python: :async yes :kernel python3 :session py :END: #+begin_src python print("Hello world!") #+end_src ```

4. Do one of the following: a. Press RET to add new line b. Press C-c ' to open an org-edit-special buffer c. Press TAB to indent a line
5. Observe the lagged behaviour (and the error message in *Messages* buffer if pet-debug is turned on)

Expected behavior

No lagged behaviour when doing any of the actions in step 4 above, no error message even when pet-debug is on. This expected behaviour is observed when pet-mode is removed from the python-mode-hook.

PET version 2023-06-21 version from MELPA

Emacs version GNU Emacs 29.0.91 (build 1, x86_64-apple-darwin22.3.0, NS appkit-2299.40 Version 13.2.1 (Build 22D68)) of 2023-06-05 (Installed via emacs-plus, although I have tested this on a different machine with WSL2 and observed the same behaviour).

Desktop

• OS: MacOS
• Version: Ventura 13.4 (22F66)

System tools versions

• pyenv v2.3.18

  Additional context

  Result of M-x (pet-verify-setup) in the org-edit-special buffer:

```conf python-shell-interpreter: ~/.pyenv/versions/3.10.5/bin/python python-shell-virtualenv-root: ~/.venv/ flycheck-python-flake8-executable: unbound flycheck-pylintrc: unbound flycheck-python-pylint-executable: unbound flycheck-python-mypy-executable: unbound flycheck-python-mypy-config: unbound flycheck-python-mypy-python-executable: unbound flycheck-python-pyright-executable: unbound flycheck-python-pycompile-executable: unbound lsp-jedi-executable-command: jedi-language-server lsp-pyls-plugins-jedi-environment: ~/.venv/ lsp-pylsp-plugins-jedi-environment: ~/.venv/ lsp-pyright-python-executable-cmd: ~/.pyenv/versions/3.10.5/bin/python lsp-pyright-venv-path: ~/.venv/ dap-python-executable: ~/.pyenv/versions/3.10.5/bin/python python-pytest-executable: pytest python-black-command: black blacken-executable: black python-isort-command: ~/.local/bin/isort yapfify-executable: ~/.local/bin/yapf eglot-server-programs (Python part only): (not available because eglot is not loaded now) exec-path: /usr/local/Cellar/pyenv-virtualenv/1.2.1/shims ~/.pyenv/shims ~/.pyenv/bin ~/.poetry/bin /usr/local/sbin /usr/local/opt/node@14/bin ~/.local/bin ~/bin /usr/local/bin /opt/local/bin /opt/local/sbin /Library/Frameworks/Python.framework/Versions/3.7/bin /Library/Frameworks/Python.framework/Versions/3.8/bin /usr/local/bin /System/Cryptexes/App/usr/bin /usr/bin /bin /usr/sbin /sbin /Library/TeX/texbin /Applications/quarto/bin /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin /var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin /usr/local/Cellar/pyenv-virtualenv/1.2.1/shims ~/.pyenv/bin ~/.poetry/bin /usr/local/sbin /usr/local/opt/node@14/bin ~/.local/bin ~/bin /opt/local/bin /opt/local/sbin /Library/Frameworks/Python.framework/Versions/3.7/bin /Library/Frameworks/Python.framework/Versions/3.8/bin ~/.cargo/bin ~/miniconda3/condabin ~/.erg/bin ~/.erg/bin /usr/local/Cellar/emacs-plus@29/29.0.90/libexec/emacs/29.0.91/x86_64-apple-darwin22.3.0 ```

Hope this is sufficiently clear, otherwise please let me know what other information I should provide. I can also provide a short video showing the lagged behaviour if required.

[wyuenho]

What's your project layout? Where did you install jupyter? Can you give me a trace of pet-executable-find and/or pet-virtualenv-root?

[fast-90]

Thanks for the suggestion to trace those functions! (Sorry still relatively new to Elisp).

See below for the trace. Reading it it seems that the error is caused by the executables it could not find (and therefore returning nil). This makes sense as I do not have those packages (e.g. black) installed in the project venv.

Do you have any suggestions on how to configure pet to avoid running python-pet-hook for org-src blocks? In my normal Python projects I do use these tools, so I would like to keep them in python-pet-hook. I just need them to not search for the executables for my Org notebooks.

I have tried the following without results:

(use-package pet
  :elpaca t
  :hook
  (python-mode . python-pet-hook)
  (org-src-mode. org-src-pet-hook)
  :init
  (defun python-pet-hook ()
    (unless (org-src-edit-buffer-p)
    (pet-mode)))
  (defun org-src-pet-hook ()
    (remove-hook 'python-mode-hook 'python-pet-hook t)))

I have also tried variants with org-in-src-block-p and (bound-and-true-p org-src-mode) without succes. Any suggestions?

Trace

``` ====================================================================== 1 -> (pet-executable-find "python") | 2 -> (pet-virtualenv-root) | 2 <- pet-virtualenv-root: "/Users/duynguyen/.venv/" 1 <- pet-executable-find: "/Users/duynguyen/.pyenv/versions/3.10.5/bin/python" ====================================================================== 1 -> (pet-virtualenv-root) 1 <- pet-virtualenv-root: "/Users/duynguyen/.venv/" ====================================================================== 1 -> (pet-executable-find "jedi-language-server") | 2 -> (pet-virtualenv-root) | 2 <- pet-virtualenv-root: "/Users/duynguyen/.venv/" 1 <- pet-executable-find: nil ====================================================================== 1 -> (pet-executable-find "pytest") | 2 -> (pet-virtualenv-root) | 2 <- pet-virtualenv-root: "/Users/duynguyen/.venv/" 1 <- pet-executable-find: nil ====================================================================== 1 -> (pet-executable-find "black") | 2 -> (pet-virtualenv-root) | 2 <- pet-virtualenv-root: "/Users/duynguyen/.venv/" 1 <- pet-executable-find: nil ====================================================================== 1 -> (pet-executable-find "isort") | 2 -> (pet-virtualenv-root) | 2 <- pet-virtualenv-root: "/Users/duynguyen/.venv/" 1 <- pet-executable-find: "/Users/duynguyen/.local/bin/isort" ====================================================================== 1 -> (pet-executable-find "yapf") | 2 -> (pet-virtualenv-root) | 2 <- pet-virtualenv-root: "/Users/duynguyen/.venv/" 1 <- pet-executable-find: "/Users/duynguyen/.local/bin/yapf" ```

[wyuenho]

I don't know. I don't use org-mode. AFAIK, similar modes like polymode and markdown-mode have(had) the same problem. This is a problem with how Emacs expose major mode switching lifecycle hooks and how these modes reuse major modes in source code blocks. There's nothing you can do about it at the moment, but perhaps you can file a ticket to Emacs.

[wyuenho]

Alternatively, you may try to detect whether you are in a project in your hook, and only activate pet mode when you are.

[fast-90]

Fair enough! I have a hacky workaround atm using (when buffer-file-name ...) (as those org-src-edit buffers don't have a file name), but will send an email to the org-mode maillist to ask for their advice.

In any case I'm closing this issue as it is not related to PET. Thanks for the replies, and thanks for this great package which is making my Python package development process a lot easier :)