Python mode is unusable because of lag between key strokes

[tanj]

Description :octocat:

Python mode is unusable because of lag between key strokes

This happens with both anaconda backend and lsp backend

Reproduction guide :beetle:

• Start Emacs
• Open python file
• Navigate 5 lines down C-n C-n C-n C-n C-n (in rapid succession)

Observed behaviour: :eyes: :broken_heart: Emacs takes >10 seconds to move the five lines

Expected behaviour: :heart: :smile: Navigation will be immediate

System Info :computer:

• OS: windows-nt
• Emacs: 28.0.50
• Spacemacs: 0.300.0
• Spacemacs branch: develop (rev. 797360a41)
• Graphic display: t
• Distribution: spacemacs
• Editing style: emacs
• Completion: helm
• Layers:

  (sql php autohotkey html restructuredtext sphinx
   (rust :variables rust-format-on-save t)
   csv yaml
   (python :variables python-formatter 'black python-format-on-save t python-backend 'anaconda)
   (typescript :variables typescript-backend 'tide typescript-fmt-on-save t typescript-fmt-tool 'prettier)
   (latex :variables latex-build-engine 'luatex)
   auto-completion better-defaults emacs-lisp git
   (helm :variables helm-buffer-max-length 80)
   lsp markdown multiple-cursors org spell-checking syntax-checking treemacs)

• System configuration features: ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NATIVE_COMP NOTIFY W32NOTIFY PDUMPER PNG RSVG SOUND THREADS TOOLKIT_SCROLL_BARS XPM ZLIB

Profiler (with running poetry-venv-workon)

           3 100% - pyvenv-track-virtualenv
           3 100%  - pyvenv-activate
           3 100%   - apply
           3 100%    - #<subr pyvenv-activate>
           3 100%     - pyvenv-deactivate
           3 100%      - apply
           3 100%       - spacemacs/python-setup-everything
           3 100%        - apply
           3 100%         - spacemacs//python-setup-shell
           3 100%          - if
           2  66%           - spacemacs/pyenv-executable-find
           2  66%            - if
           2  66%               executable-find
           1  33%           - progn
           1  33%            - if
           1  33%             - version<
           1  33%              - replace-regexp-in-string
           1  33%               - shell-command-to-string
           1  33%                - format
           1  33%                 - string-trim
           1  33%                  - spacemacs/pyenv-executable-find
           1  33%                   - if
           1  33%                      executable-find
           0   0% - ...
           0   0%    Automatic GC

Profiler (without running poetry-venv-workon)

           2 100% - ...
           2 100%  - pyvenv-deactivate
           2 100%   - apply
           2 100%    - spacemacs/python-setup-everything
           2 100%     - apply
           2 100%      - spacemacs//python-setup-shell
           2 100%       - if
           2 100%        - progn
           2 100%         - if
           2 100%          - version<
           2 100%           - replace-regexp-in-string
           2 100%            - shell-command-to-string
           2 100%             - format
           2 100%              - string-trim
           2 100%               - spacemacs/pyenv-executable-find
           2 100%                - if
           2 100%                   executable-find
           0   0%    Automatic GC

[ryanprior]

Tried to reproduce this on GNU/Linux with Emacs 27, no dice.

It seems significant to me that this appears on Windows and that the trace shows a lot of time in executable-find. I used Emacs on Windows years ago and found that things which had to find and run many small programs (especially magit) were unbearably slow, but largely went unnoticed because the same routines are quite lightweight on Mac & GNU/Linux.

If we're running executable-find on every line change though, that seems excessive in any case & there's perhaps a hook being abused or something.

[tanj]

I pulled the most recent version of develop 4f7246da07e7eb1e07a26de23a80cd2d89a89a7c and as of this revision I am no longer having this issue. I'm not sure when whatever it was fixed the issue. I was at 63056ecb50f93808781b97feab1c3225d35c7aa1 before this pull, but I don't recall if I checked if LSP was working without issue while on that rev.

[tanj]

I think I found the root cause. Yesterday when I wasn't having the issues I was testing python lsp-mode on a python file that wasn't using a virtual env. This morning I went to work on a project that was using a virtual env and the issue was back.

After poking around, I found that spacemacs was looking for a virtualenv folder called .venv. I had tried out vscode and the docs for that had said to put the virtual env in a folder called venv. I renamed the folder from venv to .venv and the lag issue was resolved.

[ryanprior]

Wow, sounds like a bug. Thanks for digging further into it! Can you document full steps to reproduce the issue, eg with a "fresh" spacemacs install & small python project?

[tanj]

Sorry for the long delay. I've not had time to look into reproducing this from fresh.

I tried to recreate this in a fresh install and a clean(er) development env and wasn't able to. Not sure if it is some quirk of my usual dev machine or what.

[ryanprior]

Well thank you for trying anyway!

[idanov]

I have experienced a similar issue on macOS. I am using miniconda and had put a .venv file with the path to my conda environment, which made my python-mode experience unbearably slow. When I deleted the .venv file and restarted my layout, everything worked normally. The slowest bits ware command-execute and pyvenv-track-virtualenv when I did the profiling.

[idanov]

After a bit of more experimenting I found out that the problem occurs when .venv has an incorrect path to the environment. Using ~ in the file also makes the path incorrect, so apparently the path should be absolute.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please let us know if this issue is still valid!