Autocompletion with Elpy is broken (garbled text) within EIN notebooks

Problem description

When I attempt to use Elpy's autocomplete in EIN notebooks, if I continue typing the name of an autocompleted symbol after the autocompletion has started, the symbol ends up totally garbled. In particular, it appears that the letters from the autocompletion are not removed/deleted as I continue typing, and the new autocompletion for the remaining letters keeps showing up (with highlighting). Furthemore, it is not possible to delete these autocompleted and highlighted letters. To get them to disappear, the notebook must be closed and reopened.

Screen Shot 2021-04-16 at 12 06 12 PM Screen Shot 2021-04-16 at 12 27 50 PM

Steps to reproduce the problem

From a clean init.el, install Elpy and EIN using package-install.
Call elpy-enable.
Create a new python file (say, elpy_test.py). This is to set up Elpy.
When prompted to install dependencies for completion etc. in the Elpy RPC virtual environment, answer yes.
Call ein:run and create a new notebook.
Get autocompletion to start. For example, you can type import itertools in the first cell, execute it, then start typing itertools in the next cell.
Continue to type the autocompleted name (itertools in our example). The garbled autocompletion will appear. Even if you delete the rest of the symbol name, the autocompleted letters will remain.

System info:

("EIN system info"
 :emacs-version "GNU Emacs 27.2 (build 1, x86_64-apple-darwin18.7.0)
 of 2021-03-25"
 :window-system nil
 :emacs-variant nil
 :build
 "--disable-silent-rules --enable-locallisppath=/usr/local/share/emacs/site-lisp --infodir=/usr/local/Cellar/emacs/27.2/share/info/emacs --prefix=/usr/local/Cellar/emacs/27.2 --with-gnutls --without-x --with-xml2 --without-dbus --with-modules --without-ns --without-imagemagick --without-selinux"
 :os
 (:uname
  "Darwin 38f9d3645774 18.7.0 Darwin Kernel Version 18.7.0: Tue Jan 12 22:04:47 PST 2021; root:xnu-4903.278.56~1/RELEASE_X86_64 x86_64
"
  :lsb-release nil)
 :jupyter
 "jupyter core     : 4.6.3
jupyter-notebook : 6.1.3
qtconsole        : 4.7.6
ipython          : 7.17.0
ipykernel        : 5.3.4
jupyter client   : 6.1.6
jupyter lab      : not installed
nbconvert        : 5.6.1
ipywidgets       : 7.5.1
nbformat         : 5.0.7
traitlets        : 4.3.3
"
 :image-types nil
 :image-types-available nil
 :request-backend curl
 :ein
 (:version "20210330.1531"
       :source-dir "/Users/tristamc/.emacs.2.d/elpa/ein-20210330.1531/")
 :lib
 ((:name "websocket"
     :path "~/.emacs.2.d/elpa/websocket-20210110.17/websocket.elc"
     :featurep t
     :version-var websocket-version
     :version "1.12")
  (:name "anaphora"
     :path "~/.emacs.2.d/elpa/anaphora-20180618.2200/anaphora.elc"
     :featurep t
     :version-var nil
     :version nil)
  (:name "request"
     :path "~/.emacs.2.d/elpa/request-20210410.2218/request.elc"
     :featurep t
     :version-var request-version
     :version "0.3.3")
  (:name "deferred"
     :path "~/.emacs.2.d/elpa/deferred-20170901.1330/deferred.elc"
     :featurep t
     :version-var deferred:version
     :version "0.5.0")
  (:name "polymode"
     :path "~/.emacs.2.d/elpa/polymode-20210413.2004/polymode.elc"
     :featurep t
     :version-var nil
     :version nil)
  (:name "dash"
     :path "~/.emacs.2.d/elpa/dash-20210330.1544/dash.elc"
     :featurep t
     :version-var nil
     :version nil)
  (:name "with-editor"
     :path "~/.emacs.2.d/elpa/with-editor-20210319.1930/with-editor.elc"
     :featurep nil
     :version-var nil
     :version nil)))

Logs:

ein:log-all:

12:05:10:905: [info] ein:jupyter-server--run: /usr/local/bin/jupyter notebook --notebook-dir=~/.emacs.2.d/ --no-browser @#<buffer .emacs.2.d>
12:05:13:008: [info] ein:jupyter-process-lines: jupyter exited with status 1 @#<buffer .emacs.2.d>
12:05:13:191: [info] Login to http://127.0.0.1:8888 complete. @#<buffer .emacs.2.d>
12:05:13:287: [verbose] Opening notebooklist at http://127.0.0.1:8888 @#<buffer .emacs.2.d>
12:05:15:758: [verbose] Opened notebook Untitled.ipynb @#<buffer *ein:notebooklist http://127.0.0.1:8888*>
12:05:15:783: [verbose] Opening notebooklist at http://127.0.0.1:8888 @#<buffer *ein:notebooklist http://127.0.0.1:8888*>
12:05:15:894: [verbose] ein:kernel-retrieve-session--success: kernel-id=f61e52f4-005b-4b1c-ac88-553db82f3444 session-id=7f63269f-18d9-4853-9a63-005b29cb026b @#<buffer *ein:notebooklist http://127.0.0.1:8888*>
12:05:15:894: [verbose] WS start: ws://127.0.0.1:8888/api/kernels/f61e52f4-005b-4b1c-ac88-553db82f3444/channels?session_id=7f63269f-18d9-4853-9a63-005b29cb026b @#<buffer *ein:notebooklist http://127.0.0.1:8888*>
12:05:16:649: [info] Notebook Untitled.ipynb is ready @#<buffer *ein:notebooklist http://127.0.0.1:8888*>
12:05:16:649: [verbose] WS opened: ws://127.0.0.1:8888/api/kernels/f61e52f4-005b-4b1c-ac88-553db82f3444/channels?session_id=7f63269f-18d9-4853-9a63-005b29cb026b @#<buffer *ein:notebooklist http://127.0.0.1:8888*>
12:05:16:670: [verbose] Notebook is saved. @#<buffer  *ein: http://127.0.0.1:8888/Untitled.ipynb*[python]>
12:05:16:673: [verbose] Opening notebooklist at http://127.0.0.1:8888 @#<buffer  *ein: http://127.0.0.1:8888/Untitled.ipynb*[python]>
12:06:32:473: [warn] ein:dev-packages: Don’t call me! @#<buffer  *temp*>

ein server:

[I 12:05:12.433 NotebookApp] Serving notebooks from local directory: /Users/tristamc/.emacs.2.d
[I 12:05:12.434 NotebookApp] Jupyter Notebook 6.1.3 is running at:
[I 12:05:12.434 NotebookApp] http://localhost:8888/?token=6e82bd7576aaddc5949deaa2ca3e9aa7a52c98e30655054e
[I 12:05:12.434 NotebookApp]  or http://127.0.0.1:8888/?token=6e82bd7576aaddc5949deaa2ca3e9aa7a52c98e30655054e
[I 12:05:12.434 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 12:05:12.448 NotebookApp] 

    To access the notebook, open this file in a browser:
        file:///Users/tristamc/Library/Jupyter/runtime/nbserver-94022-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=6e82bd7576aaddc5949deaa2ca3e9aa7a52c98e30655054e
     or http://127.0.0.1:8888/?token=6e82bd7576aaddc5949deaa2ca3e9aa7a52c98e30655054e
[W 12:05:13.040 NotebookApp] 403 POST /login (127.0.0.1): '_xsrf' argument missing from POST
[W 12:05:13.123 NotebookApp] 403 POST /login (127.0.0.1) 83.12ms referer=None
[I 12:05:13.145 NotebookApp] 302 POST /login (127.0.0.1) 1.42ms
[I 12:05:13.146 NotebookApp] 302 GET / (127.0.0.1) 0.34ms
[I 12:05:15.690 NotebookApp] Creating new notebook in 
[I 12:05:15.884 NotebookApp] Kernel started: f61e52f4-005b-4b1c-ac88-553db82f3444, name: python3
[I 12:05:16.660 NotebookApp] Saving file at /Untitled.ipynb

Elpy Configuration

Emacs.............: 27.2 Elpy..............: 1.35.0 Virtualenv........: rpc-venv (/Users/tristamc/.emacs.2.d/elpy/rpc-venv) Interactive Python: python 3.8.6 (/Users/tristamc/.emacs.2.d/elpy/rpc-venv/bin/python) RPC virtualenv....: rpc-venv (/Users/tristamc/.emacs.2.d/elpy/rpc-venv) Python...........: python 3.8.6 (/Users/tristamc/.emacs.2.d/elpy/rpc-venv/bin/python) Jedi.............: 0.18.0 Rope.............: 0.18.0 Autopep8.........: 1.5.6 Yapf.............: 0.31.0 Black............: 20.8b1 Syntax checker....: flake8 (/Users/tristamc/.emacs.2.d/elpy/rpc-venv/bin/flake8)

Warnings

Options

Square brackets indicate buttons; type RET or click mouse-1 on a button to invoke its action. Invoke [+] to expand a group, and [-] to collapse an expanded group. Invoke the [Group], [Face], and [Option] buttons below to edit that item in another window.

[+]-- [Group] Elpy [+]-- [Group] Python [+]-- [Group] Virtual Environments (Pyvenv) [+]-- [Group] Completion (Company) [+]-- [Group] Call Signatures (ElDoc) [+]-- [Group] Inline Errors (Flymake) [+]-- [Group] Code folding (hideshow) [+]-- [Group] Snippets (YASnippet) [+]-- [Group] Directory Grep (rgrep) [+]-- [Group] Search as You Type (ido) [+]-- [Group] Django extension [+]-- [Group] Autodoc extension

millejoh / emacs-ipython-notebook