[Bug] lsp-request: Timeout while waiting for response

On Emacs 29.1 I upgraded lsp-mode from commit 11c0c7af611 (2022-05-18) to latest commit ee58d9eb6 (2023-08-09), and now when I try to use xref-find-definitions on any Python file in any project the command stalls for awhile before failing with this message:

lsp-request: Timeout while waiting for response.  Method: textDocument/definition

Highlighting of items under cursor is also not working, so I think something related to lsp parsing broke.

I tried updating xref and all other packages to latest, no change. The same command with lsp-dart works fine.

I am not sure if this is a bug in lsp-pyright or lsp-mode, but since it seems to only affect lsp-pyright I figured I'd post here first.

Cause

I have not analyzed the root cause, but I found the breaking lsp-mode commit using git bisect and restarting Emacs on each commit (simply rebuilding lsp-mode and restarting the lsp workspace did not trigger the problem when starting from a good commit and moving to a bad commit).

dfda673decbc82fc23079b897915f23f04c02a1c is the first bad commit
commit dfda673decbc82fc23079b897915f23f04c02a1c
Author: Ivan Yonchovski <yyoncho@users.noreply.github.com>
Date:   Mon Oct 31 14:37:12 2022 +0200
    Revert "Dispatch each lsp--parser-on-message in next message loop (#3087)" (#3791)

    This reverts commit e64f604ea405fc26f756fa960f6ac3ad473bd7f9.
 lsp-mode.el                  | 18 ++++--------------
 test/lsp-integration-test.el |  1 -
 2 files changed, 4 insertions(+), 15 deletions(-)

Using any commit before this one (including the one just before, 31645f9fc) does not break xref or item under cursor highlighting, even with the latest lsp-pyright (cea14f6) and pyright:

> ~/.emacs.d/.cache/lsp/npm/pyright/bin/pyright --version
pyright 1.1.321

Hopefully this bit of info will give a clue for someone more familiar with lsp-mode internals.

emacs-lsp / lsp-pyright

[Bug] lsp-request: Timeout while waiting for response #86

Cause