s-kostyaev / ellama

Ellama is a tool for interacting with large language models from Emacs.
GNU General Public License v3.0
520 stars 33 forks source link

Emacs unresponsive when using ellama-chat #170

Open ArthurHeymans opened 1 day ago

ArthurHeymans commented 1 day ago

When using ellama chat (in this case with gpt4-o), emacs becomes completely unresponsive a few times. When emacs becomes responsive again a lof of the llm response is suddenly filled in. Including those unresponsive times it takes a lot longer to get a response.

s-kostyaev commented 1 day ago

@ahyatt do you have any ideas?

ahyatt commented 1 day ago

I wonder if something might be happening where some elisp running in Emacs is not relinquishing control back to Emacs, causing it not to be able to update the UI. @ArthurHeymans is there anything else you are in doing in the background in Emacs that might be interfering? Or is there any other clues you might have about how to reproduce this?

ArthurHeymans commented 1 day ago

So with vanilla emacs I don't have this problem, but with my configuration of doom emacs it is a problem. https://github.com/ArthurHeymans/.doom.d/blob/main/config.org is my configuration btw.

Weirdly enough if I use doom emacs with https://github.com/marienz/nix-doom-emacs-unstraightened on nixos the problem is also not present.

s-kostyaev commented 1 day ago

Looks like it's a problem with your configuration. Try to localize it first.

ArthurHeymans commented 21 hours ago

Ok I figured it out: By default the sessions were placed in "#doomdir/.local/cache/ellama-sessions" . Doom emacs has a separate mode for org-mode files in there, which does not work well when text is being streamed to it.. Not using doom emacs or using nix had those session files in a different place. Setting ellama-sessions-directory some place else fixed it.

Do you think this worth mentioning in the ellama readme as doom emacs is rather popular? I can send a PR if you think it's a good idea

s-kostyaev commented 21 hours ago

I think this makes sense. It's interesting why this mode so resource heavy. Maybe it's worth also raise an issue in doom emacs repo.