search

s-kostyaev / ellama

Ellama is a tool for interacting with large language models from Emacs.
GNU General Public License v3.0
378 stars 27 forks source link

Would it be possible to reload or reference a prior session? #18

Closed stephenwithav closed 6 months ago

stephenwithav commented 7 months ago

I'm a solo developer so I decided to test a daily standup with ellama this morning (ollama backend). Would it be possible to resume this session tomorrow?

I'm assuming not because Ollama's responses aren't static, but it would be kinda cool.

s-kostyaev commented 7 months ago

Not yet. But it's possible in principle. I will think about it.

s-kostyaev commented 7 months ago

Could you explain your use case more?

tusharhero commented 7 months ago

I am working around that by saving the session's log on my disk. When I come back I say hi to the LLM, and then copy paste it in the *ellama* buffer but I think it could be much more streamlined.

s-kostyaev commented 7 months ago

I will see, how to implement it. Thank you for request.

stephenwithav commented 7 months ago

Could you explain your use case more?

Here's a gist of this morning's fake standup. If promptnig in the morning with ellama-ask with yesterday's conversation allowed me to resume, it would be helpful.

The context window for llama2 maxes out at 4096, I think, so this may be a useless request.

Do you paste the whole log, @tusharhero? or do you C-x h M-x ellama-ask-about?

stephenwithav commented 7 months ago

Ollama supports LoRAs. Would it be possible to add to the LoRA after each interaction?

s-kostyaev commented 7 months ago

Ollama supports LoRAs. Would it be possible to add to the LoRA after each interaction?

I don't think it will work inside one session without resending previous messages.

stephenwithav commented 7 months ago

I don't think it will work inside one session without resending previous messages.

You're right that LoRAs aren't the answer; I was thinking about that in bed this morning. LoRAs are more for creating a derivative model. They're cool, but not quite what we need here.

tusharhero commented 7 months ago

Do you paste the whole log, @tusharhero? or do you C-x h M-x ellama-ask-about?

I just paste the whole thing.

s-kostyaev commented 7 months ago

@stephenwithav Feature to manage ellama sessions is planned. I will think more how to implement it. For now you can try to save your standups into file, open it, write new question and use new command ellama-complete - it will continue your buffer. Not sure if it will works good, but you can try and return with feedback.

stephenwithav commented 7 months ago

Thanks, @s-kostyaev. I'll test when it's available in MELPA.

s-kostyaev commented 7 months ago

Thanks, @s-kostyaev. I'll test when it's available in MELPA.

It's there. Update ellama and try out.

stephenwithav commented 7 months ago

It's there. Update ellama and try out.

Doom Emacs was the culprit. I didn't realize doom sync required a -u flag to update to the newest packages.

My current approach is here, so I'll test tomorrow.

Do you still want me to try pasting in the previous days standup each morning?

s-kostyaev commented 7 months ago

Try to open file with your standups, add what you want and call ellama-complete to get response from model. Maybe you need to insert model nick first, but you can write elisp helper for that. And return wit feedback. I would like to know if it works for you.

stephenwithav commented 7 months ago

Thanks for the tip, I'll try that.

One thing I find useful is to just M-: and evaluate the following s-expression: (while t (call-interactively #'ellama-ask)). It's as close as I can get to a Bard/ChatGPT-like experience. Is there a plan to incorporate something like that?

s-kostyaev commented 7 months ago

M-x ellama-ask-selection

stephenwithav commented 7 months ago

M-x ellama-ask-selection

This just summarized the existing selection.

ellama-complete did as well.

s-kostyaev commented 7 months ago

Bad. So, you can wait when I will implement proper session saving and loading. It will take some time. Thank you for feedback.

s-kostyaev commented 6 months ago

Fixed in #49

Feedback is welcome

stephenwithav commented 6 months ago

The session management definitely has potential. While I'm unable to resume earlier sessions, it's useful to have a history of earlier chats.

My initial thoughts:

  1. Why use doom-docs-mode instead of org-mode? The Suggest edits and Help links don't work.
  2. Running concurrent sessions would be useful.
  3. Adding the date to the session filename, similar to how org-journal works, would be useful. Each daily standup conversation begins with "Good morning", so C-c e a i reloads the earlier session if I accidentally save it.
  4. Scratch-buffer style, disposable chats would be useful.
  5. ellama-code-review buffers aren't interactive. Should they be?
s-kostyaev commented 6 months ago
  1. You definitely can resume previous sessions with ellama-load-session.
  2. This in your configuration. It's just plain org file.
  3. You can have multiple active sessions. You can start new one with C-u ellama-chat
  4. I will think about it.
  5. I don't understand. Will search it.
  6. No. But maybe should be changed. Thank you for feedback!
stephenwithav commented 6 months ago
  1. Doom Emacs uses .org files in ~/.emacs.d/.local/cache as Doom Docs. Changing ellama-sessions-directory to ~/.emacs.d/.ellama-sessions resolved this issue.
  2. Doom Emacs uses SPC u as an alternative to C-u... I didn't realize that.
  3. Thanks!
  4. The default *scratch* buffer is never saved due to changes. It would be useful, imo, for ellama to have similar buffers that are never saved. Suppose you can't remember something, like the name of/keybinding from a new-to-you-package, a quick C-c e i d for a disposable buffer could be helpful.

Thank you for ellama!