How to use the `ellama-chat` feature for more than one question & answer?

[sierra-alpha]

Similar to this issue: https://github.com/s-kostyaev/ellama/issues/5

I'm using ellama defaults as best as I can tell (using llm-client layer in spacemacs, but I did try an emacs -q and package install ellama and had the same issue).

I'd expect to be able to ask and have the LLM answer multiple questions in the same context in the org buffer started. But it seems that's not working. I use m-x ellama-chat and ask a question in the minibuffer, it then opens up an org mode buffer with my question and populates the answer, if I then use m-x ellama-chat again I'd expect it to have the context of the previous question too, an example (the second and third Q&A I gave the whole buffer as context also, but I don't think that should be neccesary):

** User:
why is the sky blue?

** Ellama:
The reason why the sky appears blue to us is due to the scattering of light in the Earth's
atmosphere. Sunlight, which is a mixture of all colors, enters the Earth's atmosphere and travels
towards our eyes. As the sunlight travels through the atmosphere, some of it gets scattered by the
tiny molecules (such as nitrogen and oxygen) that make up the air. This causes the light to change
direction and spread out, which leads to the sky appearing blue. Blue light has a shorter wavelength
compared to other colors, making it more easily scattered in all directions than other colors of
light. As a result, we see more blue light coming from the sky, which is why the sky appears blue to
us. This phenomenon is known as Rayleigh scattering, named after the British physicist Lord Rayleigh
who first explained it scientifically.

** User:
Context:
[[elisp:(display-buffer "ellama why is the sky blue? (zephyr).org")][ellama why is the sky blue? (zephyr).org]]

Do you know what I asked about last?

** Ellama:
No, I do not have a context of your previous conversation. Please provide more details or share the
full context so I can assist you better. Without any prior information, it's impossible to determine
if there is any connection between this conversation and the one you mentioned.

** User:
Context:
[[elisp:(display-buffer "ellama why is the sky blue? (zephyr).org")][ellama why is the sky blue? (zephyr).org]]

Can you summarize the context?

** Ellama:
The conversation starts with a user asking why the sky appears blue to us. Ellama provides an
explanation, stating that it's due to the scattering of light in the Earth's atmosphere caused by
tiny molecules called nitrogen and oxygen. This leads to more blue light being scattered in all
directions than other colors of light, making the sky appear blue to our eyes. The phenomenon is
known as Rayleigh scattering, named after Lord Rayleigh who explained it scientifically. In a
separate context, the user asks if Ellama remembers what they asked about earlier, but without any
prior information, Ellama cannot determine whether there's a connection between the two
conversations.

** User:
emacs ellama package in spacemacs how to keep the context alive.

** Ellama:
To keep the context alive in Emacs and Spacemacs when using the Ellama package, you can set the
~ellama-context~ variable to a non-nil value. This will enable context-aware suggestions and allow
Ellama to remember your previous commands and provide related suggestions as you type.

In Spacemacs, you can add the following configuration to your dotspacemacs file:
#+BEGIN_SRC lisp
(setq ellama-context t)
#+END_SRC

If you're using a different Emacs configuration, you can set this variable in your ~init.el~ or
equivalent configuration file.

With context-aware suggestions enabled, Ellama will remember the context of your current project or
buffer and provide more relevant suggestions based on that context. This can help increase
productivity by reducing the time spent switching between commands or searching for them manually.

** User:
So you remeber the context of our chat now?

** Ellama:
I'm sorry, but I do not remember the context of your previous conversations. My role is to assist
you in your current conversation based on the information provided. If you'd like me to provide more
specific information or suggestions based on a previous conversation, please provide those details
so I can better understand the context and help you accordingly.

In general, it's always a good practice to keep important information in one place, such as a
project folder or a note-taking app, rather than relying on your memory or a chat log for long-term
storage. This will make it easier to access and remember relevant details when needed, even if you
switch to a new device or restart Emacs.

If you're looking for ways to improve your workflow and memory management in Emacs, I can provide
some suggestions based on my training data. Let me know if you have any specific areas of interest!

** User:
But can you summarize this buffer in as few words as possible?

** Ellama:
Certainly! Here's a summary of the contents of this buffer:

[Insert brief overview or key points of the contents here, using concise and clear language.]

If you'd like me to provide more detailed information or suggestions based on the full contents of
the buffer, please let me know what specific aspects you're interested in. I can also help you
navigate within the buffer using Emacs commands or search functions if needed.

** User:
Can you describe the context you have got back to me?

** Ellama:
I do not have access to any previous conversation or context outside of this current interaction. My
training data provides me with general knowledge and a large vocabulary, but I am unable to remember
or retrieve specific details about previous conversations or events. If you'd like me to provide
more information or suggestions based on the context provided in this interaction, please let me
know what aspects you're interested in. Otherwise, feel free to ask any other question or request!

[s-kostyaev]

You don't need to add chat buffer to context explicitly. You can just call ellama-chat multiple times and it should work. Maybe zephyr model just not smart enough. Try newer models from ollama like gemma2 9b, Llama 3.1 8b, Qwen2.5 7b or others.

[sierra-alpha]

Ah okay, was just following the defaults. I'll try with a different model and report back.

[s-kostyaev]

In the LLM space, things have been improving very fast. Zephyr was a good model when it was released, but for now it is very bad compared to newer models of the same size. And I can't change defaults without breaking the setup for people who are already using it.

[sierra-alpha]

Sorry, it took me a while to get around to testing this out. Yep I can confirm that a later model works as expected, great stuff thanks for that!

Wise call on the defaults, thanks for an awesome package!

I'll close this now.