search

cztomsik / ava

All-in-one desktop app for running LLMs locally.
https://avapls.com
Other
400 stars 15 forks source link

Automatic conversation titles #1

Open derikk opened 9 months ago

derikk commented 9 months ago

ChatGPT creates a title for each new conversation after the first AI response. Perhaps this could be implemented in Ava as well? The prompt is something like "Summarize the above conversation to create a title for the conversation in exactly 1-4 words, in the same language as the conversation."

cztomsik commented 9 months ago

Thanks for opening this, it was doing exactly this before but it's very hard to find correct prompt for different models. Especially the smaller ones are super-sensitive to any prompt changes. So in the end, I left it out for later.

Edit: if anyone wants to pick it up, the further I got was something like:

USER: Can you summarize my request in one sentence, suitable as title?
ASSISTANT: Sure! Here's the conversation summarized in one sentence, which could be used as a title:

And then the model sometimes got it right, sometimes it was okish and sometimes it was plain wrong. Also, it might be simpler to do with JSON sampling.

pmarreck commented 9 months ago

I'd say "Summarize this conversation as a short document title. Do not editorialize, only output the best title for this interaction."