Mintplex-Labs / anything-llm

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, and more.
https://anythingllm.com
MIT License
27.48k stars 2.77k forks source link

Use workspace LLM to create thread name #2701

Closed scottybo closed 16 minutes ago

scottybo commented 6 hours ago

Pull Request Type

Relevant Issues

resolves #2699

What is in this change?

Uses the workspace's chosen LLM to rename the thread title to something more descriptive, rather than the first sentence of the conversation.

Additional Information

Developer Validations

timothycarambat commented 16 minutes ago

https://github.com/Mintplex-Labs/anything-llm/issues/2262#issuecomment-2344463292

The above comment is the primary reason why we dont support this. We offer an application on many various devices and not all of them are capable of concurrent generation or have the resources to do so - which is why the naming is static now since the latency could be high for low-end machines and the UX is not worth it considering that use-case.

Perhaps this could be something that could be done via a system toggle for appearance? This PR in its current form though does not enable that and should not make use of the APIChatHandler as while that works, is a different system that shouldn't be used for text generation in the backend