Open max-arnold opened 9 months ago
A little feedback on Ellama:
C-u
?llm-chat-streaming
but not llm-chat-async
Hi @max-arnold
A little feedback on Ellama:
* An easy provider selection (from a list of predefined ones) would be nice. Maybe with `C-u`?
Planned.
* Is there a way to use async API calls like this? https://cloud.yandex.com/en/docs/yandexgpt/api-ref/v1/TextGenerationAsync/completion Ellama only calls `llm-chat-streaming` but not `llm-chat-async`
Could you explain use case for this?
* Temperature setting and system role prompt could be useful as well
Can be set in llm provider. I don't think it will be useful for interactive usage.
* Is there a way to use async API calls like this? https://cloud.yandex.com/en/docs/yandexgpt/api-ref/v1/TextGenerationAsync/completion Ellama only calls `llm-chat-streaming` but not `llm-chat-async`
Could you explain use case for this?
I'm not 100% sure that it is needed, but an async chat is a part of the llm library API. Also, here is stated that async responses are cheaper and more precise: https://cloud.yandex.ru/docs/yandexgpt/concepts/#working-mode
Sure, but ellama is a tool for interactive usage. For other use cases may be other packages. And I prefer python for this cases because of more robust ecosystem in ai domain.
Speaking of Python, maybe writing various LLM plugins in Elisp is not very practical, and a completely different approach would be to write an Elisp wrapper for some popular library which supports many backends...
The tradeoff is that it will be more complex to install and update
And support.
I have a couple ideas. But this will be implemented in other packages. I don't think that wrappers around python is the way to go.
This is a Emacs LLM provider that supports YandexGPT API
What is missing for initial release:
FYI: @ahyatt @s-kostyaev