Open LondonX opened 8 months ago
Hi,
Sorry, I don't have an answer ready yet, just wanted to let you know I've seen your question.
I'd like to say that I'll get back to you soon but I honestly have no idea when.
It's fair to say that I should solve (or answer) the earlier reported issues first before coming back to you with an answer. I hope you understand.
I will update aub_ai to sync with the latest llama.cpp changes this week, mainly due to support Google's new Gemma model. Not sure if aub_ai will have bindings for create_chat_completion, it depends on whether this method is coming from llama.cpp directly (not llama-cpp-python).
Thanks, Daniel
Hi,
Some of the models in Hugeface shows the support of
create_chat_completion
, but now this plugin seems only support theSimple inference
, willChat Completion
be supported in the future version?https://huggingface.co/TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF