Open abasu0713 opened 4 months ago
Not sure if it's the same, but I'm using koboldcpp - perhaps try using v1/completions
, not v1/chat/completions
?
Hi, it's indeed the /v1/completions
endpoint and not /v1/chat/completions
. Also, you shouldn't need to add the path anymore, you can set llm.url
as http[s]://{hostname}
This issue is stale because it has been open for 30 days with no activity.
Hi, it's indeed the
/v1/completions
endpoint and not/v1/chat/completions
. Also, you shouldn't need to add the path anymore, you can setllm.url
ashttp[s]://{hostname}
I am going to give this a try tomorrow and report back. And sorry I didn't get back sooner. I just saw a note from github notification. Thank you for the reply. I was using /v1/chat/completions
since I was using the llama-insruct models. Which require that endpoint no?
Which require that endpoint no?
They might yes, the extension doesn't support chat models atm. The model you use must be compatible with code completion, either with fill in the middle or not (but I strongly advise to use FIM as it generates more relevant completions).
This issue is stale because it has been open for 30 days with no activity.
Requesting a little help here. Trying to test out copilot functionality with
llama-cpp-python
with this extension. Below is my configuration setting.I am seeing there is inference going on the server:
So I am not entirely sure what I am missing. Additionally I am trying to see the extension logs.. for the worker calls. But I don't see anything. Would you be able to give any guidance or some step by step explanation on how this can be done.
Thank you so much