search

Mozilla-Ocho / llamafile

Distribute and run LLMs with a single file.
https://llamafile.ai
Other
16.81k stars 833 forks source link

Out of spec `/completions` endpoint #410

Closed yourbuddyconner closed 1 month ago

yourbuddyconner commented 1 month ago

I have some "legacy" code that still uses the openai.completion.create endpoint. It makes a POST request to /v1/completions endpoint.

llamafile supports a /completion endpoint which I assume is compatible however is out of spec with the openai SDK by default.

Probably an easy fix!

yourbuddyconner commented 1 month ago

If anyone runs across this -- it's because of the dependency on llama.cpp which only has a chat completions endpoint.