Open gorkem opened 6 months ago
Here is some more clarification on the requests.
POST /completion
is an API specific to llama.cpp server. There is example code available for its usage which we will adjust for code generation feature.
POST /v1/chat/completions
The endpoint for chat completion API is compatible with the OpenAI endpoint. We can use the existing OpenAPI libraries for code generation.
The initial thought was to reduce the options to match OpenAI. However, with further thinking and considering the options are supported by both endpoints. We should continue to use all options but categorize them better.
I have not really found a good reason to keep the multiple page. It is unintuitive to go back and forth for changing the parameters. I suggest we do a single page implementation.
--> We talked about how the generated code for chat mode can get really long and therefore messy. @gorkem I assume the first two lines are your solution. How would it look in the UI?
@annigro no, Gorkem's first two lines are related to internal code usage, and should be transparent for the UI other than one line. We talked about doing something like this for it:
message: [{ actor: 'user', content: 'foo bar fooz' }]
and when is too long:
message: [{ actor: 'user', content: 'foo bar ...' }] <-- ellipsis but for the message's content only, and only if is too long.
but clicking on "copy code" would always copy the whole thing, regardless of the ellipsis.
@gorkem i gave a try to the open ai
api but looks like both the payload and the response is different than the one we already have in llama.cpp (which makes sense). I vote to keep using the completion
llama.cpp endpoint instead or redoing everything including the api layer just to support openai v1 endpoints. Thoughts?
@gorkem Figma file to review
I did a pass left a few comments.
Describe the problem you're trying to solve Dev UI should be more helpful with helping the application developers integrate with the model.
Describe the solution you'd like