search

lmg-anon / mikupad

LLM Frontend in a single html file
https://lmg-anon.github.io/mikupad/mikupad.html
Creative Commons Zero v1.0 Universal
175 stars 24 forks source link

Chat completions support #65

Open lmg-anon opened 1 month ago

lmg-anon commented 1 month ago

I'm not entirely sure if this is useful, but the recent instruct templates feature gave me the idea for this new feature. Currently it's enabled with chat mode, but it might be optimal to make a separate checkbox for it.

image

neCo2 commented 1 month ago

What's the difference to normal completions? Does the back-end handle adding the correct instruct template?

lmg-anon commented 1 month ago

What's the difference to normal completions? Does the back-end handle adding the correct instruct template?

That's correct, but the main point is allowing the use of endpoints that only support chat completions, such as the endpoint for most OpenAI models. However, I guess the OpenAI endpoint isn't great for this because it doesn't support pre-filling.

Edit: Actually, the OpenAI API does seem to handle prefills well, interesting... image