Open Huge opened 10 months ago
Token probabilities are currently not available over the API. KoboldCpp only implements a core subset of all openai methods.
Alright. During my "investigation" I have noticed one bug, that the input schema disappears when "Try it out" button.
And https://github.com/swagger-api/swagger-ui/issues/6528#issuecomment-785837911 would be great to enable experimenting by default, when the bug above can be fixed.
OpenAI (re)added support for outputting probabilities of generated tokens to most of their LLMs: https://cookbook.openai.com/examples/using_logprobs
logprobs are ignored in my calls to http://localhost:5001/api#/v1/post_v1_generate
->