search

LostRuins / koboldcpp

Run GGUF models easily with a KoboldAI UI. One File. Zero Install.
https://github.com/lostruins/koboldcpp
GNU Affero General Public License v3.0
4.66k stars 334 forks source link

Is there an API endpoint? #32

Closed MKwareContributions closed 1 year ago

MKwareContributions commented 1 year ago

When I try to access API endpoint (like with TavernAI) it throws this: kobold And on Tavern logs this: tavern Same thing when trying to access localhost:5001/api with browser: browser

Does that mean that there's no API endpoint to connect from other programs? Using noavx2 build, just in case.

Kagamma commented 1 year ago

It should. Try to send POST request to /api/latest/generate

LostRuins commented 1 year ago

Ah i see what is happening, somehow this fork of TavernAI is requesting from a new endpoint that I did not implement. It will be added in the next release of KoboldCpp.

henk717 commented 1 year ago

This is also bad design on the fork's part, they should never request /latest/ in case we change things up. They should target a version we specifically defined for future compatibility.

LostRuins commented 1 year ago

This should be fixed in the latest version, currently v1.3 at time of writing. https://github.com/LostRuins/koboldcpp/releases/latest

LostRuins commented 1 year ago

Closing issue as it seems to be resolved. Please open a new one if there are other problems