search

oxagast / Franklin

Franklin is a LLM powered AI IRC chat bot
https://franklin.oxasploits.com
Other
10 stars 5 forks source link

Using local api #14

Open mkayhko opened 8 months ago

mkayhko commented 8 months ago

Hello, I'm trying to use a local OpenAI-compatible API (by oobabooga) at http://localhost:5001/v1/completions and I get no replies from the bot. There's nothing in the log that would help me.

It works if I query it with curl, for example.

oxagast commented 8 months ago

Hi! Which branch are you using? I've recently brought the Cohere branch up even with frank_master, so frank_master will not work with OpenAI formed API requests. You'll need to specifify either the Davinci 003 branch, or the GPT 3.5 Turbo Instruct branch (whichever your API you're running fits). There also is no direct option to change the API from the Irssi controls, as it wasn't designed to work like that, you'll need to dig into the src and change the calls manually, there are two, one in callapi() and another in asshat() and replace it with your API location. If you have done all that, make sure there are not any other minor nuiances in the API calls that may need to be modified to fit your local model.

Hope this helps, oxagast