search

LostRuins / lite.koboldai.net

A zero dependency web UI for KoboldAI Horde
GNU Affero General Public License v3.0
78 stars 37 forks source link

Suggestion: Add support for Featherless.ai API #81

Closed CallMeAl1973 closed 1 month ago

CallMeAl1973 commented 3 months ago

The Featherless service offers access to the largest collection of models I've seen so far, for a very reasonable price. according to them, you can run any Llama model from Hugging Face, they claim 1762 available at the time of this writing. You can create an account and get an API key for free, but not sure how useful that key will be for testing without a subscription. https://featherless.ai/ Access to this many models from Kobold Lite would be amazing!

LostRuins commented 3 months ago

Took a look at it but was unable to find any public API documentation. Do you have a link to it?

henk717 commented 3 months ago

Aren't they just OpenAI compatible? Did you try the OpenAI API mode?

CallMeAl1973 commented 3 months ago

It's beyond me why they don't seem to have any official API documentation in their FAQ. I was talking to someone on their discord who said their API is OpenAI compatible at https://api.featherless.ai/v1. I tried using my key and that API URL in Kobold Lite but no dice. Featherless does work in SillyTavern, but it has its own setting and doesn't expose the API URL, leading me to believe it probably isn't open AI compatible after all? ...

LostRuins commented 3 months ago

Is there any way I can get a free working key so I can at least test without having to subscribe? (don't post the key here, DM me on discord if you have one)

It is very strange that there does not seem to be documentation online.

Btw, I have recently added MistralAI api, you can check that out too.

LostRuins commented 3 months ago

Hi, sorry to backtrack on the featherless API so soon, but I have received complaints from some members in our community who have objected to directly promoting this service, because they are selling commercial access to non commercial models, so we cannot directly promote it at this time.

We are looking forward to listing them once this issue has been resolved on their side. For now if you do want to use them you can add https://api.featherless.ai/v1 to the OpenAI API, this method should allow you to continue to access their service.

LostRuins commented 1 month ago

Hi @CallMeAl1973, if you're still interested, it has now been added back, give it a try!

jeffkelly2020 commented 1 month ago

Awesome. Great to hear!

I'll switch my Featherless key to that slot and use the OpenAI Compatible slot for something else.

LostRuins commented 1 month ago

Great, let me know if there are any issues.