search

lamm-mit / PDF2Audio

Apache License 2.0
1.09k stars 138 forks source link

Custom API Base #3

Closed mitkox closed 2 months ago

mitkox commented 2 months ago

Hey, when I select "Custom API Base" on my local deployment and point it to my local Llama.cpp server it still asks for openAI API key? I thought when selecting "openai/custom_model" it doesn't use the cloud OpenAI API but rather the local API from the llama.cpp server?

mitkox commented 2 months ago

I fixed it. Now works locally.

lamm-mit commented 2 months ago

Thanks @mitkox , glad to hear. Please post any changes you had to make or things to consider, it would be helpful for others.

moebiussurfing commented 1 month ago

Yes, these tweaks to use llama as a free model alternative would be cool.