search

lamm-mit / PDF2Audio

Apache License 2.0
796 stars 86 forks source link

Custom API Base #3

Closed mitkox closed 3 days ago

mitkox commented 3 days ago

Hey, when I select "Custom API Base" on my local deployment and point it to my local Llama.cpp server it still asks for openAI API key? I thought when selecting "openai/custom_model" it doesn't use the cloud OpenAI API but rather the local API from the llama.cpp server?

mitkox commented 3 days ago

I fixed it. Now works locally.

lamm-mit commented 3 days ago

Thanks @mitkox , glad to hear. Please post any changes you had to make or things to consider, it would be helpful for others.

moebiussurfing commented 2 days ago

Yes, these tweaks to use llama as a free model alternative would be cool.