search

intellibar / main

IntelliBar is an AI-powered assistant for the Mac.
https://intellibar.app/
34 stars 0 forks source link

Enhancement: Custom api endpoints #22

Open FetchFast opened 8 months ago

FetchFast commented 8 months ago

Suggestion: Enhancement

An example of a custom endpoint would be: LLaMA 13B https://api.runpod.ai/v2/yourServer/runsync

It would: 1) allow Intellibar to be used in places where the OpenAI API is not available. 2) cheaper than openai, possibly by a lot

astoilkov commented 8 months ago

We've been thinking about this idea ourselves.

I'll be glad if more people join the discussion and share their use cases. That will help us with the details, direction, and importance.

FetchFast commented 8 months ago

Sure. I'd like to 1) point the endpoint at my serverless runpod llama, because it's vastly cheaper than openAI 2) point the endpoint to my server running memgpt, loaded with my own reference material to give me an unlimited context window.

Does that sound useful?

astoilkov commented 8 months ago

Yes, thanks! I hope others join the discussion as well.

rb81 commented 2 months ago

+1 Ollama and more external endpoints.