search

FireCubeStudios / Run

WinUI 3 Run
https://www.microsoft.com/store/apps/9NQGV64S5136
MIT License
150 stars 10 forks source link

(Feature Request) Allow setting a custom OpenAI API endpoint to experiment with locally hosted LLMs #32

Open PicoPlanetDev opened 1 year ago

PicoPlanetDev commented 1 year ago

Some open-source, locally run LLMs have the capability to emulate key features of the OpenAI API to "appear" to an application as GPT. Can you add support for a custom API endpoint to experiment with this and see if the performance of something like WizardLM or Falcon may be up to the task of generating commands? I would then envision an input field in the Settings window with the default OpenAI API endpoint URL replaceable with your own.

FireCubeStudios commented 1 year ago

Would i tbe possible to test this "see if the performance of something like WizardLM or Falcon may be up to the task of generating commands" in a seperate prototype app first?

I have another WIP all in one LLM app which could support custom LLM but thats only for that app. I plan to stick to GPT 3/4 for Run