Add LM Studio Server option for a custom OpenAI compatible endpoint

mrwadams / stride-gpt

An AI-powered threat modeling tool that leverages OpenAI's GPT models to generate threat models for a given application based on the STRIDE methodology.

https://stridegpt.streamlit.app

MIT License

494 stars 155 forks source link

Add LM Studio Server option for a custom OpenAI compatible endpoint #37

Open willasaywhat opened 4 months ago

willasaywhat commented 4 months ago

This PR adds support for a custom, user defined endpoint URI for use with LM Studio. This allows for self hosted, local AI model usage with an OpenAI compatible API.

th3f001 commented 3 months ago

+1 to merge this ....and possibly add also ollama as local LLM-inference provider

chenweisomebody126 commented 3 months ago

Yeah, I like this

chenweisomebody126 commented 3 months ago

@willasaywhat I can merge your branch into mine like what I did in https://github.com/chenweisomebody126/stride-gpt/pull/1. I feel I would like to support more models as you suggested.

willasaywhat commented 3 months ago

Sure, that makes sense! I still need to check out the branch that the repo owner put together as well: https://github.com/mrwadams/stride-gpt/issues/25#issuecomment-2267488775