Look into local LLMs support

minimaxir / simpleaichat

Python package for easily interfacing with chat apps, with robust features and minimal code complexity.

MIT License

3.43k stars 224 forks source link

Look into local LLMs support #98

Open minimaxir opened 7 months ago

minimaxir commented 7 months ago

It appears local LLMs are at a state now where it's easy enough to use them for the average user, and I just got a M3 Pro MacBook Pro for testing.

The servers packaged with apps like LM Studio follow OpenAI's API spec, which means that it should be fine to work with simpleaichat, pending a few warnings about unsupported config parameters.

hugokoopmans commented 6 months ago

can you let me know if i can use this tool to talk to GPT4all.io in server mode (it provides openAI API compatible API)

minimaxir commented 6 months ago

Not currently but that functionality would be theoretically covered by this change.