myshell-ai / AIlice

AIlice is a fully autonomous, general-purpose AI agent.

MIT License

876 stars 136 forks source link

Will there be gguf support? #8

Closed babybirdprd closed 9 months ago

babybirdprd commented 10 months ago

Ollama, Litellm, lmstudio, etc

stevenlu137 commented 10 months ago

Yes, I will consider adding support for more inference engines. However, the current priority is to add support for multimodal models(this will lead to truly amazing features), and I need to prioritize getting this done first.

stevenlu137 commented 10 months ago

10 #6 Hey guys, I have just completed support for the inference servers compatible with OpenAI API. Interested friends please get the latest version of the dev branch code and try it. Please note that config.json has been modified this time. It is best to delete config.json before running ailice.

For details, you need to read the "Models(or Services) compatible with OpenAI Interface" section in README.md.

babybirdprd commented 10 months ago

Awesome! Will check it shortly.

stevenlu137 commented 9 months ago

Various inference services have been well supported, let's close the issue.