search

Azure / kaito

Kubernetes AI Toolchain Operator
MIT License
348 stars 35 forks source link

Support for other LLMs? #174

Open munishm opened 7 months ago

munishm commented 7 months ago

Hi, Is there any roadmap to support other open source LLM's? If there is any documentation already in place, please share.

Fei-Guo commented 7 months ago

No roadmap for exactly what models will be added yet. The idea is to collect user feedback or based on model popularity for a priority list. Technically, we will optimize dev workflow for adding new models so in the end, it can be a community driven process.

thegovind commented 5 months ago

So, if I want to host phi-2, do I have to fork this repo and configure it myself?

Fei-Guo commented 5 months ago

I think we can support phi-2 formally in kaito. We are working on a serial of documents about how to propose/contribute to add new models in Kaito. Stay tuned.

Fei-Guo commented 5 months ago

https://github.com/Azure/kaito/blob/main/docs/How-to-add-new-models.md

Here is the workflow we designed for new model onboarding.