search

leptonai / leptonai

A Pythonic framework to simplify AI service building
https://lepton.ai/
Apache License 2.0
2.62k stars 169 forks source link

Adding Phi-3 model family as another option in Lepton AI Playground. #455

Closed HaiyangDING closed 1 week ago

HaiyangDING commented 1 month ago

Is your feature request related to a problem? Please describe.

Phi-3 is a family of small language models and multi-modal models from Microsoft. Having some Phi-3 models available in Leptonai playground not only enriches the Leptonai ecosystem, but also benefits Phi-3 as more users could be able to access it with ease (thanks to Lepton!).

Describe the solution you'd like

I would like to work on bringing some Phi-3 models to Lepton AI Playground as new options (a new block along side with Llama, Mistral, OpenChat, etc.). The plan is to start with microsoft/Phi-3-vision-128k-instruct.

However I did not find the doc on how to add a new model to the playground. Is there a doc for me to start with? (I am trying to build a photon locally now.)

Please let me know how you think about this idea. I volunteer to do all the required work.

Describe alternatives you've considered

I could also try to build new Phi-3 photon using Lepton framework and share with users in some other way. But I really want to make the model(s) available on Lepton playground to make an easy life for all the users.

Additional context

Phi-3 models are under MIT license. The license information

bddppq commented 2 weeks ago

Currently we do not plan to run Phi-3 as serverless API. You can make a photon and run a dedicated deployment on our platform.

HaiyangDING commented 1 week ago

Currently we do not plan to run Phi-3 as serverless API. You can make a photon and run a dedicated deployment on our platform.

Hi, thanks for the kind instruction. This is what I am trying to do now. Let me close the issue for now.