Deploying this fine-tuned model

[babla9]

Thanks so much for your work on this!

How can I deploy this fine-tuned model (expose via API endpoint)? Can I use vLLM or a library like this: https://github.com/EricLBuehler/mistral.rs, which supports Phi3-Vision?

Thanks!

[2U1]

@babla9 vLLM and mistral.rs both supports phi3-vision. But, merging the lora weights should be more easier, when you use it.

[kevintee]

@2U1 What do you mean by merging the lora weights? Lora for phi-3 vision is currently not supported in vLLM.

[2U1]

@kevintee When you fine tune the model with the code. You will get a adapter weight(Because it finetunes phi3 with lora). You should merge in to the original model to use it with vLLM.