Llava style model support in Onnx?

microsoft / onnxruntime-genai

Generative AI extensions for onnxruntime

MIT License

425 stars 99 forks source link

Llava style model support in Onnx? #315

Closed kinchahoy closed 4 months ago

kinchahoy commented 4 months ago

Hey folks,

I was thinking about training a small Phi-3 + OpenClip Llava style model (building off the MobileVLM work). Do you plan to support that sort of multi-modal model with Onnx (or ideally onnxruntime-genai)? Let me know if I can help test etc.

natke commented 4 months ago

@kinchahoy Yes we do have this support on our road map. I am moving to our Discussions forum to track it there