Request for Support: Phi-3 Vision Model

unslothai / unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

https://unsloth.ai

Apache License 2.0

15.56k stars 1.04k forks source link

Request for Support: Phi-3 Vision Model #908

Open rahatarinasir opened 1 month ago

rahatarinasir commented 1 month ago

Hello Unsloth Team,

I am trying to finetune the dwb2023/phi-3-vision-128k-instruct-quantized model using Unsloth, but I encountered a NotImplementedError. The error message indicates that this model is not currently supported.(NotImplementedError: Unsloth: dwb2023/phi-3-vision-128k-instruct-quantized not supported yet!) I would like to know which vLLM models are currently supported by Unsloth. Could you please provide a list or any documentation regarding compatible vLLM models?

danielhanchen commented 1 month ago

Sorry - currently vision LLMs are not supported yet!

rdzotz commented 2 weeks ago

A subtle upvote here to get the newer microsoft/Phi-3.5-vision-instruct model implementation also.

I am starting here for others also interested https://github.com/microsoft/Phi-3CookBook/blob/main/code/04.Finetuning/vision_finetuning/finetune_hf_trainer_nlvr2.py