search

unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
18.46k stars 1.29k forks source link

Request for Support: Phi-3 Vision Model #908

Open rahatarinasir opened 3 months ago

rahatarinasir commented 3 months ago

Hello Unsloth Team,

I am trying to finetune the dwb2023/phi-3-vision-128k-instruct-quantized model using Unsloth, but I encountered a NotImplementedError. The error message indicates that this model is not currently supported.(NotImplementedError: Unsloth: dwb2023/phi-3-vision-128k-instruct-quantized not supported yet!) I would like to know which vLLM models are currently supported by Unsloth. Could you please provide a list or any documentation regarding compatible vLLM models?

danielhanchen commented 3 months ago

Sorry - currently vision LLMs are not supported yet!

rdzotz commented 2 months ago

A subtle upvote here to get the newer microsoft/Phi-3.5-vision-instruct model implementation also.

I am starting here for others also interested https://github.com/microsoft/Phi-3CookBook/blob/main/code/04.Finetuning/vision_finetuning/finetune_hf_trainer_nlvr2.py

shimmyshimmer commented 1 day ago

@rdzotz @taesiri @NuiMrme

Hey guys apologies for the delays. Vision models are now supported in Unsloth! Please update Unsloth however we are still working on Phi3 Vision support :)

Read our blogpost: https://unsloth.ai/blog/vision Tweet: https://x.com/UnslothAI/status/1859667930075758793 GitHub post: https://github.com/unslothai/unsloth/releases/tag/November-2024 Vision Model uploads: https://huggingface.co/collections/unsloth/vision-multimodal-models-673eb9908fc2cb3deebd2fa3