search

unslothai / unsloth

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
15.18k stars 1.01k forks source link

Unsloth: unsloth/Phi-3-mini-4k-instruct not supported yet #378

Open KingNish24 opened 4 months ago

KingNish24 commented 4 months ago

How we can fine tune Multimodal LLM like Phi 3 using unsloth package ?

 config.json: 100%
  684/684 [00:00<00:00, 43.8kB/s]
 ---------------------------------------------------------------------------
 NotImplementedError                       Traceback (most recent call last)
 [<ipython-input-2-aba42707d972>](https://localhost:8080/#) in <cell line: 4>()
       2 import numpy as np
       3 from unsloth import FastLanguageModel
 ----> 4 model, tokenizer = FastLanguageModel.from_pretrained(
       5     model_name = "HuggingFaceM4/idefics2-8b",
       6     max_seq_length = 2048,

 [/usr/local/lib/python3.10/dist-packages/unsloth/models/loader.py](https://localhost:8080/#) in from_pretrained(model_name, max_seq_length, dtype, load_in_4bit, token, device_map, rope_scaling, fix_tokenizer, trust_remote_code, use_gradient_checkpointing, *args, **kwargs)
     120             dispatch_model = FastGemmaModel
     121         else:
 --> 122             raise NotImplementedError(
     123                 f"Unsloth: {model_name} not supported yet!\n"\
     124                 "Make an issue to https://github.com/unslothai/unsloth!",

 NotImplementedError: Unsloth: unsloth/Phi-3-mini-4k-instruct not supported yet!
 Make an issue to https://github.com/unslothai/unsloth!
danielhanchen commented 4 months ago

Working on it!!! (for Phi-3) Multimodal not yet supported - its on our roadmap

sumukshashidhar commented 4 months ago

@danielhanchen Looking forward to Phi-3 support! Do let me know if you need any help regarding this!

danielhanchen commented 4 months ago

@sumukshashidhar Ill try adding it today!

NickyDark1 commented 4 months ago

image

erwe324 commented 4 months ago

I guess work is still going on. Let's wait and watch, fingers crossed.

CrispStrobe commented 4 months ago

or you can just use a llamafied version of the model

reneric commented 4 months ago

Glad to hear you're working on Phi-3. Thanks for the hard work!

CrispStrobe commented 4 months ago

for illustration: Phi3_ORPO_Unsloth.ipynb

danielhanchen commented 4 months ago

Supported! https://colab.research.google.com/drive/1NvkBmkHfucGO3Ve9s1NKZvMNlw5p83ym?usp=sharing

https://huggingface.co/unsloth/Phi-3-mini-4k-instruct

danielhanchen commented 4 months ago

Had to mistral-fy it since Phi has sliding window attention and fixed the 2047/2048 SWA bug and unfused everything

CrispStrobe commented 4 months ago

nice! (btw in unsloth/Phi-3-mini-4k-instruct modelcard also says q4 but that is ofc unquantized)

danielhanchen commented 4 months ago

@CrispStrobe Thanks for the quick notice!!

NickyDark1 commented 4 months ago

Does this modify the model much?

danielhanchen commented 4 months ago

Oh no not really - it's just mistral-fied - also I fixed a few bugs