unslothai / unsloth

Finetune Llama 3, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
12.17k stars 791 forks source link

Support T5 models #643

Open tahirahmad2030 opened 2 weeks ago

tahirahmad2030 commented 2 weeks ago

I tried to load a T5 model but it seems not supported.

---------------------------------------------------------------------------
NotImplementedError                       Traceback (most recent call last)
Cell In[5], line 7
      4 dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
      5 load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.
----> 7 model, tokenizer = FastLanguageModel.from_pretrained(
      8     model_name = "google-t5/t5-large", # Choose ANY! eg mistralai/Mistral-7B-Instruct-v0.2
      9     max_seq_length = max_seq_length,
     10     dtype = dtype,
     11     load_in_4bit = load_in_4bit,
     12     # token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
     13 )

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/unsloth/models/loader.py:127, in FastLanguageModel.from_pretrained(model_name, max_seq_length, dtype, load_in_4bit, token, device_map, rope_scaling, fix_tokenizer, trust_remote_code, use_gradient_checkpointing, resize_model_vocab, revision, *args, **kwargs)
    125     dispatch_model = FastQwen2Model
    126 else:
--> 127     raise NotImplementedError(
    128         f"Unsloth: {model_name} not supported yet!\n"\
    129         "Make an issue to https://github.com/unslothai/unsloth!",
    130     )
    131 pass
    133 # Check if this is local model since the tokenizer gets overwritten

NotImplementedError: Unsloth: google-t5/t5-large not supported yet!
Make an issue to https://github.com/unslothai/unsloth!
danielhanchen commented 2 weeks ago

All model support in the future will make T5s work!

valentynbez commented 2 weeks ago

Bump, this would be great!