TypeError: Unsloth: Your model already has LoRA adapters. No need to run this again!

Hi, I performed the CPT for my domain following the Continued Pretraining notebook. Now when I am trying to perform the fine-tuning and applying a different LoRA config, it gives me error TypeError: Unsloth: Your model already has LoRA adapters. No need to run this again! For fine-tuning, I wanted to use different LoRA config(smaller r and different target_modules than pre-training) but I am not able to apply this config.

Currently, I changed the model_name to the checkpoint obtained after CPT for the fine-tuning. Below is the code for reference:-

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "/home/ec2-user/SageMaker/ES/ES-cpt-Phi-3-mini-4k-instruct/checkpoint-2550", 
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    cache_dir ='./hf_cache'
)

model = FastLanguageModel.get_peft_model(
    model,
    r = 8, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
                      "gate_proj", "up_proj", "down_proj",],
    lora_alpha = 16,
    lora_dropout = 0, # Supports any, but = 0 is optimized
    bias = "none",    # Supports any, but = "none" is optimized
    # [NEW] "unsloth" uses 30% less VRAM, fits 2x larger batch sizes!
    use_gradient_checkpointing = "unsloth", # True or "unsloth" for very long context
    random_state = 3407,
    use_rslora = False,  # We support rank stabilized LoRA
    loftq_config = None, # And LoftQ
)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[12], line 1
----> 1 model = FastLanguageModel.get_peft_model(
      2     model,
      3     r = 8, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128
      4     target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
      5                       "gate_proj", "up_proj", "down_proj",],
      6     lora_alpha = 16,
      7     lora_dropout = 0, # Supports any, but = 0 is optimized
      8     bias = "none",    # Supports any, but = "none" is optimized
      9     # [NEW] "unsloth" uses 30% less VRAM, fits 2x larger batch sizes!
     10     use_gradient_checkpointing = "unsloth", # True or "unsloth" for very long context
     11     random_state = 3407,
     12     use_rslora = False,  # We support rank stabilized LoRA
     13     loftq_config = None, # And LoftQ
     14 )

File ~/anaconda3/envs/pytorch_p310/lib/python3.10/site-packages/unsloth/models/llama.py:1409, in FastLlamaModel.get_peft_model(model, r, target_modules, lora_alpha, lora_dropout, bias, layers_to_transform, layers_pattern, use_gradient_checkpointing, random_state, max_seq_length, use_rslora, modules_to_save, init_lora_weights, loftq_config, temporary_location, **kwargs)
   1406 transformers_set_seed(random_state)
   1408 if isinstance(model, PeftModelForCausalLM):
-> 1409     raise TypeError(
   1410         "Unsloth: Your model already has LoRA adapters. No need to run this again!"
   1411     )
   1412 pass
   1414 if loftq_config is None: loftq_config = {}

TypeError: Unsloth: Your model already has LoRA adapters. No need to run this again!

Can someone please suggest me what changes are required to perform fine-tuning with different LoRA config than the CPT?

Also, if I ignore applying new LoRA config and use the same applied during the pre-training, the loss doesnt seem to be converging at all.

unslothai / unsloth

TypeError: Unsloth: Your model already has LoRA adapters. No need to run this again! #631