issue while finetuning llama 3.1, 'tuple' object has no attribute 'remove_unused_columns'

[Abhisekgit1994]

I am trying to finetune Llama 3.1 with below settings Unsloth 2024.8: Fast Llama patching. Transformers = 4.44.2. GPU: NVIDIA A10G. Max memory: 21.988 GB. Platform = Linux. Pytorch: 2.1.0+cu118. CUDA = 8.6. CUDA Toolkit = 11.8. Bfloat16 = TRUE. FA [Xformers = 0.0.22.post7. FA2 = True]

remove_unused_columns=args.remove_unused_columns if args is not None else True, AttributeError: 'tuple' object has no attribute 'remove_unused_columns' for trl < 0.9

AttributeError: 'tuple' object has no attribute 'packing' for trl > 0.9

[Abhisekgit1994]

same issue with different package versions too

Unsloth 2024.8: Fast Llama patching. Transformers = 4.44.2. GPU: NVIDIA A10G. Max memory: 21.988 GB. Platform = Linux. Pytorch: 2.3.0+cu118. CUDA = 8.6. CUDA Toolkit = 11.8. Bfloat16 = TRUE. FA [Xformers = 0.0.26.post1+cu118. FA2 = True]

Unsloth 2024.8 patched 32 layers with 32 QKV layers, 32 O layers and 32 MLP layers. trainer = SFTTrainer( File "/home/ubuntu/anaconda3/lib/python3.9/site-packages/trl/trainer/sft_trainer.py", line 292, in init remove_unused_columns=args.remove_unused_columns if args is not None else True, AttributeError: 'tuple' object has no attribute 'remove_unused_columns'

when I checked the soruce code it has args.method instead of self.args.method

[Abhisekgit1994]

I dont know how you have built it I cant get it started whatever environment or package combination I use. This is the complicated library I have ever seen.

[danielhanchen]

Apologies on the delay @Abhisekgit1994 ! Do you know which notebook you used or is this a custom script?

[Abhisekgit1994]

the llama 3.1 notebook on custom data

from unsloth import FastLanguageModel import torch from trl import SFTTrainer from transformers import TrainingArguments from datasets import load_dataset from unsloth import is_bfloat16_supported max_seq_length = 8192 # Choose any! We auto support RoPE Scaling internally! dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+ load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.

4bit pre quantized models we support for 4x faster downloading + no OOMs.

fourbit_models = [ "unsloth/Meta-Llama-3.1-8B-bnb-4bit", # Llama-3.1 15 trillion tokens model 2x faster! "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit", "unsloth/Meta-Llama-3.1-70B-bnb-4bit", "unsloth/Meta-Llama-3.1-405B-bnb-4bit", # We also uploaded 4bit for 405b! "unsloth/Mistral-Nemo-Base-2407-bnb-4bit", # New Mistral 12b 2x faster! "unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit", "unsloth/mistral-7b-v0.3-bnb-4bit", # Mistral v3 2x faster! "unsloth/mistral-7b-instruct-v0.3-bnb-4bit", "unsloth/Phi-3-mini-4k-instruct", # Phi-3 2x faster!d "unsloth/Phi-3-medium-4k-instruct", "unsloth/gemma-2-9b-bnb-4bit", "unsloth/gemma-2-27b-bnb-4bit", # Gemma 2x faster! ] # More models at https://huggingface.co/unsloth

model, tokenizer = FastLanguageModel.from_pretrained( model_name = "Meta-Llama-3.1-8B-Instruct-bnb-4bit", max_seq_length = max_seq_length, dtype = dtype, load_in_4bit = load_in_4bit,

token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf

)

data_files = {"train": "llama3.1_chat_train.csv", "test": "llama3.1_chat_test.csv"} dataset = load_dataset("csv", data_files=data_files) print(dataset)

model = FastLanguageModel.get_peft_model( model, r = 32, # Choose any number > 0 ! Suggested 8, 16, 32, 64, 128 target_modules = ["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj",], lora_alpha = 32, lora_dropout = 0, # Supports any, but = 0 is optimized bias = "none", # Supports any, but = "none" is optimized

[NEW] "unsloth" uses 30% less VRAM, fits 2x larger batch sizes!

use_gradient_checkpointing = "unsloth", # True or "unsloth" for very long context
random_state = 3407,
use_rslora = False,  # We support rank stabilized LoRA
loftq_config = None, # And LoftQ

)

training_args = TrainingArguments( num_train_epochs = 10, per_device_train_batch_size = 4, per_device_eval_batch_size = 4, gradient_accumulation_steps = 4, eval_strategy = "epoch", warmup_steps = 5,

num_train_epochs = 1, # Set this for 1 full training run.

    learning_rate = 2e-4,
    fp16 = not is_bfloat16_supported(),
    bf16 = is_bfloat16_supported(),
    logging_steps = 10,
    optim = "paged_adamw_8bit",
    weight_decay = 0.01,
    lr_scheduler_type = "linear",
    seed = 3407,
    group_by_length= True,
    output_dir = "outputs",
    report_to = "tensorboard",
),

trainer = SFTTrainer( model = model, tokenizer = tokenizer, train_dataset = dataset["train"], eval_dataset= dataset["test"], dataset_text_field = "text", max_seq_length = max_seq_length, packing = False, # Can make training 5x faster for short sequences. args = training_args )

trainer_stats = trainer.train()

model.save_pretrained("lora_model") # Local saving tokenizer.save_pretrained("lora_model")

model.save_pretrained_merged("merged_model", tokenizer, save_method = "merged_16bit",)

[Ammar-Alnagar]

I faced the same issue a while before , i think its an issue with the dataset , try a different dataset or use the one in the premade notebook .

[Abhisekgit1994]

okay