Open brunopistone opened 6 days ago
The error message is quite strange, since it's pointing to a docstring, which somehow suggests that the code was changed on the fly. Not sure what is going there.
Regardless of that, a couple of things you could try:
get_peft_model
, instead pass the lora_config
to SFTTrainer
as peft_config=lora_config
argument.modules = find_all_linear_names(model)
, pass target_modules="all-linear"
to LoraConfig
.prepare_model_for_kbit_training
call.Could you please check if any of these changes (or a combination of them) fixes the issue?
System Info
transformers
version: 4.45.1Who can help?
@BenjaminBossan @sayakpaul
Information
Tasks
examples
folderReproduction
Script executed with Amazon SageMaker, instance type g5.12xlarge (4 GPUs):
train_dataset:
train_dataset[0]["text"] (mock):
Error returned:
requirements.txt:
Expected behavior
The script was adapted from run_fsdp_qlora.py, which seems to work. Switched to
SFTConfig
as recommended in the documentation. Expected behavior is to execute successfully the training script.