Script about qlora.

[yeppp27]

Hi! Thanks for your great work. Can you provide the script with Qlora fine-tuning?

[R3xpook]

It's on the training argument --bits (int): Quantization bits (default: 16). Not all arguments are in the training script Training arguments

--deepspeed (str): Path to DeepSpeed config file (default: "scripts/zero2.json").
--data_path (str): Path to the LLaVA formatted training data (a JSON file). (Required)
--image_folder (str): Path to the images folder as referenced in the LLaVA formatted training data. (Required)
--model_id (str): Path to the Qwen2-VL model. (Required)
--output_dir (str): Output directory for model checkpoints
--num_train_epochs (int): Number of training epochs (default: 1).
--per_device_train_batch_size (int): Training batch size per GPU per forwarding step.
--gradient_accumulation_steps (int): Gradient accumulation steps (default: 4).
--freeze_vision_tower (bool): Option to freeze vision_model (default: False).
--freeze_llm (bool): Option to freeze LLM (default: False).
--tune_merger (bool): Option to tune projector (default: True).
--num_lora_modules (int): Number of target modules to add LoRA (-1 means all layers).
--vision_lr (float): Learning rate for vision_model.
--merger_lr (float): Learning rate for merger(projector).
--learning_rate (float): Learning rate for language module.
--bf16 (bool): Option for using bfloat16.
--fp16 (bool): Option for using fp16.
--min_pixels (int): Option for minimum input tokens.
--max_pixles (int): OPtion for maximum maxmimum tokens.
--lora_namespan_exclude (str): Exclude modules with namespans to add LoRA.
--max_seq_length (int): Maximum sequence length (default: 32K).
--bits (int): Quantization bits (default: 16).
--disable_flash_attn2 (bool): Disable Flash Attention 2.
--report_to (str): Reporting tool (choices: 'tensorboard', 'wandb', 'none') (default: 'tensorboard').
--logging_dir (str): Logging directory (default: "./tf-logs").
--lora_rank (int): LoRA rank (default: 128).
--lora_alpha (int): LoRA alpha (default: 256).
--lora_dropout (float): LoRA dropout (default: 0.05).
--logging_steps (int): Logging steps (default: 1).
--dataloader_num_workers (int): Number of data loader workers (default: 4).

[2U1]

@R3xpook Thanks for the comment. @yeppp27 That's right. You can add bits too use qlora.

[yeppp27]

Thanks!