The gpu cost when running finetune/finetune_lora_ds.sh

[zhengshuo1]

Hello, may I ask about the gpu cost when running finetune_lora_ds.sh. I have a 49GB*4 machine, however, when I ran the shell, it appeared to be out of memory.

[njucckevin]

Hi,

Our fine-tuning takes about 24h on 8*A100(80G). I suppose 49G*4 GPU should be ok, referring to Qwen-VL's fine-tuning guide. You may need to reduce the micro-batch-size.

[zhengshuo1]

Hi,

Our fine-tuning takes about 24h on 8A100(80G). I suppose 49G4 GPU should be ok, referring to Qwen-VL's fine-tuning guide. You may need to reduce the micro-batch-size.

thanks, when I reduce micro-batch-size to 1, it is ok. I have another question. If I use the parameter max-length, it is shown "Unknown parameter passed".

[njucckevin]

The parameters are defined in finetune/finetune_lora_ds.sh, which include --max-length. What is your fine-tuning command?

[zhengshuo1]

The parameters are defined in finetune/finetune_lora_ds.sh, which include --max-length. What is your fine-tuning command?

sorry. I tried it again, and there was nothing wrong.