search

BAAI-DCAI / Bunny

A family of lightweight multimodal models.
Apache License 2.0
894 stars 67 forks source link

about finetune train #90

Closed ZuyongWu closed 2 months ago

ZuyongWu commented 4 months ago

I only have 4x4090 cards，under this circumstance， can i finetune a MLLM？ How should I do to train， thanks a lot.

RussRobin commented 3 months ago

Thank you for your interest in Bunny. 24G per device is enough to pretrain and finetune Bunny. However, the actual GPU memory consumption depends on your base model, image resolution and data.

For finetuning, setting per-device-batch-size to 2 or 4 may be good to you. In order to use the default learning rate in finetune_lora.sh, we recommend keeping global batch size 128. Global batch size = num of GPU batch size per GPU accumulation step. In your case, num of GPU is 4. All these parameters can be set in finetune_lora.sh. Similarly, set batch size that fits for you in pretraining, of full parameter tuning.

Feel free to further comment on this issue if you meet any problems in using Bunny.

Regards Russell

RussRobin commented 2 months ago

I'll close this issue since no further discussion is raised yet. Please reopen it if you still have concerns.