search

DAMO-NLP-SG / VideoLLaMA2

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
Apache License 2.0
464 stars 27 forks source link

Finetune with LoRA and QLoRA #5

Open thisurawz1 opened 3 weeks ago

thisurawz1 commented 3 weeks ago

Can you provide a detailed guide on how to use LoRA or QLoRA finetuning of VideoLLaMA2 and also for inference. Moreoever, can you provide the guide on how to make the finetuning dataset for images , videos and both.

sennnnn commented 2 weeks ago

Parameter-efficient finetuning~(e.g., Lora/Qlora) is under developing. Please wait a moment, this part is coming up soon.

thisurawz1 commented 2 weeks ago

Thank you for the response. Can you tell us when? Also, is it possible to make a WeChat group and share the QR here? So we can post our problems related to Videollama2 there.

lixin4ever commented 2 weeks ago

Sorry, due to the lack of manpower, we only respond to the Github issues, and no plan to maintain a WeChat group regarding VideoLLaMA2.

lixin4ever commented 1 day ago

@thisurawz1 Fine-tuning with LoRA/QLoRA has been available at https://github.com/DAMO-NLP-SG/VideoLLaMA2/tree/main/scripts/custom, please refer to the corresponding scripts for more details.

thisurawz1 commented 1 day ago

thank you so much. can you please provide a detailed guide on finetuning using QLoRA? because when with the .sh file it's so confusing how to do it.