search

PKU-YuanGroup / Chat-UniVi

[CVPR 2024 Highlight🔥] Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding
https://arxiv.org/abs/2311.08046
Apache License 2.0
755 stars 41 forks source link

What's the minimal device requirement on this project? #19

Open leeaction opened 8 months ago

leeaction commented 8 months ago

I'm trying this model , please provide the minimal device requirement information , such as GPU(type/memory) CPU SSD

jpthu17 commented 8 months ago

If you intend to perform full parameter fine-tuning, it should be carried out on the 8*A100 (80G). If opting for Lora, the tuning process is feasible on 4*V100 (32G).

To mitigate hardware demands, consider utilizing a smaller language model, like phi-2. Fine-tuning for phi-2 is achievable on 3090 GPUs. I intend to enhance the code in the upcoming weeks to facilitate fine-tuning for phi-2.

lucasjinreal commented 8 months ago

Hi，any plan to support some decent Chinese LLM models? There are now some hight quality Chinese video instruct data now.

jpthu17 commented 8 months ago

Hi，any plan to support some decent Chinese LLM models? There are now some hight quality Chinese video instruct data now.

We will support fine-tuning of the Qwen and mixture of experts in the update.

lucasjinreal commented 8 months ago

@jpthu17 Hoping for it! Please pin me once you guys got some upates.