search

DAMO-NLP-SG / VideoLLaMA2

VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs
Apache License 2.0
922 stars 60 forks source link

⭐ [Feat] Supporting qlora finetuning. #37

Closed clownrat6 closed 4 months ago

clownrat6 commented 4 months ago

BUG 1

Since deepspeed is not compatible with bitsandbytes according to this issue, qlora finetuning is not supported with deepspeed zero3 enviroment.

Solution

According to some documents, if you still want to train quantized model with deepspeed. There are two ways to achieve this:

  1. Adopting deepspeed zero2.
  2. Adopting accelerate to manage training progress. Please refer to accelerate document.

BUG 2

AttributeError: 'torch.dtype' object has no attribute 'element_size'

Solution

Updating torch==2.2.0

BUG 3

You are calling `save_pretrained` to a 4-bit converted model, but your `bitsandbytes` version doesn't support it. If you want to save 4-bit models, make sure to have `bitsandbytes>=0.41.3` installed.

Solution

Updating bitsandbytes==0.43.0