search

haotian-liu / LLaVA

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.
https://llava.hliu.cc
Apache License 2.0
20.31k stars 2.24k forks source link

Error when load model in 4bit #1638

Open rin2401 opened 3 months ago

rin2401 commented 3 months ago

Don't need kwargs['load_in_4bit'] = True when use quantization_config

https://github.com/haotian-liu/LLaVA/blob/c121f0432da27facab705978f83c4ada465e46fd/llava/model/builder.py#L34-L40

drzraf commented 2 months ago

Happens with transformers 4.44.2 . The ValueError was introduced here in transformers (https://github.com/huggingface/transformers/pull/21579) (instead of a slow deprecation)

This was merged in https://github.com/huggingface/transformers/commit/3668ec17165dbb7823f3bc7e190e1733040c3af8 (part of v4.27.0 and onward)

and since this project requires transformers==4.37.2, there is no reason to keep passing deprecated booleans which trigger this error.