search

RupertLuo / Valley

The official repository of "Video assistant towards large language model makes everything easy"
199 stars 13 forks source link

light weight version of Valley #4

Closed Xinxinatg closed 1 year ago

Xinxinatg commented 1 year ago

Thanks for your wonderful work, I think this repo deserves much more attention than other gpt based video understanding, because this one generate the best output compared to other gpt-based video understanding tools according to my preliminary test.

I am wondering whether there are light weight valley available that I can deploy on A30 server with 24 GB vram? I think the light weighted one will further broaden the user case.

RupertLuo commented 1 year ago

Thank you for your approval of this project. We are exploring on a smaller language model to achieve similar results. Currently, we are trying vicuna-7b and chatglm-6b. These two will be released soon and should be able to be used on a30 to deploy. In addition, we are also working on the quantization of 13b.

Xinxinatg commented 1 year ago

thanks for your prompt reply, looking forward to the update!

ohmygoobness commented 1 year ago

can this model be converted to 4-bit GPTQ

RupertLuo commented 1 year ago

can this model be converted to 4-bit GPTQ

I haven't tried quantizing the weights yet.