search

mbzuai-oryx / Video-ChatGPT

[ACL 2024 🔥] Video-ChatGPT is a video conversation model capable of generating meaningful conversation about videos. It combines the capabilities of LLMs with a pretrained visual encoder adapted for spatiotemporal video representation. We also introduce a rigorous 'Quantitative Evaluation Benchmarking' for video-based conversational models.
https://mbzuai-oryx.github.io/Video-ChatGPT
Creative Commons Attribution 4.0 International
1.17k stars 102 forks source link

Llava Weights not available for running demo offline #80

Closed chakrabortyrajatsubhra closed 8 months ago

chakrabortyrajatsubhra commented 8 months ago

I tried to do this way -- "download the ready LLaVA-Lightening-7B weights from mmaaz60/LLaVA-Lightening-7B-v1-1" but there were no weights there how to get the converted weights?? I tried the first way also-- "Get the original LLaMA weights in the huggingface format by following the instructions here." but never received any weights. Can you provide with the LLaVa weights?

mmaaz60 commented 8 months ago

Hi @chakrabortyrajatsubhra,

Thank you for your interest in our work. You can use the following git lfs commands to get the LLaVA weights,

git lfs install
git clone https://huggingface.co/mmaaz60/LLaVA-7B-Lightening-v1-1

Thanks.

chakrabortyrajatsubhra commented 8 months ago

Thank you so much, I have one additional question to ask if you can answer. While doing the offline inference if I want to use CLIP -VIT -Base patch 32 as the feature extractor instead of vit large patch 14 is that possible? And where in the code should i make changes? Looking forward for your reply. Thanks Rajatsubhra Chakraborty

On Wed, Jan 17, 2024 at 12:42 AM Muhammad Maaz @.***> wrote:

Hi @chakrabortyrajatsubhra https://github.com/chakrabortyrajatsubhra,

Thank you for your interest in our work. You can use the following git lfs commands to get the LLaVA weights,

git lfs install git clone https://huggingface.co/mmaaz60/LLaVA-7B-Lightening-v1-1

Thanks.

— Reply to this email directly, view it on GitHub https://github.com/mbzuai-oryx/Video-ChatGPT/issues/80#issuecomment-1894977156, or unsubscribe https://github.com/notifications/unsubscribe-auth/ALIDRLBYUX422LIL7H3XEDLYO5QCVAVCNFSM6AAAAABB3UN3E2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOJUHE3TOMJVGY . You are receiving this because you were mentioned.Message ID: @.***>