PKU-YuanGroup Video-LLaVA issues

PKU-YuanGroup / Video-LLaVA

【EMNLP 2024🔥】Video-LLaVA: Learning United Visual Representation by Alignment Before Projection

https://arxiv.org/pdf/2311.10122.pdf

Apache License 2.0

2.88k stars 207 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Can't reproduce results on MSRVTT and MSVD dataset

#191 1999Lyd closed 1 week ago
0
Issues with Converting the video-llava Model to ONNX

#190 Ark1a closed 1 month ago
0
Pretrain and Finetune template versions

#189 xin-li-67 opened 1 month ago
1
When I evaluated the ‘TGIF_Zero_Shot_QA’ dataset, the accuracy was only 13%. Should I train first to achieve the 70% accuracy in the paper?

#188 FanshuoZeng opened 1 month ago
0
May I ask what is the api_base used for evaluation?

#187 FanshuoZeng opened 1 month ago
0
missing file: preprocessor_config.json

#186 JunanPan closed 2 months ago
0
Can this model apply a few-shot when inference?

#185 Ijustakid opened 2 months ago
0
ImportError: cannot import name '_expand_mask' from 'transformers.models.clip.modeling_clip'

#184 qiuchen001 opened 2 months ago
4
Update llava_arch.py for device synchronization of mm_projector outputs

#183 FangXinyu-0913 opened 2 months ago
0
Valley video not found during pretraining.

#182 Aakriti05 opened 2 months ago
0
Api is not running properly getting errors In each endpoint

#181 RAJA102002 opened 2 months ago
0
Questions about LanguageBind Usage

#180 lingjunzhao opened 2 months ago
0
Multi-GPU inference problem.

#179 jiazheng-xing opened 2 months ago
0
Request for Inference Parameters on VideoLLava

#178 adrianwestmoon opened 2 months ago
0
Is it possible to train with languages other than English, and are the 8 frames sampled uniformly across different video lengths?

#177 YoungjaeDev opened 2 months ago
0
size mismatch

#176 cs19469 opened 2 months ago
0
error:RuntimeError: Error(s) in loading state_dict for CLIPVisionModel: size mismatch for vision_model.embeddings.class_embedding: copying a param with shape torch.Size([1024]) from checkpoint, the shape in current model is torch.Size([768]).

#175 zapqqqwe opened 3 months ago
3
total_frames zero error

#174 OliverLeeXZ opened 3 months ago
0
pretrained checkpoint

#173 OliverLeeXZ opened 3 months ago
0
How to increase the sample frames amount

#172 sherlock666 opened 3 months ago
0
Issues with finetune_lora.sh

#171 shag1802 opened 3 months ago
2
Error with Gradio Client: Please Upgrade Gradio to 4.x and Redeploying HuggingFace Space

#170 zhanwenchen opened 3 months ago
0
Question Regarding Video Frame Processing

#169 Kkkaystone opened 3 months ago
0
How to install videollava together with xformer?

#168 zengbohan0217 opened 3 months ago
0
extremely slow with transformers

#167 RaulKite opened 3 months ago
0
Training help

#166 felmoreno1726 opened 3 months ago
0
About class embedding

#165 feiyu12138 opened 3 months ago
0
Video-LLava Upgradation

#164 Tortoise17 opened 4 months ago
1
ERROR opening+moov atom not found+mmco: unref short failure

#163 Frank-Dg opened 4 months ago
2
Can the confidence coefficient of an answer be obtained?

#162 IsabelJimenez99 closed 4 months ago
0
Inference model path unclear

#161 Ali2500 opened 4 months ago
0
Fix typo and additional instruction in readme

#160 nahidalam closed 4 months ago
0
Please specifiy library versions

#159 nahidalam opened 4 months ago
0
Update README.md

#158 OliverGrace opened 4 months ago
0
Uri validation issue on Replicate

#157 Gab1988 closed 4 months ago
1
Video-LLaVa now available in the Transformers library!

#156 zucchini-nlp opened 4 months ago
49
chore: Update processing_video.py

#155 githubartema opened 4 months ago
0
The problem about the environment

#154 swiftCC closed 4 months ago
0
Some weights of the model checkpoint at "./Video-LLaVA-7B" were not used when initializing LlavaLlamaForCausalLM:

#153 ssuncheol opened 4 months ago
0
Size mismatch error when running locally.

#152 ssuncheol closed 4 months ago
3
Problem about pretrain parameter dim size is differen to the model dim size?

#151 NEC09818 opened 4 months ago
1
how to load pretrained weight on local (offline)?

#150 jusepv opened 4 months ago
0
Warnings about weights, temperature, top_p, and embedding layer, but it still works. Should I worry about them?

#149 secretlycarl opened 4 months ago
0
Impossible to install on windows

#148 secretlycarl opened 5 months ago
0
add-llama3

#147 Namzakku closed 5 months ago
0
推理多张图片时报错 IndexError: list index out of range

#146 Qinger27 opened 5 months ago
1
Multi-GPU inference enabled following LLaVA repo

#145 shouborno opened 5 months ago
0
训练时报错AttributeError: 'DeepSpeedCPUAdam' object has no attribute 'ds_opt_adam'

#144 Qinger27 opened 5 months ago
5
Unable to install flash attn module

#143 anantalp closed 5 months ago
1
Seems it has very limited understanding ability..

#142 advenTure423 opened 5 months ago
0