search

PKU-YuanGroup / Video-LLaVA

【EMNLP 2024🔥】Video-LLaVA: Learning United Visual Representation by Alignment Before Projection
https://arxiv.org/pdf/2311.10122.pdf
Apache License 2.0
3.03k stars 220 forks source link

Eval w/ multiple GPUs #68

Open yaolug opened 10 months ago

yaolug commented 10 months ago

why https://github.com/PKU-YuanGroup/Video-LLaVA/blob/main/llava/eval/video/run_inference_video_qa.py#L122 this line is commented out?

orrzohar commented 9 months ago

HI @yaolug, I have ran eval with multiple GPUs; notice that when you have multiple GPUs available; https://github.com/PKU-YuanGroup/Video-LLaVA/blob/e93f4927eaa926ed8450b481fde95c994ed23d2d/scripts/v1_5/eval/run_qa_msrvtt.sh#L14-L17

in the forward eval files (for example, in msrvtt): https://github.com/PKU-YuanGroup/Video-LLaVA/blob/main/scripts/v1_5/eval/run_qa_msrvtt.sh

split the dataset into chunks which are combined after the run is finished: https://github.com/PKU-YuanGroup/Video-LLaVA/blob/e93f4927eaa926ed8450b481fde95c994ed23d2d/scripts/v1_5/eval/run_qa_msrvtt.sh#L41-L43