-
Hello,
I am trying to evaluate LLaVA OneVision 72B, but finding I need to use tensor-parallelism to fit it on memory. However, when I do, evaluating on datasets (e.g., MLVU) takes 90+hrs on 4 A100s…
-
When I open http://analysis.a1.luyouxia.net:23226/, it tells:
使用 TCP 映射用于 HTTP 协议访问时，请使用分配的域名加端口进行访问，不支持使用其它域名访问。
（无效主机头: analysis.a1.luyouxia.net）
-
### Hello, I have an issue with executing the evaluation with MMBench-Video dataset:
I try to execute `run.py` with _MMBench-Video_ dataset, but there are two scenarios that happen everytime I try to…
-
Hi, I have encountered the same problem. I have two questions about the evaluation:
(1) How many video frames are used when assessing performance using the 5 benchmarks, including MVBench, MLVU, MM…
-
## タイトル: LongVU: 長時間ビデオ言語理解のための時空間適応圧縮
## リンク: https://arxiv.org/abs/2410.17434
## 概要:
マルチモーダル大規模言語モデル（MLLM）は、ビデオコンテンツの理解と分析において著しい進歩を遂げてきました。しかし、長いビデオの処理は、LLMのコンテキストサイズによって制限される大きな課題として残っています。この制…
-
Hello, thanks for your excellent work and I have some questions here:
1. What's the difference between MLVU dataset and MLVU_Test dataset?
2. I clone the project from GitHub to the local host, whet…
-
Hi,
Why are the results in the mini-leaderboard and full leaderboard as well as paper table different for the same methods?
-
-
Thanks for your work in MLVU. I found some errors in the benchmark, i.e., some answers are not in the candidates:
-
Hi authors, thanks for your great work.
I try to evaluate Oryx-7b on MVBench and use the script:
export HF_HOME="/mnt/data/datasets/hub"
export GPT_EVAL_VERSION="gpt-3.5-turbo"
export OPENAI_A…