two bugs 1. I can not use quick start from the README.md . 2 'vlmutil check Llama-3.2-11B-Vision-Instruct ' will lead to error which is 'File "path/VLMEvalKit/vlmeval/dataset/__init__.py", line 168, in DATASET_TYPE if 'openended' in dataset.lower(): AttributeError: 'NoneType' object has no attribute 'lower''

[GoogleAlphaZero]

两个bug 1.在readme界面，无法访问到quick start 2.测试模型能否使用的时候，’vlmutil check Llama-3.2-11B-Vision-Instruct‘，会显示bug，定位了下代码，发现本质是没给dataset参数
Traceback (most recent call last): …… File "path/VLMEvalKit/vlmeval/tools.py", line 415, in cli CHECK(m) File "path/VLMEvalKit/vlmeval/tools.py", line 271, in CHECK res = model.generate(msg) File "path/VLMEvalKit/vlmeval/vlm/base.py", line 115, in generate return self.generate_inner(message, dataset) File "path/VLMEvalKit/vlmeval/vlm/llama_vision.py", line 197, in generate_inner if DATASET_TYPE(dataset) == 'MCQ' or DATASET_TYPE(dataset) == 'Y/N': File "path/VLMEvalKit/vlmeval/dataset/init.py", line 168, in DATASET_TYPE if 'openended' in dataset.lower(): AttributeError: 'NoneType' object has no attribute 'lower'

[kennymckormick]

Hi, @GoogleAlphaZero , The first problem has been fixed in PR #625 . The second problem has been fixed in a recent commit.

You can have another try to check.