-
I don't know how to solve it, seem to be a problem with huggingface evaluate module.
Traceback (most recent call last):
File "/home/aka/anaconda3/envs/lmms/lib/python3.10/runpy.py", line 196, in _…
-
At the current stage there's a somehow simple way to include new tasks using the --include_external flag, yet there's no way to include external models except from cloning the lmms-eval repository and…
-
In the file `/root/lmms-eval/lmms_eval/models/__init__.py`, no output is printed if an ImportError occurs. This can lead to confusion for users who encounter a ValueError stating 'Attempted to load mo…
-
There is no multi-image for llava-onevision-data, https://huggingface.co/datasets/lmms-lab/LLaVA-OneVision-Data; Is the data complete?
-
Hi,
When testing models on llava model and videomme benchmarks, there is one error when loading videos. The `process_images` function can not read mp4 files.
https://github.com/EvolvingLMMs-Lab/…
-
Before you open an issue, please check if a similar issue already exists or has been closed before.
### When you open an issue, please be sure to include the following
- [ ] A descriptive title:…
-
Hello!
I am evaluating the llava-next-llama-3-8b model using lmms-eval, meeting this bug:
```
File "lmms-eval/lmms_eval/models/llava.py", line 358, in generate_until
conv = copy.deepcopy(con…
-
### Bug Summary
LMMS crashes when I try to save a project. This happens seemingly at random - I can't figure out what causes it.
Terminal output at time of error: /tmp/.mount_lmms-1BFjLc8/AppRun…
-
https://github.com/EvolvingLMMs-Lab/lmms-eval/blob/24dc435908d921e9f1a5706e3141b12e5d838d18/lmms_eval/models/instructblip.py#L9
It seems that the mmmu task does not have this utils_group_img file.
-
Getting ValueError: Unknown vision tower: google/siglip-so400m-patch14-384 on running https://github.com/LLaVA-VL/LLaVA-NeXT/blob/5fbcf27e32935f4e09d6b8b9f8abed4a572240b0/docs/LLaVA_OneVision_Tutorial…