When I trying to use local dataset path instead of lmms-lab/MME inlmms_eval/tasks/mme/mme.yaml, No tasks specified, or no tasks found ERROR occurred.
This is the only one change in mme.yaml filedataset_path: D:/xxxx/hf_cache/MME dataset_kwargs: token: false
This is the full traceback:
warnings.warn(
SGLang is not installed. If you want to use llava_sglang, please install it using pip install 'sglang[all]'
06-11 00:01:07 [lmms-eval\lmms_eval__main.py:165] INFO Verbosity set to INFO
06-11 00:01:07 [lmms-eval\lmms_eval__main.py:226] INFO Verbosity set to INFO
06-11 00:01:09 [lmms-eval\lmms_eval__main.py:261] INFO Evaluating on 1 tasks.
06-11 00:01:09 [lmms-eval\lmms_eval__main__.py:267] ERROR Tasks were not found: mme. Try lmms-eval --tasks list for list of available tasks
06-11 00:01:09 [lmms-eval\lmms_eval\main__.py:272] INFO Selected Tasks: []
Traceback (most recent call last):
File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\main.py", line 199, in cli_evaluate
results, samples = cli_evaluate_single(args)
File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\main.py", line 286, in cli_evaluate_single
results = evaluator.simple_evaluate(
File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\utils.py", line 450, in _wrapper
return fn(*args, **kwargs)
File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\evaluator.py", line 83, in simple_evaluate
assert tasks != [], "No tasks specified, or no tasks found. Please verify the task names."
AssertionError: No tasks specified, or no tasks found. Please verify the task names.
06-11 00:01:09 [lmms-eval\lmms_eval\main__.py:213] ERROR Error during evaluation: No tasks specified, or no tasks found. Please verify the task names.
Traceback (most recent call last):
File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\main.py", line 199, in cli_evaluate
results, samples = cli_evaluate_single(args)
File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\main__.py", line 286, in cli_evaluate_single
results = evaluator.simple_evaluate(
File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\utils.py", line 450, in _wrapper
return fn(*args, **kwargs)
File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\evaluator.py", line 83, in simple_evaluate
assert tasks != [], "No tasks specified, or no tasks found. Please verify the task names."
AssertionError: No tasks specified, or no tasks found. Please verify the task names.
But I momentarily solve it by delete one line code in api/task.py ,delete this line will fix this problem self.dataset_path = inspect.getfile(import_module(self.dataset_path))
When I trying to use local dataset path instead of
lmms-lab/MMEinlmms_eval/tasks/mme/mme.yaml,No tasks specified, or no tasks foundERROR occurred.This is the only one change in
mme.yamlfiledataset_path: D:/xxxx/hf_cache/MME dataset_kwargs: token: falseThis is the full traceback: warnings.warn( SGLang is not installed. If you want to use llava_sglang, please install it using pip install 'sglang[all]' 06-11 00:01:07 [lmms-eval\lmms_eval__main.py:165] INFO Verbosity set to INFO 06-11 00:01:07 [lmms-eval\lmms_eval__main.py:226] INFO Verbosity set to INFO 06-11 00:01:09 [lmms-eval\lmms_eval__main.py:261] INFO Evaluating on 1 tasks. 06-11 00:01:09 [lmms-eval\lmms_eval__main__.py:267] ERROR Tasks were not found: mme. Try
lmms-eval --tasks listfor list of available tasks 06-11 00:01:09 [lmms-eval\lmms_eval\main__.py:272] INFO Selected Tasks: [] Traceback (most recent call last): File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\main.py", line 199, in cli_evaluate results, samples = cli_evaluate_single(args) File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\main.py", line 286, in cli_evaluate_single results = evaluator.simple_evaluate( File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\utils.py", line 450, in _wrapper return fn(*args, **kwargs) File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\evaluator.py", line 83, in simple_evaluate assert tasks != [], "No tasks specified, or no tasks found. Please verify the task names." AssertionError: No tasks specified, or no tasks found. Please verify the task names. 06-11 00:01:09 [lmms-eval\lmms_eval\main__.py:213] ERROR Error during evaluation: No tasks specified, or no tasks found. Please verify the task names. Traceback (most recent call last): File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\main.py", line 199, in cli_evaluate results, samples = cli_evaluate_single(args) File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\main__.py", line 286, in cli_evaluate_single results = evaluator.simple_evaluate( File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\utils.py", line 450, in _wrapper return fn(*args, **kwargs) File "C:\Users\Administrator\PycharmProjects\lmms-eval\lmms_eval\evaluator.py", line 83, in simple_evaluate assert tasks != [], "No tasks specified, or no tasks found. Please verify the task names." AssertionError: No tasks specified, or no tasks found. Please verify the task names.But I momentarily solve it by delete one line code in
api/task.py,delete this line will fix this problemself.dataset_path = inspect.getfile(import_module(self.dataset_path))Thank you for your contributions!