EvolvingLMMs-Lab / lmms-eval

Accelerating the development of large multimodal models (LMMs) with lmms-eval
https://lmms-lab.github.io/
Other
1.03k stars 54 forks source link

Propose to print error information when import models #123

Open SDaoer opened 1 week ago

SDaoer commented 1 week ago

In the file /root/lmms-eval/lmms_eval/models/__init__.py, no output is printed if an ImportError occurs. This can lead to confusion for users who encounter a ValueError stating 'Attempted to load model '...', but no model for this name found!' in the file /root/lmms-eval/lmms_eval/api/registry.py, specifically at line 32, within the get_model function. This error might actually be triggered by a previous ImportError from {model}.py, but because the import error doesn’t display any associated messages, users remain unaware of the underlying issue.

recommend:

for model_name, model_class in AVAILABLE_MODELS.items():
    try:
        exec(f"from .{model_name} import {model_class}")
    except ImportError as e:
        print(f"Failed to import {model_class} from {model_name}: {e}")
        pass
Luodian commented 1 week ago

Thanks for this proposal, I will integrate it in a new PR.