search

OpenGVLab / LAMM

[NeurIPS 2023 Datasets and Benchmarks Track] LAMM: Multi-Modal Large Language Models and Applications as AI Agents
https://openlamm.github.io/
296 stars 16 forks source link

How to evaluate on LLaVA and MiniGPT-4？ #19

Closed waltonfuture closed 1 year ago

waltonfuture commented 1 year ago

Can you release the codes about how to send the datasets into LLaVA and MiniGPT-4 for eval? There are only codes about how to evaluate the context generated by LLaVA and MiniGPT-4.

Coach257 commented 1 year ago

Thanks for the issue. We have not yet integrated other models into our code framework, so for now please use the official code to generate answers and then run our common_eval_2d.py script. LLaVA and MiniGPT-4 provide demo code that allows you to easily obtain model answers by inputting the "image" and "query". We provide the data['image'] and data['query'] in the test data JSON file. Additionally, it's important to use the same system message in [src/datasets/system_msg.py](https://github.com/OpenLAMM/LAMM/blob/main/src/datasets/system_msg.py) to ensure a fair comparison.