Closed waltonfuture closed 1 year ago
Thanks for the issue. We have not yet integrated other models into our code framework, so for now please use the official code to generate answers and then run our common_eval_2d.py script. LLaVA and MiniGPT-4 provide demo code that allows you to easily obtain model answers by inputting the "image" and "query". We provide the data['image']
and data['query']
in the test data JSON file. Additionally, it's important to use the same system message in [src/datasets/system_msg.py](https://github.com/OpenLAMM/LAMM/blob/main/src/datasets/system_msg.py) to ensure a fair comparison.
Can you release the codes about how to send the datasets into LLaVA and MiniGPT-4 for eval? There are only codes about how to evaluate the context generated by LLaVA and MiniGPT-4.