OSU-NLP-Group / SeeAct

[ICML'24] SeeAct is a system for generalist web agents that autonomously carry out tasks on any given website, with a focus on large multimodal models (LMMs) such as GPT-4V(ision).
https://osu-nlp-group.github.io/SeeAct/
Other
571 stars 69 forks source link

Lack of evalution code of offline evalution of mm-mind2web #33

Open leoozy opened 3 months ago

leoozy commented 3 months ago

Could you please provide the complete offline evaluation code for mm-mind2web? Currently, only the prediction demo code is available, lacking the full dataset loop and evaluation metric to reproduce the results in Table 2. Thank you.