search

hkust-nlp / AgentBoard

An Analytical Evaluation Board of Multi-turn LLM Agents
219 stars 22 forks source link

[Request] Google Gemini Pro #4

Closed abdinal1 closed 2 months ago

abdinal1 commented 6 months ago

Have you or can you evaluate Google Gemini Pro to put as comparison?

chang-github-00 commented 6 months ago

Thanks for the suggestion, though we have not yet tested the performance of Google Gemini Pro on AgentBoard. We would evaluate it once we have the resources. BTW, we also welcome everyone to contribute evaluation results of other models to our email. If you have run Google Gemini Pro on AgentBoard, you can kindly send the log file to our email (llmagentboard@gmail.com).

chang-github-00 commented 2 months ago

We've recently add evaluation on Google Gemini-flash, please check our updated arxiv.