[Issue]: Would be Nice to Have a Powerful WebSurfer Team Submitted to LMSys.org - Githubissues

microsoft / autogen

A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

https://microsoft.github.io/autogen/

Creative Commons Attribution 4.0 International

28.11k stars 4.1k forks source link

[Issue]: Would be Nice to Have a Powerful WebSurfer Team Submitted to LMSys.org #2898

Open Josephrp opened 3 weeks ago

Josephrp commented 3 weeks ago

Describe the issue

LMSys.org

Large Model Systems Organization (LMSYS Org) is an open research organization founded by students and faculty from UC Berkeley in collaboration with UCSD and CMU.

Chatbot Arena

Scalable and gamified evaluation of LLMs via crowdsourcing and Elo rating systems.

Rationale

LMSys would/could give visibility to autogen
LMSys basically now allows "agents" to be validated/verified
hosting a permament provider would provide a good reason to get a pretty huge budget
it would be nice for a "basic" autogen agent team to "win" on a leaderboard

Steps to reproduce

get a budget
host a websurfing team
demonstrate autogen in production
register the host on LMSys
get potentially hundreds of thousands of users evaluating autogen

Screenshots and logs

Additional Information

maybe it's not a good idea , i dont really know :-) close this issue at your convenience , but you heard it here first ;-)

ghost commented 3 weeks ago

I didn't submit this

qingyun-wu commented 2 weeks ago

Great suggestion! @skzhang1 @LeoLjl @JieyuZ2 @LinxinS97 @yiranwu0 to consider!