search

mlfoundations / evalchemy

Automatic Evals for Instruction-Tuned Models
70 stars 3 forks source link

Integrate Berkeley Function Calling Leaderboard into Evalchemy #28

Open CharlieJCJ opened 1 day ago

CharlieJCJ commented 1 day ago

Source: https://gorilla.cs.berkeley.edu/leaderboard.html https://github.com/ShishirPatil/gorilla/tree/main/berkeley-function-call-leaderboard

CharlieJCJ commented 1 day ago

I'll lay down the TODOs here, I'll be main point of contact for the integration.

For visibility @HuanzhiMao @Fanjia-Yan @ShishirPatil

CharlieJCJ commented 1 day ago

FYI, evalchemy use github subtree for updating external codebase https://github.com/mlfoundations/evalchemy?tab=readme-ov-file#adding-external-evaluation-repositories, so our changes in BFCL codebase can be actively reflected in evalchemy.