Closed neubig closed 2 years ago
Hi, @neubig thanks for these PRs, which are great and I think reflected many things we discussed last meeting.
I carefully read these PRs, instead of commenting on them one by one, I made a holistic suggestion here: https://github.com/neulab/explainaboard_client/issues/38 (since the final goal is all about the users, so maybe having a nice overall picture w.r.t the UI is good.
Right now the API is complicated, requiring creation of undocumented data members and use of two libraries
explainaboard_client
andexplainaboard_api_client
. We should redesign this to be simpler. Here are some functions to be implemented.evaluate_system_file()
: Perform evaluation from system output files (@neubig is working on this now)evaluate_system()
: Perform evaluation from data that is stored in memoryfind_systems()
: Find systemsdelete_system()
: Delete a system by ID