We need a simple GUI using Gradio to facilitate blind pairwise evaluations. The interface should allow users to compare two options (A vs B) without knowing which is which, with the options being shuffled randomly for each evaluation. This will help us gather unbiased feedback before deploying new changes.
Requirements
Create a Gradio-based GUI.
Implement functionality for blind pairwise comparisons (A vs B).
Ensure that the options are randomly shuffled for each evaluation.
Collect and store user feedback for further analysis.
TODO's
[ ] Adapt LMSys Gradio UI
[ ] Create necessary DB tables
[ ] Ensure everything is working together on the backend, including options, shuffling the options, logging to the DB, multi-user support, etc.
Additional Notes
Consider potential issues arising from multiple users using the same Gradio app.
Description
We need a simple GUI using Gradio to facilitate blind pairwise evaluations. The interface should allow users to compare two options (A vs B) without knowing which is which, with the options being shuffled randomly for each evaluation. This will help us gather unbiased feedback before deploying new changes.
Requirements
TODO's
Additional Notes