Develop Gradio A/B Testing Interface for Pre-Deployment Feedback

[abdullah-alnahas]

Description

We need a simple GUI using Gradio to facilitate blind pairwise evaluations. The interface should allow users to compare two options (A vs B) without knowing which is which, with the options being shuffled randomly for each evaluation. This will help us gather unbiased feedback before deploying new changes.

Requirements

• Create a Gradio-based GUI.
• Implement functionality for blind pairwise comparisons (A vs B).
• Ensure that the options are randomly shuffled for each evaluation.
• Collect and store user feedback for further analysis.

TODO's

• [x] Adapt LMSys Gradio UI
• [x] Create necessary DB tables
• [x] Ensure everything is working together on the backend, including options, shuffling the options, logging to the DB, multi-user support, etc.

Additional Notes

• Consider potential issues arising from multiple users using the same Gradio app.

[abdullah-alnahas]

I created a separate repo for this app. iA, I am planning to

• [x] remove what is irrelevant to the ansari-backend from the feature/a-b-testing-gradio-app branch
• [ ] merge feature/a-b-testing-gradio-app to api-v2
• [ ] delete the feature/a-b-testing-gradio-app branch