A simple script to collect responses from OYD through the sdk (reusing the app.py module code) for evaluation.
Description
The script a) sets up the client simply by using the .env settings identical to those used in the web app; b) loops over a json file of questions; c) collects the responses from the OYD endpoint and d) formats them as required by AI Studio's evaluation system.
Contribution Checklist
[x] This is a standalone script - I haven't tested any regressions against the core web app.
[ ] I have built and tested the code locally and in a deployed app
[ ] For frontend changes, I have pulled the latest code from main, built the frontend, and committed all static files.
[ ] This is a change for all users of this app. No code or asset is specific to my use case or my organization.
[ ] I didn't break any existing functionality :smile:
Motivation and Context
A simple script to collect responses from OYD through the sdk (reusing the app.py module code) for evaluation.
Description
The script a) sets up the client simply by using the .env settings identical to those used in the web app; b) loops over a json file of questions; c) collects the responses from the OYD endpoint and d) formats them as required by AI Studio's evaluation system.
Contribution Checklist