microsoft / FLAML

A fast library for AutoML and tuning. Join our Discord: https://discord.gg/Cppx2vSPVP.
https://microsoft.github.io/FLAML/
MIT License
3.76k stars 495 forks source link

add tests for prompts with math problems. #1205

Closed yiranwu0 closed 7 months ago

yiranwu0 commented 10 months ago

Why are these changes needed?

Add test of 17 math problems to "openai" workflow when being deployed. The prompt must solve at least 8/17 problems to pass the test.

Related issue number

Checks