hendrycks / apps

APPS: Automated Programming Progress Standard (NeurIPS 2021)
MIT License
414 stars 55 forks source link

update the computation of scores and errors #12

Closed loubnabnl closed 2 years ago

loubnabnl commented 2 years ago

Hi @xksteven, following on our discussion yesterday #11, I just had the chance to run the changes on actual solutions and there are some problems

Feel free to run more tests or if you have another suggestion to fix this.

xksteven commented 2 years ago

Do you think we can add a test case to avoid any more bugs? Want to make sure the inputs and outputs are what's expected.