Why don't add sorted(problems) to pin problems sorting?

hendrycks / apps

APPS: Automated Programming Progress Standard (NeurIPS 2021)

MIT License

414 stars 55 forks source link

Why don't add sorted(problems) to pin problems sorting? #3

Closed guoday closed 3 years ago

guoday commented 3 years ago

Hi, thanks for your hard works.

Whether should we add a line of code problems = sorted(problems) here to pin some ordering? Just like: https://github.com/hendrycks/apps/blob/473a49713a708191b51175691553d88291f8e907/eval/generate_gpt_codes.py#L113

If I have any mistake, please point out.

Thanks

xksteven commented 3 years ago

Thanks for the suggestion! Feel free to make a pull request next time.
I did the fix and pushed it. Let me know if you encounter any other issues or have other suggestions :)