One-line summary: Pre-commit hooks were failing. I identified the main cause, and then fixed all secondary pre-commit issues. I only changed the logic in one place, oiaevalset.py.
I was having issues with type-hinting and identified that the old typings directory was causing the from openai import OpenAI import to register as an error. I decided to go through and fix all the issues that appeared in pre-commit run --all-files.
NOTE:
I changed the logic in oaievalset.py by adding a continue statement if an eval or eval.key was missing.
As far as I can tell this should basically never happen, but is correct behavior.
Another option would be to assert that eval and eval.key are not None but forcing an error here doesn't match what I interpret as intended behavior.
The manual work involved was mainly:
Deleting the typings directory, which was interfering with openai type-hints (such as from openai import OpenAI)
Fixing type issues in oaievalset.py.
Moving the client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY")) line below all the imports.
Breaking lines of length >767 into smaller chunks using line continuation.
Thus this PR is broken into three parts:
Deleting typings (first commit)
Manually cleaning up issues (middle commits)
Applying autofixes from the pre-commit hooks (last commit)
(Not an eval)
One-line summary: Pre-commit hooks were failing. I identified the main cause, and then fixed all secondary pre-commit issues. I only changed the logic in one place,
oiaevalset.py.I was having issues with type-hinting and identified that the old
typingsdirectory was causing thefrom openai import OpenAIimport to register as an error. I decided to go through and fix all the issues that appeared inpre-commit run --all-files.NOTE:
oaievalset.pyby adding acontinuestatement if anevaloreval.keywas missing.evalandeval.keyare notNonebut forcing an error here doesn't match what I interpret as intended behavior.The manual work involved was mainly:
typingsdirectory, which was interfering withopenaitype-hints (such asfrom openai import OpenAI)oaievalset.py.client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))line below all the imports.Thus this PR is broken into three parts:
typings(first commit)