openai_utils.py throws Unclosed connector Error

[maksimstw]

I was using openai_utils.py for ChatGPT inference. It always worked fine for the first couple hundred samples, and then it always crushed with the following error. I tried to lower request_per_minute but the problem persists.

Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x000001FC0B9CFD10>
 92%|████████████████████████████████████████████████████████████████████████      | 1360/1473 [10:27<00:52,  2.17it/s]
Unclosed connector
connections: ['[(<aiohttp.client_proto.ResponseHandler object at 0x000001FC0C137AF0>, 482899.734), (<aiohttp.client_proto.ResponseHandler object at 0x000001FC0C2A5550>, 482900.89)]']
connector: <aiohttp.connector.TCPConnector object at 0x000001FC0B9CFD50>

[neubig]

Thanks for the report! I've run on much larger datasets without encountering this error, so this might be an environment-specific issue. Could you share some information about:

1. What OS are you using
2. What Python version are you using
3. What version of the OpenAI library are you using

[maksimstw]

Thanks for the quick reply!

1. OS: Windows 11
2. Python version: 3.11.3
3. OpenAI Library: api_type = "azure" api_version = "2023-03-15-preview" model="gpt-3.5-turbo"

[neubig]

@maksimstw : could you please try pulling this branch and see if it fixes your issues? https://github.com/zeno-ml/zeno-build/pull/125

If this doesn't work, it will probably be hard for me to debug the issue as I can't reproduce the problem, but I would definitely welcome any fixes!

[maksimstw]

The error is mainly due to "openai.error.APIConnectionError" and "openai.error.Timeout", which are not handled in your script. Those exceptions frequently occur when the user is using a VPN (especially for users in China). Those exceptions further result in an unclosed connector error. After properly handling those 2 exceptions, the script works as expected. You could also consider handling openai.error.InvalidRequestError as some prompts might be filtered by OpenAI API.

[neubig]

Oh, great, thanks a lot! If you could send a PR with the additional exception handling that would be greatly appreciated.

[maksimstw]

@neubig: could you please review my pull request #126? Thank you!

[neubig]

Done, thanks for the contribution! Closing the issue: https://github.com/zeno-ml/zeno-build/pull/126