search

allenai / open-instruct

Apache License 2.0
1.21k stars 166 forks source link

Get Stuck while using alpaca-farm #141

Closed YSLIU627 closed 6 months ago

YSLIU627 commented 6 months ago

Hi, we are using your evaluation module for alpaca-farm task. We successfully got all the inference results for our model for evaluation. But it got stuck during the GPT4-annotation process. The last trace is INFO:httpx:HTTP Request: POST https://api.openai.com/v1/chat/completions "HTTP/1.1 200 OK" and there is no response anymore. I would appreciate it if you could offer help.

hamishivi commented 6 months ago

Hi, yeah I've seen this once or twice before, and it seems somewhat random. I would try just rerunning the eval. AlpacaEval caches annotations so it shouldn't cost too much extra. I'm making a PR for a newer version of AlpacaEval that hopefully fixes some of this! (#139)

YSLIU627 commented 6 months ago

Thank you very much!