search

tatsu-lab / alpaca_eval

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.
https://tatsu-lab.github.io/alpaca_eval/
Apache License 2.0
1.46k stars 232 forks source link

Will the annotator alpaca_eval_gpt4_turbo_fn also change? #410

Closed hsqmlzno1 closed 1 week ago

hsqmlzno1 commented 1 week ago

May I confirm if the annotator alpaca_eval_gpt4_turbo_fn will also consistently change and affect the final (length control) win rates?

YannDubs commented 1 week ago

Yes, that depends on OpenAI API and there's nothing we can do about the model silently changing.