Can Azure openai GPT-4 work as judge?

lm-sys / FastChat

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Apache License 2.0

36.56k stars 4.51k forks source link

Can Azure openai GPT-4 work as judge? #3371

Open don-tpanic opened 4 months ago

don-tpanic commented 4 months ago

Thanks for the effort.

I was wondering when using LLM as judge, can I choose which versions of GPT-4 to be used as judge? Specifically, is it possible to instead use Azure openai GPT-4 API? I couldn't seem to find relevant instructions. Thanks!

maxim-saplin commented 4 months ago

I have created a PR for that: https://github.com/lm-sys/FastChat/pull/3240