Closed LITONG99 closed 2 months ago
The evaluation prompt follows the reference paperSelf-instruct: Aligning language model with self generated instructions.
. We provide user and neighbors information as background text, then ask ChatGPT to rate 3 question-answer pairs.
Dear authors, could you provide the template to employ ChatGPT as the evaluator to perform quantitative analysis?
According to the paper, the model is input with question-answering pairs of each test sample and uses the four-level (A, B, C, D) rating system. I am wondering what information is provided to the evaluator to make the judgment. For example, does it include the original text, neighborhood information, or the trained prompt?
Thank you.