search

jongwooko / distillm

Official PyTorch implementation of DistiLLM: Towards Streamlined Distillation for Large Language Models (ICML 2024)
https://arxiv.org/abs/2402.03898
145 stars 21 forks source link

The metric of GPT-4 Eval #2

Closed liuxy1103 closed 8 months ago

liuxy1103 commented 8 months ago

How is GPT-4 Eval calculated? Can you provide the following assessment codes in addition to rouge-L?

jongwooko commented 8 months ago

Hi. Thank you for showing interest in our work.

The baseline code for our GPT-4 evaluation is available at https://github.com/raymin0223/fast_robust_early_exit/tree/main/gpt_eval.

We will make our customized evaluation code available as soon as possible.