search

jzhang38 / TinyLlama

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
Apache License 2.0
7.3k stars 425 forks source link

How to compute metrics like ROUGE, BLEU in sft script? #160

Open dopu2k16 opened 4 months ago

dopu2k16 commented 4 months ago

Hi,

May you tell me how to use compute_metrcs() for sft tasks using LLM? https://github.com/jzhang38/TinyLlama/blob/main/sft/finetune.py @jzhang38 @ChaosCodes @Green-Sky @joennlae

jzhang38 commented 3 months ago

https://github.com/EleutherAI/lm-evaluation-harness

Probably this is the repo that you should look for.