search

huggingface / lighteval

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.
MIT License
471 stars 55 forks source link

Fix #66 #67

Closed clefourrier closed 4 months ago

clefourrier commented 4 months ago

Fix #66 We actually don't need to use the tokenizer.apply_chat_template for the possible choices, since the correct logic is taken care of in get_examples_with_chat_template, which adds a generation prompt start before the tokens

clefourrier commented 4 months ago

Can confirm it works well

lewtun commented 4 months ago

I've tested it as well and confirm it works - shall we merge?

clefourrier commented 4 months ago

Not merging without maintainer approval :) (in this case, @NathanHB , who's off on Tuesdays ^^)