search

Zyphra / Zyda_processing

Apache License 2.0
22 stars 1 forks source link

About aggregate evaluation #4

Closed ftgreat closed 3 months ago

ftgreat commented 3 months ago

Thanks for your great paper. One question about aggregate evaluation, why metrics like MMLU are not included in aggregate evaluation?

yury-tokpanov commented 3 months ago

We do MMLU evaluation for our Zamba model: https://huggingface.co/Zyphra/Zamba-7B-v1 .

For the dataset ablations we performed the same evaluations as Dolma and Refinedweb for comparison purposes.