search

Y-IAB / lm-evaluation-harness

A framework for few-shot evaluation of language models.
https://www.eleuther.ai
MIT License
0 stars 0 forks source link

[#25] Add Task List Metadata #26

Closed rifqiyan closed 5 months ago

rifqiyan commented 5 months ago

Background

It's hard for Puree to list available tasks on lm eval harness. Puree will need to scrape lot of yaml files to list available tasks. Current Solution: create a metadata file that list available tasks Next Step: I think it's not good to put the metadata here, we will move the metadata file to GCS or other place.

Changes