Add CrowS-Pairs to Full Benchmark

bigscience-workshop / evaluation

Code and Data for Evaluation WG

Other

41 stars 24 forks source link

Add CrowS-Pairs to Full Benchmark #37

Open epavlick opened 3 years ago

trishalaneeraj commented 3 years ago

I can do this one.

oskarvanderwal commented 3 years ago

@trishalaneeraj, do you need any help?

aneveol commented 3 years ago

@trishalaneeraj We are discussing how to contribute to this issue in the bias, fairness and social impact evaluation subgroup. It would be great to coordinate on this. We have an upcoming meeting Monday September 6 at 10 pm CET (details are in slack channel)

oskarvanderwal commented 2 years ago

@aneveol and I will work on making CrowS-Pairs ready for the prompt evaluation:

[ ] Uploading the updated version by @aneveol's team to the HuggingFace Dataset hub.
[ ] Write prompts for the dataset (both in English and French).
[ ] Write evaluation code.

jzf2101 commented 2 years ago

See also PR bigscience-workshop/promptsource#742