How can I use BigBIO (Biomedical Dataset Library) in lm-evaluation-harness ?

bigscience-workshop / biomedical

Tools for curating biomedical training data for large-scale language modeling

454 stars 116 forks source link

How can I use BigBIO (Biomedical Dataset Library) in lm-evaluation-harness ? #908

Closed Davidwhw closed 10 months ago

Davidwhw commented 10 months ago

I want to use BigBIO library in the way described here biomedical/notebooks/promptengineering/. But I find that there is no base.py in lm-evaluation-harness. Did I miss some information about the latest way to use BigBIO in lm-evaluation-harness?

The following is the issue I submitted inlm-evaluation-harness: https://github.com/bigscience-workshop/lm-evaluation-harness/issues/160#issue-2022515420