Adapt loader to changes in source data - Githubissues

bigscience-workshop / biomedical

Tools for curating biomedical training data for large-scale language modeling

439 stars 111 forks source link

Adapt loader to changes in source data #903

Closed leonweber closed 8 months ago

leonweber commented 8 months ago

Adapt data loader to the following changes:

directory structure in source data changed
source data lost primary_evidence_index and secondary_evidence_index