Open Muennighoff opened 1 year ago
Would be great if the HFDataLoader would support dataset revisions.
Also imo it would be much better to merge the qrels HF repos with the regular ones & load the entire dataset via .load_dataset() using a dataset script on the repos.
.load_dataset()
Would be great if the HFDataLoader would support dataset revisions.
Also imo it would be much better to merge the qrels HF repos with the regular ones & load the entire dataset via
.load_dataset()
using a dataset script on the repos.