I am trying to train on a custom dataset however I can not process the dataset. Mapping gives this error " Column to remove ['validation'] not in the dataset. Current columns in the dataset: ['text']". I am using the below code as similar to other datasets. Could you give a working example of a custom dataset like the one I am using?
babylm = datasets.load_dataset("asparius/babylm-10m","all.txt")e_babylm = ELECTRAProcessor(babylm).map(num_proc=1)
I am trying to train on a custom dataset however I can not process the dataset. Mapping gives this error " Column to remove ['validation'] not in the dataset. Current columns in the dataset: ['text']". I am using the below code as similar to other datasets. Could you give a working example of a custom dataset like the one I am using?
babylm = datasets.load_dataset("asparius/babylm-10m","all.txt")e_babylm = ELECTRAProcessor(babylm).map(num_proc=1)