How to create train, test, dev splits?

soskek / bookcorpus

Crawl BookCorpus

MIT License

812 stars 110 forks source link

Closed kgarg8 closed 3 years ago

kgarg8 commented 3 years ago

After downloading the entire corpus as a single txt file, how can we create the train, test, dev splits?

soskek commented 3 years ago

I didn't define any splits. You are free to choose your split with your crawled one. Thanks!