The dataset I am using is Book Corpus having 18000 books
The system i am training on is having 64GB of RAM
When I am trying to generate the pretraining data using create_pretraining_data.py it is getting killed in between and also only a single core is getting used.
Please give me a solution to this.
sairamgajavelli
commented
2 years ago
The dataset I am using is Book Corpus having 18000 books The system i am training on is having 64GB of RAM When I am trying to generate the pretraining data using create_pretraining_data.py it is getting killed in between and also only a single core is getting used. Please give me a solution to this.