Open shikharsingla opened 4 years ago
Yes, it should theoretically be possible. https://stackoverflow.com/questions/63459657/how-to-load-large-dataset-to-gensim-word2vec-model
However, most corpora should normally not be more than a few GB. Are you sure your corpus is too large to be loaded into memory? How much memory does your computer have?
my corpus is too big to be put in one large file, my computer runs out of memory in doing that.
Is it possible to run this code on multiple files? or run it using iterator?