Is there a way to batch train GLoVe models?

roamanalytics / mittens

A fast implementation of GloVe, with optional retrofitting

Apache License 2.0

243 stars 31 forks source link

Is there a way to batch train GLoVe models? #20

Open ianchute opened 2 years ago

ianchute commented 2 years ago

I have a gigantic corpus of text to train on that leads to memory issues. Wondering if there was a way to do a batch training for GLoVe and/or Mittens models similar to partial_fit in some scikit-learn models?

RichardScottOZ commented 2 years ago

What size, out of interest? I have a fairly large one to try in the near future.