It seems that lines like this create a new iterator every time the function is called? If so, this will not correctly transverse the epoch and also super slow when the corpus is large. I feel the right way to do it might be to create an static iterator self.train_iterator = iter(self.train_iter) at initialization time, and only next(self.train_iterator) in next_batch.
It seems that lines like this create a new iterator every time the function is called? If so, this will not correctly transverse the epoch and also super slow when the corpus is large. I feel the right way to do it might be to create an static iterator
self.train_iterator = iter(self.train_iter)at initialization time, and onlynext(self.train_iterator)innext_batch.