Closed elk-cloner closed 4 years ago
I think you are right
Isn't there the same issue in other places? E.g. in trainer.py: https://github.com/huggingface/transformers/blob/97a375484c618496691982f62518130f294bb9a8/src/transformers/trainer.py#L305-L307
I forgot to re-add this in Trainer when merging #3800
It's on my todo-list, but feel free to open a PR if you can do it faster than I can
Great. Personally I've not yet upgraded to the newer version with trainer.py, so I'll leave it for you, thanks.
🐛 Bug
Information
I'm trying to fine-tune BERT model using
run_language_modeling.py
.Language I am using the model on is Persian:
The problem arises when using:
The tasks I am working on is:
But according to this issue there is a bug in
torch.utils.data.distributed.DistributedSampler
so that during different epochs shuffling operation doesn't work properly(it's not working). To solve this problem: according to pytorch official example here, we should addtrain_sampler.set_epoch(epoch)
before each new epoch at this lineTo reproduce
Steps to reproduce the behavior:
Expected behavior
Environment info
transformers
version: transformers==2.8.0