Closed jxmorris12 closed 2 years ago
Random sampling with replacement seems to fix the sawtooth shape we saw in the loss, especially on fine-tuning QQP.
See PyTorch forum post: Observing strange loss jumps between epochs
Builds on #23
Random sampling with replacement seems to fix the sawtooth shape we saw in the loss, especially on fine-tuning QQP.
See PyTorch forum post: Observing strange loss jumps between epochs
Builds on #23