search

lucidrains / reformer-pytorch

Reformer, the efficient Transformer, in Pytorch
MIT License
2.1k stars 254 forks source link

error in eval loss & ppl #147

Open dar-tau opened 3 years ago

dar-tau commented 3 years ago

Hey! https://github.com/lucidrains/reformer-pytorch/blob/2cbc36bb280c0a6de46d838baad11a533a015fc3/pretraining/self-supervised.py#L327

you are dividing the eval_loss & perplexity each time you pass through the loop

Cheers, Guy :)