Thanks for posting a really nice repo!
While I was studying the code, I found that in 'train_dense_encoder.py' line 669 and 691 the following:
'''
surrogate = surrogate * (trainer.distributed_factor / 8.)
'''
which I actually don't fully understand the reason of the multiplication part.
Can you explain any reason? Thank you 👍
Thanks for posting a really nice repo! While I was studying the code, I found that in 'train_dense_encoder.py' line 669 and 691 the following: ''' surrogate = surrogate * (trainer.distributed_factor / 8.) ''' which I actually don't fully understand the reason of the multiplication part. Can you explain any reason? Thank you 👍