I have run your code and observed that during pretraining the generator, in the 'helper.py', function 'prepare_generator_batch', the start data of input is replaced to zero as a start letter. I am curious about the purpose of this operation and how it influences the loss calculation.
hello suragnair,
I have run your code and observed that during pretraining the generator, in the 'helper.py', function 'prepare_generator_batch', the start data of input is replaced to zero as a start letter. I am curious about the purpose of this operation and how it influences the loss calculation.
Looking forward to your reply. Thank you.