Closed fhrzn closed 2 years ago
Found the issue, there were a mismatch on assigning variables at denoising input function.
Need to modify run_batch()
in the file model_t5.py
. Go to line 100 and 101, then change these lines
encoder_inputs[i] = msk
encoder_masks[i] = inp
into
encoder_inputs[i] = inp
encoder_masks[i] = msk
Oh my.. thank you for finding this bug. Really appreciate it!
Hello, first of all thanks for your amazing work! I am trying to barely run your code in Google Colab but got the training loss always nan when starting pretraining encoder. Among 15 epochs, it just stop after 1st epoch as the EarlyStopCallback noticed that the loss keep infinite since the beginning. If it possible, could you please check to reproduce my errors?
My environment:
Looking forward for your response, thank you)