Open james20141606 opened 1 week ago
Thanks for the question. However, I do not have validation dataset incorporated during training. Feel free to try it by your own!
Thanks for your reply! By the way do you have validation data in the finetuning stage?
And I have two extra questions which I am confused with:
feature
Hi, I am trying to redo the pretrain step as you described in the readme doc. The training loss converges pretty fast. I find the logs in wandb and it turned out to be only containing the training loss. I wonder if you could add other metrics, like validation loss and perplexity.
Thanks a lot!