Open SujayTalanki opened 9 months ago
Currently, the model uses F.cross_entropy as the loss function. I was wondering if we could use F.mse_loss as the loss function, and what the hyperparameters to that function would look like?
I was wondering how this block would change. Thank you!
I don't think so. We have no meaningful labels in a language model for calculating mean squared error.
@SujayTalanki try looking into xVal, they used mse
Currently, the model uses F.cross_entropy as the loss function. I was wondering if we could use F.mse_loss as the loss function, and what the hyperparameters to that function would look like?
I was wondering how this block would change. Thank you!