Question about the mosei dataset

[TianhaoFu]

Hi, Thanks for your code.

When I use your code to train a model in mosei dataset, I find that after 10 epoch, the model became overfiting. Is this normal?

By the way, in your example, you train your model 1000 epoch. Is this superparameter the result of your experiment?

Thanks!

My model is as follows:

encoders=[GRU(35,300,dropout=True,has_padding=False).cuda(), \
    GRU(74,300,dropout=True,has_padding=False).cuda(),\
    GRU(300,300,dropout=True,has_padding=False).cuda()]
head=MLP(300,150,1).cuda()
fusion =add()

[Vanvan2017]

Yeah, thanks for the question, for the large dataset, it will be overfitting very quickly, so you can set the epoch to a small value, actually, 20 to 100 is a good choice, and you can also apply the early stop with patience 7 to 20 ~

[TianhaoFu]

Thanks for your reply!

When i train my own model, I found that after 100 epoch training , the valid loss from 0.76 changed to 0.62. It seems that after training, the model effect is not significantly improved, is this normal? @Vanvan2017

[Vanvan2017]

One possible reason is that the loss is L1Loss, so the improvement on the regression may not lead to the improvement on the classification as well, but generally good MAE will promise a good Acc (this may fail on some small sentiment score especially between -1.0 to 1.0). Maybe there are also some overfitting problems.

[TianhaoFu]

So should I use some classification loss functions to train my model? By the way, why you train the classification task as a regression task? Is this a common practice?

Thanks!

One possible reason is that the loss is L1Loss, so the improvement on the regression may not lead to the improvement on the classification as well, but generally good MAE will promise a good Acc (this may fail on some small sentiment score especially between -1.0 to 1.0). Maybe there are also some overfitting problems.

[Vanvan2017]

So should I use some classification loss functions to train my model? By the way, why you train the classification task as a regression task? Is this a common practice?

Thanks!

One possible reason is that the loss is L1Loss, so the improvement on the regression may not lead to the improvement on the classification as well, but generally good MAE will promise a good Acc (this may fail on some small sentiment score especially between -1.0 to 1.0). Maybe there are also some overfitting problems.

Sure, you can use BCE or CrossEntropy to train that, but by default, the MOSEI is a regression dataset, and previous literature uses regression loss (MAE), it may loss the information of how strong the sentiments when just using 0 or 1 label I think. But actually, take everything into traditional classification setting is also a good standard way to evaluate your own model.