About MSE on validation and test

[bhagiradh]

Why was test MSE lower than the validation MSE? I was seeing the ipynb notebook you ran.

[wuhaixu2016]

I think this depends on the similarity of train and test data. Most datasets split train, val, test datasets based on chronological order. Considering the weather forecasting task, the train set contains spring, summer and autumn; the val set will be winter; the test set could be spring again. Thus, I think comparing mse on the test and validating sets is kind of meaningless.

[bhagiradh]

ThankYou for the response. Does the same thing apply for all other datasets ? for weather data thats a good explanation. But in your predcit.ipynb you were using ETTh1 dataset.