a academic question about large time series model

[Sample-design-alt]

Thanks for you all great works for the large time series models! I am interested in the time series classification. But I have some questions about large model, I hope I can get some inspiration from your answers. 1、I found that the pretrained models, such as moment, UniTS, increase the performance mirror, the behind reason can be attributed to the lack of pretrained classification datasets? 2、Base on the above question, If we use a categorical dataset to pretrain model, what can we gain from the classification datasets? Labeling information (but the pre-trained labels don't share the label mapping of the downstream task)? Pattern features? (What are the explicit pattern differences between supervised classification datasets and self-supervised forecasting datasets?)

[WenWeiTHU]

Hi, if you want to pre-train a large model, large-scale unlabeled data is generally involved in the self-supervised paradigm. Supervised training is conducted in the downstream stage. However, there are some exceptions, such as in image recognition, where there is sufficient labeled data (such as ImageNet) to support pre-trained models. However, a common paradigm is to use only the feature extractor (a.k.a. model backbone) of the pre-trained model (removing the part of the classification head) to retrain a new task head on the previously unseen label.

[Sample-design-alt]

Thanks for your insight!