Have you ever tried using pre-train Tensor for prediction instead of finetune?

yuqinie98 / PatchTST

An offical implementation of PatchTST: "A Time Series is Worth 64 Words: Long-term Forecasting with Transformers." (ICLR 2023) https://arxiv.org/abs/2211.14730

Apache License 2.0

1.51k stars 262 forks source link

Have you ever tried using pre-train Tensor for prediction instead of finetune? #23

Closed LiuHao-THU closed 1 year ago

LiuHao-THU commented 1 year ago

Have you ever tried using pre-train Tensor for prediction instead of finetune? Bert Using Pretrained Tensor for Classification, The Results can compare with finetune results, can this be same in time series task?

namctin commented 1 year ago

@LiuHao-THU We did try that and the result was not so good. We did linear probing as well in which only the last layer is retrained. You can see the results in the paper.