search

yuqinie98 / PatchTST

An offical implementation of PatchTST: "A Time Series is Worth 64 Words: Long-term Forecasting with Transformers." (ICLR 2023) https://arxiv.org/abs/2211.14730
Apache License 2.0
1.38k stars 248 forks source link

a question about head type of self-supervised PatchTST #8

Closed stillwaterman closed 1 year ago

stillwaterman commented 1 year ago

Hi, I got a question about the head of self-supervised PatchTST. In my opinion, self-supervised PatchTST use a D × P linear layer(self.create_pretrain_head or PretrainHead) to do the pretrain process，then remove this head and attach a PredictionHead(Flatten_Head) to do end-to-end finetune or linear probing, am I right? By the way, there are two versions of PatchTST in PatchTST_self_supervised and PatchTST_supervised, which one is the lastest version？

yuqinie98 commented 1 year ago

Hi @stillwaterman , thanks for your attention! Yes your understanding is correct.

The two folders here are both the latest versions of PatchTST. PatchTST_supervised is developed based on some previous papers like Informer, Autoformer, FEDformer, DLinear... we here use the same codebase to make a fair comparison. PatchTST_self_supervised is developed based on tsai repo, which is another popular codebase of building time series ai models.

stillwaterman commented 1 year ago

Thanks for your immediate reply, I understand!