search

timeseriesAI / tsai

Time series Timeseries Deep Learning Machine Learning Python Pytorch fastai | State-of-the-art Deep Learning library for Time Series and Sequences in Pytorch / fastai
https://timeseriesai.github.io/tsai/
Apache License 2.0
5.15k stars 643 forks source link

adjust PatchTST into a classification mission. #773

Open Valhir924 opened 1 year ago

Valhir924 commented 1 year ago

If I want to use PatchTST into a classification project, what else should I do if using tsai. I've read the tutorials of PatchTST in a prediction work, I wonder if it is accustomed in TSclassifier.

vrodriguezf commented 1 year ago

Out of the box that's not possible, you should change the head of the model (now the head is in the class FlattenHead or sth like that). Anyway, I'm not sure that the premises of PatchTST, especially channel independence, are the most appropriate for time series classification.

Valhir924 commented 1 year ago

Thanks for reply. : ) I'm considering the feasibility of premises too, still confused. Maybe I will try to change the head to verify it. Yesterday I happened to read a similar issue about the dimension of output Y. The link is attached below. [(https://github.com/timeseriesAI/tsai/issues/713#issuecomment-1481507146)]

713

Oguiza mentioned that a plus version of PatchTST is under construction. It's great. May I ask how the progress of the project is going?

oguiza commented 1 year ago

Hi @Valhir924 , My exact comment was: "I'd like to develop a PatchTSTPlus model that allows to use it in scenarios like the one you described above. This is the normal process that I've followed in the library. The standard version replicates as close as possible the model published in the paper/ code. And then a Plus version add additional functionality. Forecasting any # variables is one of those scenarios. Another one is Classification or Regression. I'd like to work on this soon, but need to find the time or resources. Would you be interested in creating a PR? If your are interested I could you give you some direction on what needs to be done." Unfortunately, nobody volunteered to collaborate on this, so it means that it's in the "wish list". No progress has been made, and there's no current timeline to add it.

Valhir924 commented 1 year ago

So sorry to hear that. But anyway, thanks for reply and thank you for so much contribution so far. : )

benmfox commented 1 year ago

Hi @oguiza, I think I can help with the PatchTSTPlus model. I saw in a related issue that you mentioned in addition to the head, the RevIN layer needs to be modified. I looked at the paper for RevIN, and it appears that it is used primarily for forecasting. Does the indicator revin simply need to be false for other tasks (i.e. classification)? I can imagine that changes in the mean and variance of time series data over time means something (from a classification perspective).