How does VisTransformer working?

timeseriesAI / tsai

Time series Timeseries Deep Learning Machine Learning Python Pytorch fastai | State-of-the-art Deep Learning library for Time Series and Sequences in Pytorch / fastai

https://timeseriesai.github.io/tsai/

Apache License 2.0

5.1k stars 639 forks source link

How does VisTransformer working? #648

Closed liyy2 closed 1 year ago

liyy2 commented 1 year ago

Not understand the difference b/w TSiT and TST

oguiza commented 1 year ago

They are similar (both are transformer models that work with time series), but built on 2 different papers:

TST is based on the "A Transformer-based Framework for Multivariate Time Series Representation Learning" paper, while
TSiT is adapted from this paper "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale", which is originally a vision transformer.

oguiza commented 1 year ago

Closing this issue due to lack of activity and progress. If necessary please, create a new one.