how to apply this transformer model to a long univariate time series data

[Dezhao-Huang]

Dear all, First of all, thank you Max for sharing this amazing transformer implementation with us! I am able to successfully run the training.ipynb by following the #34 solutions using the x_train_LsAZgHU.csv and y_train_EFo1WyE.csv datasets. I have read the data descriptions on the Ozechallege_benchmark, for the input datasets(x_train_LsAZgHU.csv), there are 7500 rows and 12116 columns for the output datasets (y_train_EFo1WyE.csv), there are 7500 rows and 5377 columns each time series is 28 * 24 = 672

My question is If I have a long single-variable time series data, for example, a NumPy array length of [1,1000000], how to feed this long univariate time-series data to this transformer model? I would like to precisely predict the trend of this time series data.

how should I prepare the initial npz format dataset? how to set the d_input, d_output, and the batch size?

[maxjcohen]

Hi, answer in two parts:

• Is transformer really adapted to very long time series ? Short answer here is no, but it can be done with a bit of work. You'll want to look at MHAWindow (or MHAChunk) which cuts the sequence into chunks in order to process the input much faster. There can be some issues with these blocks if your time length is not a multiple of the window size (see #15).
• From an implementational perspective, simply consider your variable as a one dimension vector, i.e. your input should have shape (BATCH_SIZE, TIME_DIMENSION, 1). Same goes for the ouput. Set d_input = d_output = 1.