Some questions about the prediction

maxjcohen / transformer

Implementation of Transformer model (originally from Attention is All You Need) applied to Time Series.

https://timeseriestransformer.readthedocs.io/en/latest/

GNU General Public License v3.0

842 stars 165 forks source link

Some questions about the prediction #45

Closed DICPZhou closed 3 years ago

DICPZhou commented 3 years ago

First of all, thanks for your wonderful work. When I train my data with this model, the following problems arise: Loss prediction The loss during training is accepted, but the difference between the predicted results and the true results is huge. The predicted results do not change over time. In fact, no matter which model is used, the predicted results do not change over time. For my datasets, the samples are 2000, the time sequence is 125, there are 11variables in X and one variable in Y. Attached is my data.

data.zip dataset_22_125.zip labels.zip

maxjcohen commented 3 years ago

Hi, your dataset doesn't seem to be normalized, do you train directly with the dataset you attached ? What are you preprocessing steps ?

DICPZhou commented 3 years ago

Sorry for the late reply，I used the Min-Max Normalization in src/dataset.py to preprocess the data. DATASET_PATH = 'datasets/data_22_125.npz' ozeDataset = OzeDataset(DATASET_PATH) dataset_train, dataset_val, dataset_test = random_split(ozeDataset, (1500, 2500, 250))

maxjcohen commented 3 years ago

Ok, so I'm assuming your rescaling your data before plotting, as your temperature is well above 1 here. I'm also assuming you're using your own dataset, as the temperature are quite high and not really smooth. So I'd encourage you to:

Try your dataset with these more traditional networks that are usually able to train on pretty much anything, such as the LSTM. There is one implemented here on the repo, you can also use the one directly available in PyTorch.
Try so good old fashion dataset debugging: plotting mean and std of your temperature at each hour, visualize input data, etc.