Closed vinayakrajurs closed 2 years ago
We run the model on the test set after every epoch as shown in the log:
--------start to test----------- normed mse:0.1018, mae:0.2485, rmse:0.3191, mape:0.1800, mspe:0.0498, corr:0.7221 denormed mse:8.5757, mae:2.2802, rmse:2.9284, mape:inf, mspe:inf, corr:0.7221 Epoch: 17, Steps: 1062 | Train Loss: 0.2532137 valid Loss: 0.2102546 Test Loss: 0.2484858
.
If you want to run the test task separately, please pass --evaluate True
as parameter, this will activate the line 114~119 in run_ETTh.py and calls exp.test() to run the test. You can add plot function in the test function then.
Hope this can help you.
Thank you, I figured how to plot the results.
Hello, I'm trying to run the Code for the ETTh1 dataset using the following run command in Google Colab:
!pythonrun_ETTh_10.py --data ETTh1 --features S --seq_len 96 --label_len 48 --pred_len 48 --hidden-size 4 --stacks 1 --levels 3 --lr 3e-3 --batch_size 8 --dropout 0.5 --model_name etth1_M_I48_O24_lr3e-3_bs8_dp0.5_h4_s1l3
and it runs successfully for train before early stopping at epoch 17 `Args in experiment: Namespace(INN=1, RIN=False, batch_size=8, c_out=1, checkpoints='exp/ETT_checkpoints/', cols=None, concat_len=0, data='ETTh1', data_path='ETTh1.csv', dec_in=1, detail_freq='h', devices='0', dilation=1, dropout=0.5, embed='timeF', enc_in=1, evaluate=False, features='S', freq='h', gpu=0, groups=1, hidden_size=4.0, inverse=False, itr=0, kernel=5, label_len=48, lastWeight=1.0, levels=3, loss='mae', lr=0.003, lradj=1, model='SCINet', model_name='etth1_M_I48_O24_lr3e-3_bs8_dp0.5_h4_s1l3', num_decoder_layer=1, num_workers=0, patience=5, positionalEcoding=False, pred_len=48, resume=False, root_path='./datasets/', save=False, seq_len=96, single_step=0, single_step_output_One=0, stacks=1, target='OT', train_epochs=100, use_amp=False, use_gpu=True, use_multi_gpu=False, window_size=12) SCINet( (blocks1): EncoderTree( (SCINet_Tree): SCINet_Tree( (workingblock): LevelSCINet( (interact): InteractorLevel( (level): Interactor( (split): Splitting() (phi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (psi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (P): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (U): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) ) ) ) (SCINet_Tree_odd): SCINet_Tree( (workingblock): LevelSCINet( (interact): InteractorLevel( (level): Interactor( (split): Splitting() (phi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (psi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (P): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (U): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) ) ) ) (SCINet_Tree_odd): SCINet_Tree( (workingblock): LevelSCINet( (interact): InteractorLevel( (level): Interactor( (split): Splitting() (phi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (psi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (P): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (U): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) ) ) ) ) (SCINet_Tree_even): SCINet_Tree( (workingblock): LevelSCINet( (interact): InteractorLevel( (level): Interactor( (split): Splitting() (phi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (psi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (P): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (U): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) ) ) ) ) ) (SCINet_Tree_even): SCINet_Tree( (workingblock): LevelSCINet( (interact): InteractorLevel( (level): Interactor( (split): Splitting() (phi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (psi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (P): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (U): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) ) ) ) (SCINet_Tree_odd): SCINet_Tree( (workingblock): LevelSCINet( (interact): InteractorLevel( (level): Interactor( (split): Splitting() (phi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (psi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (P): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (U): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) ) ) ) ) (SCINet_Tree_even): SCINet_Tree( (workingblock): LevelSCINet( (interact): InteractorLevel( (level): Interactor( (split): Splitting() (phi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (psi): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (P): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) (U): Sequential( (0): ReplicationPad1d((3, 3)) (1): Conv1d(1, 4, kernel_size=(5,), stride=(1,)) (2): LeakyReLU(negative_slope=0.01, inplace=True) (3): Dropout(p=0.5, inplace=False) (4): Conv1d(4, 1, kernel_size=(3,), stride=(1,)) (5): Tanh() ) ) ) ) ) ) ) ) (projection1): Conv1d(96, 48, kernel_size=(1,), stride=(1,), bias=False) (div_projection): ModuleList() )However Result Folder is not being formed containing trues.npy and preds.npy files. Why is the code not executing for the test data ? Is there another separate script? only the model is being saved in ETT_checkpoint Please help, so the results could be plotted.