无法复现论文结果

问题描述

按照 README 中的教程下载数据集，未改动任何超参，同样使用教程中给出的命令运行模型，得到的结果与论文不一致。不是变差，而是比论文要好不少。

数据集：PEMS08，PEMS04

PEMS08

我在不同的两台服务器上运行，得到了基本一致的结果。

服务器1结果：

          MAE  MAPE       RMSE  masked_MAE  masked_MAPE  masked_RMSE
1   11.744327   inf  19.637644   11.760401     0.077948    19.529140
2   11.975752   inf  20.247381   11.992254     0.079423    20.141357
3   12.196908   inf  20.769762   12.214051     0.080845    20.666855
4   12.393086   inf  21.220171   12.410814     0.082166    21.121649
5   12.565434   inf  21.609114   12.583639     0.083352    21.512920
6   12.720485   inf  21.951965   12.739080     0.084445    21.857193
7   12.865274   inf  22.262390   12.884212     0.085499    22.168737
8   13.001018   inf  22.545931   13.020285     0.086475    22.453295
9   13.128123   inf  22.803295   13.147656     0.087415    22.711014
10  13.249768   inf  23.042545   13.269598     0.088333    22.951004
11  13.386254   inf  23.260063   13.406418     0.089319    23.169426
12  13.558510   inf  23.498171   13.579021     0.090486    23.408354

手动计算第一列的非mask的mae 的均值，可以等效得到12步的总体mae。

计算结果为 12.75，比论文中标注的 13.583 要好上不少。其实只看step 12也能发现，最后一步的mae已经小于13.58了，整体算下来肯定是要小很多的。

服务器2结果：

          MAE  MAPE       RMSE  masked_MAE  masked_MAPE  masked_RMSE
1   11.807277   inf  19.672308   11.823156     0.078038    19.558163
2   12.032962   inf  20.286419   12.049309     0.079441    20.176619
3   12.252482   inf  20.817410   12.269487     0.080858    20.713125
4   12.448793   inf  21.282396   12.466464     0.082153    21.183153
5   12.624876   inf  21.685398   12.643200     0.083311    21.590067
6   12.784041   inf  22.039949   12.802795     0.084481    21.947374
7   12.932391   inf  22.358686   12.951554     0.085542    22.268145
8   13.071898   inf  22.650635   13.091407     0.086552    22.561703
9   13.202084   inf  22.917021   13.221920     0.087506    22.829172
10  13.329255   inf  23.166691   13.349422     0.088458    23.079922
11  13.472559   inf  23.394999   13.493119     0.089432    23.309145
12  13.643632   inf  23.640915   13.664682     0.090525    23.555864

同样计算得到总体mae为12.80，和服务器1基本一致。

PEMS04

只测了一次。

服务器1结果：

          MAE  MAPE       RMSE  masked_MAE  masked_MAPE  masked_RMSE
1   16.488237   inf  27.033958   16.616440     0.109217    26.958183
2   16.749134   inf  27.546349   16.873692     0.110845    27.452517
3   16.980698   inf  27.983900   17.102612     0.112179    27.875938
4   17.177589   inf  28.347172   17.297112     0.113359    28.227507
5   17.348188   inf  28.657040   17.466003     0.114348    28.527237
6   17.499729   inf  28.929235   17.615850     0.115272    28.789883
7   17.641754   inf  29.181208   17.756535     0.116133    29.032999
8   17.773027   inf  29.412554   17.886360     0.116933    29.255606
9   17.896318   inf  29.628168   18.008062     0.117671    29.462416
10  18.012926   inf  29.829189   18.122938     0.118415    29.654417
11  18.128349   inf  30.023409   18.236618     0.119193    29.839643
12  18.251507   inf  30.222811   18.357948     0.120025    30.030327

总体mae：17.50，同样比论文中给出的18.321好不少。

额外测试：PEMSBAY

我在libcity官方处下载了PEMSBAY的原子文件，放在PDFormer里也是兼容的，可以直接运行。

服务器1结果：

         MAE  MAPE      RMSE  masked_MAE  masked_MAPE  masked_RMSE
1   0.873613   inf  1.658810    0.869416     0.016877     1.571239
2   1.013528   inf  2.034907    1.009336     0.020192     1.964113
3   1.124029   inf  2.354755    1.119844     0.022965     2.293847
4   1.214753   inf  2.623137    1.210573     0.025361     2.568609
5   1.290269   inf  2.846914    1.286094     0.027425     2.796754
6   1.354344   inf  3.034048    1.350173     0.029210     2.987044
7   1.409635   inf  3.192185    1.405468     0.030763     3.147555
8   1.457894   inf  3.326295    1.453729     0.032124     3.283493
9   1.500738   inf  3.441869    1.496577     0.033332     3.400532
10  1.539042   inf  3.541891    1.534883     0.034409     3.501742
11  1.573877   inf  3.630036    1.569720     0.035390     3.590883
12  1.606230   inf  3.710091    1.602076     0.036290     3.671806

可以看到3 step mae=1.12，6 step mae=1.35，12 step mae=1.60。这个结果已经远超现在的SOTA了。

使用的超参（仿照其他数据集写的，没有刻意调）：

PEMSBAY.json

{
    "dataset_class": "PDFormerDataset",
    "input_window": 12,
    "output_window": 12,
    "train_rate": 0.7,
    "eval_rate": 0.1,
    "batch_size": 16,
    "add_time_in_day": true,
    "add_day_in_week": true,
    "step_size": 2500,
    "max_epoch": 200,
    "bidir": true,
    "far_mask_delta": 7,
    "geo_num_heads": 4,
    "sem_num_heads": 2,
    "t_num_heads": 2,
    "cluster_method": "kshape",
    "cand_key_days": 21,
    "seed": 1,
    "type_ln": "pre",
    "set_loss": "huber",
    "huber_delta": 2,
    "mode": "average"
}

运行日志

PEMS08-服务器1

``` 2023-05-15 20:46:37,450 - INFO - Log directory: ./libcity/log 2023-05-15 20:46:37,451 - INFO - Begin pipeline, task=traffic_state_pred, model_name=PDFormer, dataset_name=PeMS08, exp_id=30930 2023-05-15 20:46:37,451 - INFO - {'task': 'traffic_state_pred', 'model': 'PDFormer', 'dataset': 'PeMS08', 'saved_model': True, 'train': True, 'local_rank': 0, 'gpu_id': [1], 'initial_ckpt': None, 'dataset_class': 'PDFormerDataset', 'input_window': 12, 'output_window': 12, 'train_rate': 0.6, 'eval_rate': 0.2, 'batch_size': 16, 'add_time_in_day': True, 'add_day_in_week': True, 'step_size': 2776, 'max_epoch': 200, 'bidir': True, 'far_mask_delta': 7, 'geo_num_heads': 4, 'sem_num_heads': 2, 't_num_heads': 2, 'cluster_method': 'kshape', 'cand_key_days': 21, 'seed': 1, 'type_ln': 'pre', 'set_loss': 'huber', 'huber_delta': 2, 'mode': 'average', 'executor': 'PDFormerExecutor', 'evaluator': 'TrafficStateEvaluator', 'embed_dim': 64, 'skip_dim': 256, 'mlp_ratio': 4, 'qkv_bias': True, 'drop': 0, 'attn_drop': 0, 'drop_path': 0.3, 's_attn_size': 3, 't_attn_size': 1, 'enc_depth': 6, 'type_short_path': 'hop', 'scaler': 'standard', 'load_external': True, 'normal_external': False, 'ext_scaler': 'none', 'learner': 'adamw', 'learning_rate': 0.001, 'weight_decay': 0.05, 'lr_decay': True, 'lr_scheduler': 'cosinelr', 'lr_eta_min': 0.0001, 'lr_decay_ratio': 0.1, 'lr_warmup_epoch': 5, 'lr_warmup_init': 1e-06, 'clip_grad_norm': True, 'max_grad_norm': 5, 'use_early_stop': True, 'patience': 50, 'task_level': 0, 'use_curriculum_learning': True, 'random_flip': True, 'quan_delta': 0.25, 'dtw_delta': 5, 'cache_dataset': True, 'num_workers': 0, 'pad_with_last_sample': True, 'lape_dim': 8, 'gpu': True, 'train_loss': 'none', 'epoch': 0, 'lr_epsilon': 1e-08, 'lr_beta1': 0.9, 'lr_beta2': 0.999, 'lr_alpha': 0.99, 'lr_momentum': 0, 'steps': [5, 20, 40, 70], 'lr_T_max': 30, 'lr_patience': 10, 'lr_threshold': 0.0001, 'log_level': 'INFO', 'log_every': 1, 'load_best_epoch': True, 'hyper_tune': False, 'grad_accmu_steps': 1, 'metrics': ['MAE', 'MAPE', 'RMSE', 'masked_MAE', 'masked_MAPE', 'masked_RMSE'], 'save_modes': ['csv'], 'geo': {'including_types': ['Point'], 'Point': {}}, 'rel': {'including_types': ['geo'], 'geo': {'cost': 'num'}}, 'dyna': {'including_types': ['state'], 'state': {'entity_id': 'geo_id', 'traffic_flow': 'num', 'traffic_occupancy': 'num', 'traffic_speed': 'num'}}, 'data_col': ['traffic_flow'], 'weight_col': 'cost', 'data_files': ['PeMS08'], 'geo_file': 'PeMS08', 'rel_file': 'PeMS08', 'output_dim': 1, 'time_intervals': 300, 'init_weight_inf_or_zero': 'zero', 'set_weight_link_or_dist': 'link', 'calculate_weight_adj': False, 'weight_adj_epsilon': 0.1, 'distributed': False, 'device': device(type='cuda', index=0), 'exp_id': 30930} 2023-05-15 20:46:37,814 - INFO - Loaded file PeMS08.geo, num_nodes=170 2023-05-15 20:46:37,816 - INFO - set_weight_link_or_dist: link 2023-05-15 20:46:37,816 - INFO - init_weight_inf_or_zero: zero 2023-05-15 20:46:37,819 - INFO - Loaded file PeMS08.rel, shape=(170, 170) 2023-05-15 20:46:37,819 - INFO - Max adj_mx value = 1.0 2023-05-15 20:46:50,510 - INFO - Loading file PeMS08.dyna 2023-05-15 20:46:52,669 - INFO - Loaded file PeMS08.dyna, shape=(17856, 170, 1) 2023-05-15 20:46:52,687 - INFO - Load DTW matrix from ./libcity/cache/dataset_cache/dtw_PeMS08.npy 2023-05-15 20:46:52,688 - INFO - Loading ./libcity/cache/dataset_cache/pdformer_point_based_PeMS08_12_12_0.6_1_0.2_standard_16_True_True_True_True_traffic_flow.npz 2023-05-15 20:47:01,183 - INFO - train x: (10700, 12, 170, 9), y: (10700, 12, 170, 9) 2023-05-15 20:47:01,184 - INFO - eval x: (3566, 12, 170, 9), y: (3566, 12, 170, 9) 2023-05-15 20:47:01,184 - INFO - test x: (3567, 12, 170, 9), y: (3567, 12, 170, 9) 2023-05-15 20:47:01,585 - INFO - StandardScaler mean: 229.8431355598314, std: 145.62553066568907 2023-05-15 20:47:01,585 - INFO - NoneScaler 2023-05-15 20:47:06,007 - INFO - Loaded file ./libcity/cache/dataset_cache/pattern_keys_kshape_PeMS08_21_3_16_5.npy 2023-05-15 20:47:06,022 - INFO - Use use_curriculum_learning! 2023-05-15 20:47:09,723 - INFO - PDFormer( (pattern_embeddings): ModuleList( (0): TokenEmbedding( (token_embed): Linear(in_features=3, out_features=64, bias=True) (norm): Identity() ) ) (enc_embed_layer): DataEmbedding( (value_embedding): TokenEmbedding( (token_embed): Linear(in_features=1, out_features=64, bias=True) (norm): Identity() ) (position_encoding): PositionalEncoding() (daytime_embedding): Embedding(1440, 64) (weekday_embedding): Embedding(7, 64) (spatial_embedding): LaplacianPE( (embedding_lap_pos_enc): Linear(in_features=8, out_features=64, bias=True) ) (dropout): Dropout(p=0, inplace=False) ) (encoder_blocks): ModuleList( (0): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): Identity() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (1): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (2): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (3): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (4): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (5): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) ) (skip_convs): ModuleList( (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (1): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (2): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (4): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (5): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) ) (end_conv1): Conv2d(12, 12, kernel_size=(1, 1), stride=(1, 1)) (end_conv2): Conv2d(256, 1, kernel_size=(1, 1), stride=(1, 1)) ) 2023-05-15 20:47:09,726 - INFO - pattern_embeddings.0.token_embed.weight torch.Size([64, 3]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - pattern_embeddings.0.token_embed.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - enc_embed_layer.value_embedding.token_embed.weight torch.Size([64, 1]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - enc_embed_layer.value_embedding.token_embed.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - enc_embed_layer.daytime_embedding.weight torch.Size([1440, 64]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - enc_embed_layer.weekday_embedding.weight torch.Size([7, 64]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - enc_embed_layer.spatial_embedding.embedding_lap_pos_enc.weight torch.Size([64, 8]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - enc_embed_layer.spatial_embedding.embedding_lap_pos_enc.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - encoder_blocks.0.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - encoder_blocks.0.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - encoder_blocks.0.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - encoder_blocks.0.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - encoder_blocks.0.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,726 - INFO - encoder_blocks.0.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,727 - INFO - encoder_blocks.0.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.0.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.1.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,728 - INFO - encoder_blocks.1.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,729 - INFO - encoder_blocks.1.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 20:47:09,730 - INFO - encoder_blocks.1.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.1.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.1.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,731 - INFO - encoder_blocks.2.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,732 - INFO - encoder_blocks.2.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.2.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.2.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.2.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.2.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.2.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,733 - INFO - encoder_blocks.3.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,734 - INFO - encoder_blocks.3.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.3.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.3.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.3.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.3.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.3.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.3.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.3.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.3.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.3.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.4.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.4.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.4.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.4.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.4.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,735 - INFO - encoder_blocks.4.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,736 - INFO - encoder_blocks.4.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.4.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.5.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,737 - INFO - encoder_blocks.5.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,738 - INFO - encoder_blocks.5.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 20:47:09,739 - INFO - encoder_blocks.5.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - encoder_blocks.5.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - encoder_blocks.5.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.0.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.0.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.1.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.1.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.2.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.2.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.3.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.3.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.4.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.4.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.5.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - skip_convs.5.bias torch.Size([256]) cuda:0 True 2023-05-15 20:47:09,740 - INFO - end_conv1.weight torch.Size([12, 12, 1, 1]) cuda:0 True 2023-05-15 20:47:09,741 - INFO - end_conv1.bias torch.Size([12]) cuda:0 True 2023-05-15 20:47:09,741 - INFO - end_conv2.weight torch.Size([1, 256, 1, 1]) cuda:0 True 2023-05-15 20:47:09,741 - INFO - end_conv2.bias torch.Size([1]) cuda:0 True 2023-05-15 20:47:09,741 - INFO - Total parameter numbers: 531165 2023-05-15 20:47:09,741 - INFO - You select `adamw` optimizer. 2023-05-15 20:47:09,742 - INFO - You select `cosinelr` lr_scheduler. 2023-05-15 20:47:09,742 - WARNING - Received none train loss func and will use the loss func defined in the model. 2023-05-15 20:47:09,744 - INFO - Number of isolated points: 0 2023-05-15 20:47:09,773 - INFO - Start training ... 2023-05-15 20:47:09,773 - INFO - num_batches:669 2023-05-15 20:47:09,826 - INFO - Training: task_level increase from 0 to 1 2023-05-15 20:47:09,826 - INFO - Current batches_seen is 0 2023-05-15 20:49:10,275 - INFO - epoch complete! 2023-05-15 20:49:10,276 - INFO - evaluating now! 2023-05-15 20:49:19,288 - INFO - Epoch [0/200] (669) train_loss: 176.9076, val_loss: 212.7165, lr: 0.000201, 129.51s 2023-05-15 20:49:19,334 - INFO - Saved model at 0 2023-05-15 20:49:19,334 - INFO - Val loss decrease from inf to 212.7165, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch0.tar 2023-05-15 20:51:20,470 - INFO - epoch complete! 2023-05-15 20:51:20,470 - INFO - evaluating now! 2023-05-15 20:51:29,537 - INFO - Epoch [1/200] (1338) train_loss: 46.8155, val_loss: 201.0543, lr: 0.000401, 130.20s 2023-05-15 20:51:29,583 - INFO - Saved model at 1 2023-05-15 20:51:29,583 - INFO - Val loss decrease from 212.7165 to 201.0543, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch1.tar 2023-05-15 20:53:30,703 - INFO - epoch complete! 2023-05-15 20:53:30,704 - INFO - evaluating now! 2023-05-15 20:53:39,776 - INFO - Epoch [2/200] (2007) train_loss: 35.4664, val_loss: 202.2281, lr: 0.000600, 130.19s 2023-05-15 20:55:40,911 - INFO - epoch complete! 2023-05-15 20:55:40,912 - INFO - evaluating now! 2023-05-15 20:55:49,978 - INFO - Epoch [3/200] (2676) train_loss: 30.8637, val_loss: 202.1862, lr: 0.000800, 130.20s 2023-05-15 20:56:08,096 - INFO - Training: task_level increase from 1 to 2 2023-05-15 20:56:08,096 - INFO - Current batches_seen is 2776 2023-05-15 20:57:51,032 - INFO - epoch complete! 2023-05-15 20:57:51,033 - INFO - evaluating now! 2023-05-15 20:58:00,114 - INFO - Epoch [4/200] (3345) train_loss: 33.3046, val_loss: 177.3387, lr: 0.000999, 130.13s 2023-05-15 20:58:00,159 - INFO - Saved model at 4 2023-05-15 20:58:00,159 - INFO - Val loss decrease from 201.0543 to 177.3387, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch4.tar 2023-05-15 21:00:01,300 - INFO - epoch complete! 2023-05-15 21:00:01,300 - INFO - evaluating now! 2023-05-15 21:00:10,372 - INFO - Epoch [5/200] (4014) train_loss: 30.6772, val_loss: 175.9210, lr: 0.000998, 130.21s 2023-05-15 21:00:10,417 - INFO - Saved model at 5 2023-05-15 21:00:10,418 - INFO - Val loss decrease from 177.3387 to 175.9210, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch5.tar 2023-05-15 21:02:11,557 - INFO - epoch complete! 2023-05-15 21:02:11,558 - INFO - evaluating now! 2023-05-15 21:02:20,628 - INFO - Epoch [6/200] (4683) train_loss: 29.2656, val_loss: 175.9500, lr: 0.000997, 130.21s 2023-05-15 21:04:21,745 - INFO - epoch complete! 2023-05-15 21:04:21,745 - INFO - evaluating now! 2023-05-15 21:04:30,809 - INFO - Epoch [7/200] (5352) train_loss: 29.0675, val_loss: 175.9398, lr: 0.000996, 130.18s 2023-05-15 21:05:07,026 - INFO - Training: task_level increase from 2 to 3 2023-05-15 21:05:07,026 - INFO - Current batches_seen is 5552 2023-05-15 21:06:31,914 - INFO - epoch complete! 2023-05-15 21:06:31,915 - INFO - evaluating now! 2023-05-15 21:06:40,975 - INFO - Epoch [8/200] (6021) train_loss: 29.3631, val_loss: 173.6917, lr: 0.000996, 130.17s 2023-05-15 21:06:41,021 - INFO - Saved model at 8 2023-05-15 21:06:41,021 - INFO - Val loss decrease from 175.9210 to 173.6917, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch8.tar 2023-05-15 21:08:42,137 - INFO - epoch complete! 2023-05-15 21:08:42,137 - INFO - evaluating now! 2023-05-15 21:08:51,216 - INFO - Epoch [9/200] (6690) train_loss: 29.4689, val_loss: 174.1300, lr: 0.000994, 130.19s 2023-05-15 21:10:52,318 - INFO - epoch complete! 2023-05-15 21:10:52,319 - INFO - evaluating now! 2023-05-15 21:11:01,392 - INFO - Epoch [10/200] (7359) train_loss: 28.8614, val_loss: 173.6709, lr: 0.000993, 130.18s 2023-05-15 21:11:01,438 - INFO - Saved model at 10 2023-05-15 21:11:01,438 - INFO - Val loss decrease from 173.6917 to 173.6709, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch10.tar 2023-05-15 21:13:02,537 - INFO - epoch complete! 2023-05-15 21:13:02,538 - INFO - evaluating now! 2023-05-15 21:13:11,699 - INFO - Epoch [11/200] (8028) train_loss: 28.6729, val_loss: 173.6092, lr: 0.000992, 130.26s 2023-05-15 21:13:11,745 - INFO - Saved model at 11 2023-05-15 21:13:11,745 - INFO - Val loss decrease from 173.6709 to 173.6092, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch11.tar 2023-05-15 21:14:06,115 - INFO - Training: task_level increase from 3 to 4 2023-05-15 21:14:06,115 - INFO - Current batches_seen is 8328 2023-05-15 21:15:12,844 - INFO - epoch complete! 2023-05-15 21:15:12,844 - INFO - evaluating now! 2023-05-15 21:15:21,980 - INFO - Epoch [12/200] (8697) train_loss: 30.4897, val_loss: 155.4179, lr: 0.000991, 130.23s 2023-05-15 21:15:22,026 - INFO - Saved model at 12 2023-05-15 21:15:22,026 - INFO - Val loss decrease from 173.6092 to 155.4179, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch12.tar 2023-05-15 21:17:23,172 - INFO - epoch complete! 2023-05-15 21:17:23,172 - INFO - evaluating now! 2023-05-15 21:17:32,253 - INFO - Epoch [13/200] (9366) train_loss: 29.1387, val_loss: 155.3557, lr: 0.000989, 130.23s 2023-05-15 21:17:32,299 - INFO - Saved model at 13 2023-05-15 21:17:32,299 - INFO - Val loss decrease from 155.4179 to 155.3557, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch13.tar 2023-05-15 21:19:33,410 - INFO - epoch complete! 2023-05-15 21:19:33,410 - INFO - evaluating now! 2023-05-15 21:19:42,476 - INFO - Epoch [14/200] (10035) train_loss: 28.8320, val_loss: 156.0940, lr: 0.000988, 130.18s 2023-05-15 21:21:43,602 - INFO - epoch complete! 2023-05-15 21:21:43,603 - INFO - evaluating now! 2023-05-15 21:21:52,662 - INFO - Epoch [15/200] (10704) train_loss: 28.6987, val_loss: 155.4764, lr: 0.000986, 130.19s 2023-05-15 21:23:05,108 - INFO - Training: task_level increase from 4 to 5 2023-05-15 21:23:05,109 - INFO - Current batches_seen is 11104 2023-05-15 21:23:53,794 - INFO - epoch complete! 2023-05-15 21:23:53,795 - INFO - evaluating now! 2023-05-15 21:24:02,862 - INFO - Epoch [16/200] (11373) train_loss: 29.5538, val_loss: 139.1658, lr: 0.000984, 130.20s 2023-05-15 21:24:02,907 - INFO - Saved model at 16 2023-05-15 21:24:02,908 - INFO - Val loss decrease from 155.3557 to 139.1658, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch16.tar 2023-05-15 21:26:04,036 - INFO - epoch complete! 2023-05-15 21:26:04,037 - INFO - evaluating now! 2023-05-15 21:26:13,102 - INFO - Epoch [17/200] (12042) train_loss: 29.1979, val_loss: 138.4889, lr: 0.000982, 130.19s 2023-05-15 21:26:13,147 - INFO - Saved model at 17 2023-05-15 21:26:13,147 - INFO - Val loss decrease from 139.1658 to 138.4889, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch17.tar 2023-05-15 21:28:14,274 - INFO - epoch complete! 2023-05-15 21:28:14,275 - INFO - evaluating now! 2023-05-15 21:28:23,355 - INFO - Epoch [18/200] (12711) train_loss: 28.9169, val_loss: 138.8547, lr: 0.000980, 130.21s 2023-05-15 21:30:24,463 - INFO - epoch complete! 2023-05-15 21:30:24,463 - INFO - evaluating now! 2023-05-15 21:30:33,536 - INFO - Epoch [19/200] (13380) train_loss: 28.7751, val_loss: 139.1307, lr: 0.000978, 130.18s 2023-05-15 21:32:04,067 - INFO - Training: task_level increase from 5 to 6 2023-05-15 21:32:04,067 - INFO - Current batches_seen is 13880 2023-05-15 21:32:34,645 - INFO - epoch complete! 2023-05-15 21:32:34,646 - INFO - evaluating now! 2023-05-15 21:32:43,706 - INFO - Epoch [20/200] (14049) train_loss: 30.0842, val_loss: 122.5573, lr: 0.000976, 130.17s 2023-05-15 21:32:43,751 - INFO - Saved model at 20 2023-05-15 21:32:43,751 - INFO - Val loss decrease from 138.4889 to 122.5573, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch20.tar 2023-05-15 21:34:44,877 - INFO - epoch complete! 2023-05-15 21:34:44,878 - INFO - evaluating now! 2023-05-15 21:34:53,944 - INFO - Epoch [21/200] (14718) train_loss: 29.4239, val_loss: 123.4738, lr: 0.000973, 130.19s 2023-05-15 21:36:55,053 - INFO - epoch complete! 2023-05-15 21:36:55,054 - INFO - evaluating now! 2023-05-15 21:37:04,132 - INFO - Epoch [22/200] (15387) train_loss: 29.0365, val_loss: 122.6535, lr: 0.000971, 130.19s 2023-05-15 21:39:05,237 - INFO - epoch complete! 2023-05-15 21:39:05,237 - INFO - evaluating now! 2023-05-15 21:39:14,313 - INFO - Epoch [23/200] (16056) train_loss: 28.8767, val_loss: 123.4840, lr: 0.000968, 130.18s 2023-05-15 21:41:02,919 - INFO - Training: task_level increase from 6 to 7 2023-05-15 21:41:02,919 - INFO - Current batches_seen is 16656 2023-05-15 21:41:15,393 - INFO - epoch complete! 2023-05-15 21:41:15,393 - INFO - evaluating now! 2023-05-15 21:41:24,503 - INFO - Epoch [24/200] (16725) train_loss: 29.4019, val_loss: 107.5789, lr: 0.000966, 130.19s 2023-05-15 21:41:24,550 - INFO - Saved model at 24 2023-05-15 21:41:24,550 - INFO - Val loss decrease from 122.5573 to 107.5789, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch24.tar 2023-05-15 21:43:25,694 - INFO - epoch complete! 2023-05-15 21:43:25,694 - INFO - evaluating now! 2023-05-15 21:43:34,808 - INFO - Epoch [25/200] (17394) train_loss: 29.2781, val_loss: 106.7459, lr: 0.000963, 130.26s 2023-05-15 21:43:34,854 - INFO - Saved model at 25 2023-05-15 21:43:34,854 - INFO - Val loss decrease from 107.5789 to 106.7459, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch25.tar 2023-05-15 21:45:35,973 - INFO - epoch complete! 2023-05-15 21:45:35,974 - INFO - evaluating now! 2023-05-15 21:45:45,061 - INFO - Epoch [26/200] (18063) train_loss: 29.1090, val_loss: 106.8568, lr: 0.000960, 130.21s 2023-05-15 21:47:46,159 - INFO - epoch complete! 2023-05-15 21:47:46,159 - INFO - evaluating now! 2023-05-15 21:47:55,238 - INFO - Epoch [27/200] (18732) train_loss: 29.0311, val_loss: 106.7355, lr: 0.000957, 130.18s 2023-05-15 21:47:55,284 - INFO - Saved model at 27 2023-05-15 21:47:55,284 - INFO - Val loss decrease from 106.7459 to 106.7355, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch27.tar 2023-05-15 21:49:56,390 - INFO - epoch complete! 2023-05-15 21:49:56,390 - INFO - evaluating now! 2023-05-15 21:50:05,451 - INFO - Epoch [28/200] (19401) train_loss: 28.9129, val_loss: 107.1959, lr: 0.000954, 130.17s 2023-05-15 21:50:11,082 - INFO - Training: task_level increase from 7 to 8 2023-05-15 21:50:11,082 - INFO - Current batches_seen is 19432 2023-05-15 21:52:06,574 - INFO - epoch complete! 2023-05-15 21:52:06,575 - INFO - evaluating now! 2023-05-15 21:52:15,655 - INFO - Epoch [29/200] (20070) train_loss: 29.8017, val_loss: 90.2712, lr: 0.000951, 130.20s 2023-05-15 21:52:15,700 - INFO - Saved model at 29 2023-05-15 21:52:15,701 - INFO - Val loss decrease from 106.7355 to 90.2712, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch29.tar 2023-05-15 21:54:16,798 - INFO - epoch complete! 2023-05-15 21:54:16,798 - INFO - evaluating now! 2023-05-15 21:54:25,879 - INFO - Epoch [30/200] (20739) train_loss: 29.2793, val_loss: 90.8969, lr: 0.000948, 130.18s 2023-05-15 21:56:26,987 - INFO - epoch complete! 2023-05-15 21:56:26,987 - INFO - evaluating now! 2023-05-15 21:56:36,094 - INFO - Epoch [31/200] (21408) train_loss: 29.1008, val_loss: 91.1215, lr: 0.000944, 130.22s 2023-05-15 21:58:37,231 - INFO - epoch complete! 2023-05-15 21:58:37,231 - INFO - evaluating now! 2023-05-15 21:58:46,348 - INFO - Epoch [32/200] (22077) train_loss: 29.0441, val_loss: 90.3488, lr: 0.000941, 130.25s 2023-05-15 21:59:10,079 - INFO - Training: task_level increase from 8 to 9 2023-05-15 21:59:10,079 - INFO - Current batches_seen is 22208 2023-05-15 22:00:49,483 - INFO - epoch complete! 2023-05-15 22:00:49,483 - INFO - evaluating now! 2023-05-15 22:00:58,594 - INFO - Epoch [33/200] (22746) train_loss: 29.6430, val_loss: 82.9265, lr: 0.000937, 132.25s 2023-05-15 22:00:58,639 - INFO - Saved model at 33 2023-05-15 22:00:58,640 - INFO - Val loss decrease from 90.2712 to 82.9265, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch33.tar 2023-05-15 22:02:59,751 - INFO - epoch complete! 2023-05-15 22:02:59,751 - INFO - evaluating now! 2023-05-15 22:03:08,814 - INFO - Epoch [34/200] (23415) train_loss: 29.3267, val_loss: 82.4248, lr: 0.000934, 130.17s 2023-05-15 22:03:08,860 - INFO - Saved model at 34 2023-05-15 22:03:08,860 - INFO - Val loss decrease from 82.9265 to 82.4248, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch34.tar 2023-05-15 22:05:10,062 - INFO - epoch complete! 2023-05-15 22:05:10,063 - INFO - evaluating now! 2023-05-15 22:05:19,125 - INFO - Epoch [35/200] (24084) train_loss: 29.1887, val_loss: 82.6262, lr: 0.000930, 130.26s 2023-05-15 22:07:20,434 - INFO - epoch complete! 2023-05-15 22:07:20,435 - INFO - evaluating now! 2023-05-15 22:07:29,517 - INFO - Epoch [36/200] (24753) train_loss: 28.9771, val_loss: 82.2882, lr: 0.000926, 130.39s 2023-05-15 22:07:29,566 - INFO - Saved model at 36 2023-05-15 22:07:29,566 - INFO - Val loss decrease from 82.4248 to 82.2882, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch36.tar 2023-05-15 22:08:11,382 - INFO - Training: task_level increase from 9 to 10 2023-05-15 22:08:11,382 - INFO - Current batches_seen is 24984 2023-05-15 22:09:30,656 - INFO - epoch complete! 2023-05-15 22:09:30,657 - INFO - evaluating now! 2023-05-15 22:09:39,742 - INFO - Epoch [37/200] (25422) train_loss: 29.6872, val_loss: 65.3815, lr: 0.000922, 130.18s 2023-05-15 22:09:39,788 - INFO - Saved model at 37 2023-05-15 22:09:39,788 - INFO - Val loss decrease from 82.2882 to 65.3815, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch37.tar 2023-05-15 22:11:41,783 - INFO - epoch complete! 2023-05-15 22:11:41,783 - INFO - evaluating now! 2023-05-15 22:11:50,863 - INFO - Epoch [38/200] (26091) train_loss: 29.4082, val_loss: 65.2779, lr: 0.000918, 131.07s 2023-05-15 22:11:50,909 - INFO - Saved model at 38 2023-05-15 22:11:50,909 - INFO - Val loss decrease from 65.3815 to 65.2779, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch38.tar 2023-05-15 22:13:52,453 - INFO - epoch complete! 2023-05-15 22:13:52,454 - INFO - evaluating now! 2023-05-15 22:14:01,532 - INFO - Epoch [39/200] (26760) train_loss: 29.3870, val_loss: 65.3193, lr: 0.000914, 130.62s 2023-05-15 22:16:02,648 - INFO - epoch complete! 2023-05-15 22:16:02,648 - INFO - evaluating now! 2023-05-15 22:16:11,723 - INFO - Epoch [40/200] (27429) train_loss: 29.1519, val_loss: 65.3760, lr: 0.000910, 130.19s 2023-05-15 22:17:11,669 - INFO - Training: task_level increase from 10 to 11 2023-05-15 22:17:11,669 - INFO - Current batches_seen is 27760 2023-05-15 22:18:12,857 - INFO - epoch complete! 2023-05-15 22:18:12,857 - INFO - evaluating now! 2023-05-15 22:18:21,932 - INFO - Epoch [41/200] (28098) train_loss: 29.8674, val_loss: 47.3732, lr: 0.000906, 130.21s 2023-05-15 22:18:21,978 - INFO - Saved model at 41 2023-05-15 22:18:21,979 - INFO - Val loss decrease from 65.2779 to 47.3732, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch41.tar 2023-05-15 22:20:23,073 - INFO - epoch complete! 2023-05-15 22:20:23,074 - INFO - evaluating now! 2023-05-15 22:20:32,171 - INFO - Epoch [42/200] (28767) train_loss: 29.6460, val_loss: 47.8657, lr: 0.000901, 130.19s 2023-05-15 22:22:33,300 - INFO - epoch complete! 2023-05-15 22:22:33,301 - INFO - evaluating now! 2023-05-15 22:22:42,381 - INFO - Epoch [43/200] (29436) train_loss: 29.4414, val_loss: 46.7937, lr: 0.000897, 130.21s 2023-05-15 22:22:42,427 - INFO - Saved model at 43 2023-05-15 22:22:42,427 - INFO - Val loss decrease from 47.3732 to 46.7937, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch43.tar 2023-05-15 22:24:43,560 - INFO - epoch complete! 2023-05-15 22:24:43,561 - INFO - evaluating now! 2023-05-15 22:24:52,639 - INFO - Epoch [44/200] (30105) train_loss: 29.2308, val_loss: 46.5489, lr: 0.000892, 130.21s 2023-05-15 22:24:52,685 - INFO - Saved model at 44 2023-05-15 22:24:52,685 - INFO - Val loss decrease from 46.7937 to 46.5489, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch44.tar 2023-05-15 22:26:10,734 - INFO - Training: task_level increase from 11 to 12 2023-05-15 22:26:10,734 - INFO - Current batches_seen is 30536 2023-05-15 22:26:53,801 - INFO - epoch complete! 2023-05-15 22:26:53,802 - INFO - evaluating now! 2023-05-15 22:27:02,880 - INFO - Epoch [45/200] (30774) train_loss: 29.6752, val_loss: 31.7393, lr: 0.000888, 130.20s 2023-05-15 22:27:02,926 - INFO - Saved model at 45 2023-05-15 22:27:02,927 - INFO - Val loss decrease from 46.5489 to 31.7393, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch45.tar 2023-05-15 22:29:04,034 - INFO - epoch complete! 2023-05-15 22:29:04,035 - INFO - evaluating now! 2023-05-15 22:29:13,117 - INFO - Epoch [46/200] (31443) train_loss: 29.5424, val_loss: 30.1564, lr: 0.000883, 130.19s 2023-05-15 22:29:13,163 - INFO - Saved model at 46 2023-05-15 22:29:13,163 - INFO - Val loss decrease from 31.7393 to 30.1564, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch46.tar 2023-05-15 22:31:14,254 - INFO - epoch complete! 2023-05-15 22:31:14,255 - INFO - evaluating now! 2023-05-15 22:31:23,336 - INFO - Epoch [47/200] (32112) train_loss: 29.4210, val_loss: 29.2227, lr: 0.000878, 130.17s 2023-05-15 22:31:23,382 - INFO - Saved model at 47 2023-05-15 22:31:23,383 - INFO - Val loss decrease from 30.1564 to 29.2227, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch47.tar 2023-05-15 22:33:24,499 - INFO - epoch complete! 2023-05-15 22:33:24,500 - INFO - evaluating now! 2023-05-15 22:33:33,584 - INFO - Epoch [48/200] (32781) train_loss: 29.3383, val_loss: 29.8412, lr: 0.000873, 130.20s 2023-05-15 22:35:35,264 - INFO - epoch complete! 2023-05-15 22:35:35,265 - INFO - evaluating now! 2023-05-15 22:35:44,388 - INFO - Epoch [49/200] (33450) train_loss: 29.2307, val_loss: 28.7010, lr: 0.000868, 130.80s 2023-05-15 22:35:44,449 - INFO - Saved model at 49 2023-05-15 22:35:44,450 - INFO - Val loss decrease from 29.2227 to 28.7010, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch49.tar 2023-05-15 22:37:46,411 - INFO - epoch complete! 2023-05-15 22:37:46,447 - INFO - evaluating now! 2023-05-15 22:37:55,596 - INFO - Epoch [50/200] (34119) train_loss: 29.0120, val_loss: 29.0933, lr: 0.000863, 131.13s 2023-05-15 22:39:56,902 - INFO - epoch complete! 2023-05-15 22:39:56,902 - INFO - evaluating now! 2023-05-15 22:40:05,996 - INFO - Epoch [51/200] (34788) train_loss: 29.0251, val_loss: 28.5246, lr: 0.000858, 130.40s 2023-05-15 22:40:06,377 - INFO - Saved model at 51 2023-05-15 22:40:06,378 - INFO - Val loss decrease from 28.7010 to 28.5246, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch51.tar 2023-05-15 22:42:07,505 - INFO - epoch complete! 2023-05-15 22:42:07,506 - INFO - evaluating now! 2023-05-15 22:42:16,580 - INFO - Epoch [52/200] (35457) train_loss: 28.8020, val_loss: 28.4806, lr: 0.000853, 130.20s 2023-05-15 22:42:16,626 - INFO - Saved model at 52 2023-05-15 22:42:16,626 - INFO - Val loss decrease from 28.5246 to 28.4806, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch52.tar 2023-05-15 22:44:17,741 - INFO - epoch complete! 2023-05-15 22:44:17,742 - INFO - evaluating now! 2023-05-15 22:44:26,808 - INFO - Epoch [53/200] (36126) train_loss: 28.7646, val_loss: 28.6212, lr: 0.000848, 130.18s 2023-05-15 22:46:28,564 - INFO - epoch complete! 2023-05-15 22:46:28,564 - INFO - evaluating now! 2023-05-15 22:46:37,633 - INFO - Epoch [54/200] (36795) train_loss: 28.6398, val_loss: 28.0059, lr: 0.000842, 130.82s 2023-05-15 22:46:37,679 - INFO - Saved model at 54 2023-05-15 22:46:37,680 - INFO - Val loss decrease from 28.4806 to 28.0059, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch54.tar 2023-05-15 22:48:42,264 - INFO - epoch complete! 2023-05-15 22:48:42,265 - INFO - evaluating now! 2023-05-15 22:48:51,394 - INFO - Epoch [55/200] (37464) train_loss: 28.5958, val_loss: 28.5361, lr: 0.000837, 133.71s 2023-05-15 22:50:52,512 - INFO - epoch complete! 2023-05-15 22:50:52,512 - INFO - evaluating now! 2023-05-15 22:51:01,619 - INFO - Epoch [56/200] (38133) train_loss: 28.4486, val_loss: 28.3511, lr: 0.000831, 130.22s 2023-05-15 22:53:03,598 - INFO - epoch complete! 2023-05-15 22:53:03,599 - INFO - evaluating now! 2023-05-15 22:53:12,689 - INFO - Epoch [57/200] (38802) train_loss: 28.4085, val_loss: 28.4636, lr: 0.000826, 131.07s 2023-05-15 22:55:15,140 - INFO - epoch complete! 2023-05-15 22:55:15,140 - INFO - evaluating now! 2023-05-15 22:55:24,246 - INFO - Epoch [58/200] (39471) train_loss: 28.3508, val_loss: 28.7907, lr: 0.000820, 131.56s 2023-05-15 22:57:26,159 - INFO - epoch complete! 2023-05-15 22:57:26,160 - INFO - evaluating now! 2023-05-15 22:57:35,249 - INFO - Epoch [59/200] (40140) train_loss: 28.1592, val_loss: 29.0723, lr: 0.000815, 131.00s 2023-05-15 22:59:35,705 - INFO - epoch complete! 2023-05-15 22:59:35,706 - INFO - evaluating now! 2023-05-15 22:59:44,812 - INFO - Epoch [60/200] (40809) train_loss: 28.1297, val_loss: 28.0462, lr: 0.000809, 129.56s 2023-05-15 23:01:46,039 - INFO - epoch complete! 2023-05-15 23:01:46,040 - INFO - evaluating now! 2023-05-15 23:01:55,142 - INFO - Epoch [61/200] (41478) train_loss: 27.9692, val_loss: 27.7356, lr: 0.000803, 130.33s 2023-05-15 23:01:55,198 - INFO - Saved model at 61 2023-05-15 23:01:55,198 - INFO - Val loss decrease from 28.0059 to 27.7356, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch61.tar 2023-05-15 23:03:56,619 - INFO - epoch complete! 2023-05-15 23:03:56,620 - INFO - evaluating now! 2023-05-15 23:04:05,699 - INFO - Epoch [62/200] (42147) train_loss: 27.9655, val_loss: 27.8048, lr: 0.000797, 130.50s 2023-05-15 23:06:06,822 - INFO - epoch complete! 2023-05-15 23:06:06,823 - INFO - evaluating now! 2023-05-15 23:06:15,886 - INFO - Epoch [63/200] (42816) train_loss: 27.8996, val_loss: 27.6909, lr: 0.000791, 130.19s 2023-05-15 23:06:15,933 - INFO - Saved model at 63 2023-05-15 23:06:15,933 - INFO - Val loss decrease from 27.7356 to 27.6909, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch63.tar 2023-05-15 23:08:17,050 - INFO - epoch complete! 2023-05-15 23:08:17,051 - INFO - evaluating now! 2023-05-15 23:08:26,121 - INFO - Epoch [64/200] (43485) train_loss: 27.7896, val_loss: 28.5716, lr: 0.000785, 130.19s 2023-05-15 23:10:27,580 - INFO - epoch complete! 2023-05-15 23:10:27,580 - INFO - evaluating now! 2023-05-15 23:10:36,653 - INFO - Epoch [65/200] (44154) train_loss: 27.7386, val_loss: 27.3878, lr: 0.000779, 130.53s 2023-05-15 23:10:36,699 - INFO - Saved model at 65 2023-05-15 23:10:36,700 - INFO - Val loss decrease from 27.6909 to 27.3878, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch65.tar 2023-05-15 23:12:38,966 - INFO - epoch complete! 2023-05-15 23:12:38,967 - INFO - evaluating now! 2023-05-15 23:12:48,040 - INFO - Epoch [66/200] (44823) train_loss: 27.5707, val_loss: 27.8695, lr: 0.000773, 131.34s 2023-05-15 23:14:49,162 - INFO - epoch complete! 2023-05-15 23:14:49,163 - INFO - evaluating now! 2023-05-15 23:14:58,269 - INFO - Epoch [67/200] (45492) train_loss: 27.6668, val_loss: 27.2899, lr: 0.000767, 130.23s 2023-05-15 23:14:58,325 - INFO - Saved model at 67 2023-05-15 23:14:58,326 - INFO - Val loss decrease from 27.3878 to 27.2899, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch67.tar 2023-05-15 23:17:00,155 - INFO - epoch complete! 2023-05-15 23:17:00,155 - INFO - evaluating now! 2023-05-15 23:17:09,235 - INFO - Epoch [68/200] (46161) train_loss: 27.4328, val_loss: 27.9190, lr: 0.000761, 130.91s 2023-05-15 23:19:10,383 - INFO - epoch complete! 2023-05-15 23:19:10,383 - INFO - evaluating now! 2023-05-15 23:19:19,426 - INFO - Epoch [69/200] (46830) train_loss: 27.4322, val_loss: 28.1491, lr: 0.000754, 130.19s 2023-05-15 23:21:20,552 - INFO - epoch complete! 2023-05-15 23:21:20,553 - INFO - evaluating now! 2023-05-15 23:21:29,622 - INFO - Epoch [70/200] (47499) train_loss: 27.3730, val_loss: 27.1894, lr: 0.000748, 130.20s 2023-05-15 23:21:29,668 - INFO - Saved model at 70 2023-05-15 23:21:29,668 - INFO - Val loss decrease from 27.2899 to 27.1894, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch70.tar 2023-05-15 23:23:31,589 - INFO - epoch complete! 2023-05-15 23:23:31,590 - INFO - evaluating now! 2023-05-15 23:23:40,678 - INFO - Epoch [71/200] (48168) train_loss: 27.3157, val_loss: 27.1088, lr: 0.000742, 131.01s 2023-05-15 23:23:40,725 - INFO - Saved model at 71 2023-05-15 23:23:40,726 - INFO - Val loss decrease from 27.1894 to 27.1088, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch71.tar 2023-05-15 23:25:41,845 - INFO - epoch complete! 2023-05-15 23:25:41,846 - INFO - evaluating now! 2023-05-15 23:25:50,968 - INFO - Epoch [72/200] (48837) train_loss: 27.2179, val_loss: 27.2593, lr: 0.000735, 130.24s 2023-05-15 23:27:52,008 - INFO - epoch complete! 2023-05-15 23:27:52,008 - INFO - evaluating now! 2023-05-15 23:28:01,105 - INFO - Epoch [73/200] (49506) train_loss: 27.1703, val_loss: 28.1718, lr: 0.000729, 130.14s 2023-05-15 23:30:02,674 - INFO - epoch complete! 2023-05-15 23:30:02,675 - INFO - evaluating now! 2023-05-15 23:30:11,724 - INFO - Epoch [74/200] (50175) train_loss: 27.1096, val_loss: 26.9710, lr: 0.000722, 130.62s 2023-05-15 23:30:11,769 - INFO - Saved model at 74 2023-05-15 23:30:11,770 - INFO - Val loss decrease from 27.1088 to 26.9710, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch74.tar 2023-05-15 23:32:12,897 - INFO - epoch complete! 2023-05-15 23:32:12,898 - INFO - evaluating now! 2023-05-15 23:32:21,962 - INFO - Epoch [75/200] (50844) train_loss: 26.9198, val_loss: 27.5913, lr: 0.000716, 130.19s 2023-05-15 23:34:23,062 - INFO - epoch complete! 2023-05-15 23:34:23,063 - INFO - evaluating now! 2023-05-15 23:34:32,176 - INFO - Epoch [76/200] (51513) train_loss: 26.9671, val_loss: 27.0215, lr: 0.000709, 130.21s 2023-05-15 23:36:33,308 - INFO - epoch complete! 2023-05-15 23:36:33,309 - INFO - evaluating now! 2023-05-15 23:36:42,394 - INFO - Epoch [77/200] (52182) train_loss: 26.9472, val_loss: 27.5327, lr: 0.000702, 130.22s 2023-05-15 23:38:43,500 - INFO - epoch complete! 2023-05-15 23:38:43,501 - INFO - evaluating now! 2023-05-15 23:38:52,613 - INFO - Epoch [78/200] (52851) train_loss: 26.8156, val_loss: 27.4629, lr: 0.000696, 130.22s 2023-05-15 23:40:53,779 - INFO - epoch complete! 2023-05-15 23:40:53,780 - INFO - evaluating now! 2023-05-15 23:41:02,907 - INFO - Epoch [79/200] (53520) train_loss: 26.7078, val_loss: 27.2953, lr: 0.000689, 130.29s 2023-05-15 23:43:04,355 - INFO - epoch complete! 2023-05-15 23:43:04,356 - INFO - evaluating now! 2023-05-15 23:43:13,488 - INFO - Epoch [80/200] (54189) train_loss: 26.7445, val_loss: 27.1459, lr: 0.000682, 130.58s 2023-05-15 23:45:14,615 - INFO - epoch complete! 2023-05-15 23:45:14,616 - INFO - evaluating now! 2023-05-15 23:45:23,697 - INFO - Epoch [81/200] (54858) train_loss: 26.6419, val_loss: 27.3671, lr: 0.000676, 130.21s 2023-05-15 23:47:25,048 - INFO - epoch complete! 2023-05-15 23:47:25,048 - INFO - evaluating now! 2023-05-15 23:47:34,127 - INFO - Epoch [82/200] (55527) train_loss: 26.6061, val_loss: 26.8214, lr: 0.000669, 130.43s 2023-05-15 23:47:34,174 - INFO - Saved model at 82 2023-05-15 23:47:34,174 - INFO - Val loss decrease from 26.9710 to 26.8214, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch82.tar 2023-05-15 23:49:35,292 - INFO - epoch complete! 2023-05-15 23:49:35,293 - INFO - evaluating now! 2023-05-15 23:49:44,342 - INFO - Epoch [83/200] (56196) train_loss: 26.6098, val_loss: 26.9367, lr: 0.000662, 130.17s 2023-05-15 23:51:45,448 - INFO - epoch complete! 2023-05-15 23:51:45,449 - INFO - evaluating now! 2023-05-15 23:51:54,496 - INFO - Epoch [84/200] (56865) train_loss: 26.4782, val_loss: 27.1145, lr: 0.000655, 130.15s 2023-05-15 23:53:55,622 - INFO - epoch complete! 2023-05-15 23:53:55,622 - INFO - evaluating now! 2023-05-15 23:54:04,703 - INFO - Epoch [85/200] (57534) train_loss: 26.4697, val_loss: 26.5064, lr: 0.000648, 130.21s 2023-05-15 23:54:04,749 - INFO - Saved model at 85 2023-05-15 23:54:04,749 - INFO - Val loss decrease from 26.8214 to 26.5064, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch85.tar 2023-05-15 23:56:05,882 - INFO - epoch complete! 2023-05-15 23:56:05,882 - INFO - evaluating now! 2023-05-15 23:56:14,950 - INFO - Epoch [86/200] (58203) train_loss: 26.4171, val_loss: 26.6283, lr: 0.000641, 130.20s 2023-05-15 23:58:16,137 - INFO - epoch complete! 2023-05-15 23:58:16,138 - INFO - evaluating now! 2023-05-15 23:58:25,216 - INFO - Epoch [87/200] (58872) train_loss: 26.4320, val_loss: 26.6800, lr: 0.000634, 130.27s 2023-05-16 00:00:26,719 - INFO - epoch complete! 2023-05-16 00:00:26,720 - INFO - evaluating now! 2023-05-16 00:00:35,826 - INFO - Epoch [88/200] (59541) train_loss: 26.3192, val_loss: 26.4698, lr: 0.000627, 130.61s 2023-05-16 00:00:35,873 - INFO - Saved model at 88 2023-05-16 00:00:35,874 - INFO - Val loss decrease from 26.5064 to 26.4698, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch88.tar 2023-05-16 00:02:37,012 - INFO - epoch complete! 2023-05-16 00:02:37,012 - INFO - evaluating now! 2023-05-16 00:02:46,090 - INFO - Epoch [89/200] (60210) train_loss: 26.2571, val_loss: 26.7087, lr: 0.000620, 130.22s 2023-05-16 00:04:47,424 - INFO - epoch complete! 2023-05-16 00:04:47,425 - INFO - evaluating now! 2023-05-16 00:04:56,490 - INFO - Epoch [90/200] (60879) train_loss: 26.1384, val_loss: 26.5153, lr: 0.000613, 130.40s 2023-05-16 00:06:57,821 - INFO - epoch complete! 2023-05-16 00:06:57,821 - INFO - evaluating now! 2023-05-16 00:07:06,892 - INFO - Epoch [91/200] (61548) train_loss: 26.1154, val_loss: 26.7831, lr: 0.000606, 130.40s 2023-05-16 00:09:08,024 - INFO - epoch complete! 2023-05-16 00:09:08,024 - INFO - evaluating now! 2023-05-16 00:09:17,097 - INFO - Epoch [92/200] (62217) train_loss: 26.0692, val_loss: 26.2425, lr: 0.000599, 130.20s 2023-05-16 00:09:17,143 - INFO - Saved model at 92 2023-05-16 00:09:17,143 - INFO - Val loss decrease from 26.4698 to 26.2425, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch92.tar 2023-05-16 00:11:18,274 - INFO - epoch complete! 2023-05-16 00:11:18,275 - INFO - evaluating now! 2023-05-16 00:11:27,340 - INFO - Epoch [93/200] (62886) train_loss: 26.0472, val_loss: 27.1175, lr: 0.000592, 130.20s 2023-05-16 00:13:28,721 - INFO - epoch complete! 2023-05-16 00:13:28,722 - INFO - evaluating now! 2023-05-16 00:13:37,791 - INFO - Epoch [94/200] (63555) train_loss: 26.0880, val_loss: 26.9319, lr: 0.000585, 130.45s 2023-05-16 00:15:38,918 - INFO - epoch complete! 2023-05-16 00:15:38,919 - INFO - evaluating now! 2023-05-16 00:15:47,994 - INFO - Epoch [95/200] (64224) train_loss: 25.9579, val_loss: 26.7201, lr: 0.000578, 130.20s 2023-05-16 00:17:49,290 - INFO - epoch complete! 2023-05-16 00:17:49,291 - INFO - evaluating now! 2023-05-16 00:17:58,367 - INFO - Epoch [96/200] (64893) train_loss: 25.8998, val_loss: 26.6308, lr: 0.000571, 130.37s 2023-05-16 00:19:59,490 - INFO - epoch complete! 2023-05-16 00:19:59,490 - INFO - evaluating now! 2023-05-16 00:20:08,544 - INFO - Epoch [97/200] (65562) train_loss: 25.8306, val_loss: 26.1762, lr: 0.000564, 130.18s 2023-05-16 00:20:08,590 - INFO - Saved model at 97 2023-05-16 00:20:08,591 - INFO - Val loss decrease from 26.2425 to 26.1762, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch97.tar 2023-05-16 00:22:09,728 - INFO - epoch complete! 2023-05-16 00:22:09,728 - INFO - evaluating now! 2023-05-16 00:22:18,776 - INFO - Epoch [98/200] (66231) train_loss: 25.8755, val_loss: 26.5263, lr: 0.000557, 130.19s 2023-05-16 00:24:19,885 - INFO - epoch complete! 2023-05-16 00:24:19,886 - INFO - evaluating now! 2023-05-16 00:24:28,932 - INFO - Epoch [99/200] (66900) train_loss: 25.7826, val_loss: 26.4622, lr: 0.000550, 130.16s 2023-05-16 00:26:30,070 - INFO - epoch complete! 2023-05-16 00:26:30,070 - INFO - evaluating now! 2023-05-16 00:26:39,103 - INFO - Epoch [100/200] (67569) train_loss: 25.7495, val_loss: 26.6455, lr: 0.000543, 130.17s 2023-05-16 00:28:41,081 - INFO - epoch complete! 2023-05-16 00:28:41,082 - INFO - evaluating now! 2023-05-16 00:28:50,186 - INFO - Epoch [101/200] (68238) train_loss: 25.6308, val_loss: 26.1583, lr: 0.000536, 131.08s 2023-05-16 00:28:50,233 - INFO - Saved model at 101 2023-05-16 00:28:50,233 - INFO - Val loss decrease from 26.1762 to 26.1583, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch101.tar 2023-05-16 00:30:51,349 - INFO - epoch complete! 2023-05-16 00:30:51,350 - INFO - evaluating now! 2023-05-16 00:31:00,441 - INFO - Epoch [102/200] (68907) train_loss: 25.6036, val_loss: 26.3403, lr: 0.000529, 130.21s 2023-05-16 00:33:01,558 - INFO - epoch complete! 2023-05-16 00:33:01,559 - INFO - evaluating now! 2023-05-16 00:33:10,619 - INFO - Epoch [103/200] (69576) train_loss: 25.5875, val_loss: 26.5419, lr: 0.000522, 130.18s 2023-05-16 00:35:11,738 - INFO - epoch complete! 2023-05-16 00:35:11,739 - INFO - evaluating now! 2023-05-16 00:35:20,789 - INFO - Epoch [104/200] (70245) train_loss: 25.6197, val_loss: 26.2249, lr: 0.000515, 130.17s 2023-05-16 00:37:21,898 - INFO - epoch complete! 2023-05-16 00:37:21,899 - INFO - evaluating now! 2023-05-16 00:37:30,950 - INFO - Epoch [105/200] (70914) train_loss: 25.5001, val_loss: 26.0971, lr: 0.000508, 130.16s 2023-05-16 00:37:30,996 - INFO - Saved model at 105 2023-05-16 00:37:30,996 - INFO - Val loss decrease from 26.1583 to 26.0971, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch105.tar 2023-05-16 00:39:32,173 - INFO - epoch complete! 2023-05-16 00:39:32,174 - INFO - evaluating now! 2023-05-16 00:39:41,290 - INFO - Epoch [106/200] (71583) train_loss: 25.4093, val_loss: 26.3680, lr: 0.000501, 130.29s 2023-05-16 00:41:42,619 - INFO - epoch complete! 2023-05-16 00:41:42,619 - INFO - evaluating now! 2023-05-16 00:41:51,713 - INFO - Epoch [107/200] (72252) train_loss: 25.3761, val_loss: 26.6766, lr: 0.000494, 130.42s 2023-05-16 00:43:52,951 - INFO - epoch complete! 2023-05-16 00:43:52,952 - INFO - evaluating now! 2023-05-16 00:44:02,025 - INFO - Epoch [108/200] (72921) train_loss: 25.3819, val_loss: 26.1781, lr: 0.000487, 130.31s 2023-05-16 00:46:03,768 - INFO - epoch complete! 2023-05-16 00:46:03,769 - INFO - evaluating now! 2023-05-16 00:46:12,860 - INFO - Epoch [109/200] (73590) train_loss: 25.3817, val_loss: 26.3514, lr: 0.000480, 130.83s 2023-05-16 00:48:13,972 - INFO - epoch complete! 2023-05-16 00:48:13,973 - INFO - evaluating now! 2023-05-16 00:48:23,068 - INFO - Epoch [110/200] (74259) train_loss: 25.3133, val_loss: 26.4645, lr: 0.000473, 130.21s 2023-05-16 00:50:25,855 - INFO - epoch complete! 2023-05-16 00:50:25,856 - INFO - evaluating now! 2023-05-16 00:50:35,065 - INFO - Epoch [111/200] (74928) train_loss: 25.3340, val_loss: 26.0298, lr: 0.000466, 132.00s 2023-05-16 00:50:35,112 - INFO - Saved model at 111 2023-05-16 00:50:35,112 - INFO - Val loss decrease from 26.0971 to 26.0298, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch111.tar 2023-05-16 00:52:36,410 - INFO - epoch complete! 2023-05-16 00:52:36,411 - INFO - evaluating now! 2023-05-16 00:52:45,511 - INFO - Epoch [112/200] (75597) train_loss: 25.2834, val_loss: 26.2654, lr: 0.000459, 130.40s 2023-05-16 00:54:46,628 - INFO - epoch complete! 2023-05-16 00:54:46,629 - INFO - evaluating now! 2023-05-16 00:54:55,724 - INFO - Epoch [113/200] (76266) train_loss: 25.2620, val_loss: 26.2033, lr: 0.000452, 130.21s 2023-05-16 00:56:56,909 - INFO - epoch complete! 2023-05-16 00:56:56,910 - INFO - evaluating now! 2023-05-16 00:57:06,003 - INFO - Epoch [114/200] (76935) train_loss: 25.1916, val_loss: 26.2536, lr: 0.000445, 130.28s 2023-05-16 00:59:07,121 - INFO - epoch complete! 2023-05-16 00:59:07,121 - INFO - evaluating now! 2023-05-16 00:59:16,229 - INFO - Epoch [115/200] (77604) train_loss: 25.1665, val_loss: 26.0291, lr: 0.000438, 130.23s 2023-05-16 00:59:16,275 - INFO - Saved model at 115 2023-05-16 00:59:16,276 - INFO - Val loss decrease from 26.0298 to 26.0291, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch115.tar 2023-05-16 01:01:17,406 - INFO - epoch complete! 2023-05-16 01:01:17,406 - INFO - evaluating now! 2023-05-16 01:01:26,479 - INFO - Epoch [116/200] (78273) train_loss: 25.1886, val_loss: 26.3292, lr: 0.000431, 130.20s 2023-05-16 01:03:27,604 - INFO - epoch complete! 2023-05-16 01:03:27,604 - INFO - evaluating now! 2023-05-16 01:03:36,661 - INFO - Epoch [117/200] (78942) train_loss: 25.1603, val_loss: 26.1293, lr: 0.000424, 130.18s 2023-05-16 01:05:37,758 - INFO - epoch complete! 2023-05-16 01:05:37,759 - INFO - evaluating now! 2023-05-16 01:05:46,810 - INFO - Epoch [118/200] (79611) train_loss: 25.0737, val_loss: 26.1572, lr: 0.000418, 130.15s 2023-05-16 01:07:47,925 - INFO - epoch complete! 2023-05-16 01:07:47,926 - INFO - evaluating now! 2023-05-16 01:07:57,007 - INFO - Epoch [119/200] (80280) train_loss: 25.0350, val_loss: 25.9133, lr: 0.000411, 130.20s 2023-05-16 01:07:57,063 - INFO - Saved model at 119 2023-05-16 01:07:57,063 - INFO - Val loss decrease from 26.0291 to 25.9133, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch119.tar 2023-05-16 01:09:58,630 - INFO - epoch complete! 2023-05-16 01:09:58,631 - INFO - evaluating now! 2023-05-16 01:10:07,698 - INFO - Epoch [120/200] (80949) train_loss: 24.9897, val_loss: 26.5156, lr: 0.000404, 130.63s 2023-05-16 01:12:08,814 - INFO - epoch complete! 2023-05-16 01:12:08,814 - INFO - evaluating now! 2023-05-16 01:12:17,875 - INFO - Epoch [121/200] (81618) train_loss: 24.9813, val_loss: 25.9094, lr: 0.000398, 130.18s 2023-05-16 01:12:17,921 - INFO - Saved model at 121 2023-05-16 01:12:17,921 - INFO - Val loss decrease from 25.9133 to 25.9094, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch121.tar 2023-05-16 01:14:19,036 - INFO - epoch complete! 2023-05-16 01:14:19,037 - INFO - evaluating now! 2023-05-16 01:14:28,103 - INFO - Epoch [122/200] (82287) train_loss: 24.9563, val_loss: 26.0171, lr: 0.000391, 130.18s 2023-05-16 01:16:29,226 - INFO - epoch complete! 2023-05-16 01:16:29,226 - INFO - evaluating now! 2023-05-16 01:16:38,307 - INFO - Epoch [123/200] (82956) train_loss: 24.9304, val_loss: 26.0247, lr: 0.000384, 130.20s 2023-05-16 01:18:39,345 - INFO - epoch complete! 2023-05-16 01:18:39,346 - INFO - evaluating now! 2023-05-16 01:18:48,443 - INFO - Epoch [124/200] (83625) train_loss: 24.8439, val_loss: 26.1996, lr: 0.000378, 130.14s 2023-05-16 01:20:53,408 - INFO - epoch complete! 2023-05-16 01:20:53,409 - INFO - evaluating now! 2023-05-16 01:21:02,484 - INFO - Epoch [125/200] (84294) train_loss: 24.9026, val_loss: 25.9366, lr: 0.000371, 134.04s 2023-05-16 01:23:03,626 - INFO - epoch complete! 2023-05-16 01:23:03,627 - INFO - evaluating now! 2023-05-16 01:23:12,780 - INFO - Epoch [126/200] (84963) train_loss: 24.8671, val_loss: 25.9025, lr: 0.000365, 130.30s 2023-05-16 01:23:12,838 - INFO - Saved model at 126 2023-05-16 01:23:12,839 - INFO - Val loss decrease from 25.9094 to 25.9025, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch126.tar 2023-05-16 01:25:14,420 - INFO - epoch complete! 2023-05-16 01:25:14,420 - INFO - evaluating now! 2023-05-16 01:25:23,526 - INFO - Epoch [127/200] (85632) train_loss: 24.8049, val_loss: 25.7937, lr: 0.000358, 130.69s 2023-05-16 01:25:23,584 - INFO - Saved model at 127 2023-05-16 01:25:23,584 - INFO - Val loss decrease from 25.9025 to 25.7937, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch127.tar 2023-05-16 01:27:25,065 - INFO - epoch complete! 2023-05-16 01:27:25,066 - INFO - evaluating now! 2023-05-16 01:27:34,137 - INFO - Epoch [128/200] (86301) train_loss: 24.7467, val_loss: 26.0347, lr: 0.000352, 130.55s 2023-05-16 01:29:35,256 - INFO - epoch complete! 2023-05-16 01:29:35,256 - INFO - evaluating now! 2023-05-16 01:29:44,345 - INFO - Epoch [129/200] (86970) train_loss: 24.7503, val_loss: 25.9023, lr: 0.000346, 130.21s 2023-05-16 01:31:45,455 - INFO - epoch complete! 2023-05-16 01:31:45,456 - INFO - evaluating now! 2023-05-16 01:31:54,582 - INFO - Epoch [130/200] (87639) train_loss: 24.7414, val_loss: 25.8638, lr: 0.000339, 130.24s 2023-05-16 01:33:55,718 - INFO - epoch complete! 2023-05-16 01:33:55,719 - INFO - evaluating now! 2023-05-16 01:34:04,823 - INFO - Epoch [131/200] (88308) train_loss: 24.7133, val_loss: 25.7375, lr: 0.000333, 130.24s 2023-05-16 01:34:04,869 - INFO - Saved model at 131 2023-05-16 01:34:04,869 - INFO - Val loss decrease from 25.7937 to 25.7375, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch131.tar 2023-05-16 01:36:06,004 - INFO - epoch complete! 2023-05-16 01:36:06,005 - INFO - evaluating now! 2023-05-16 01:36:15,080 - INFO - Epoch [132/200] (88977) train_loss: 24.6878, val_loss: 25.9103, lr: 0.000327, 130.21s 2023-05-16 01:38:16,985 - INFO - epoch complete! 2023-05-16 01:38:16,986 - INFO - evaluating now! 2023-05-16 01:38:26,077 - INFO - Epoch [133/200] (89646) train_loss: 24.6237, val_loss: 26.2328, lr: 0.000321, 131.00s 2023-05-16 01:40:27,346 - INFO - epoch complete! 2023-05-16 01:40:27,347 - INFO - evaluating now! 2023-05-16 01:40:36,447 - INFO - Epoch [134/200] (90315) train_loss: 24.6078, val_loss: 26.0244, lr: 0.000315, 130.37s 2023-05-16 01:42:38,057 - INFO - epoch complete! 2023-05-16 01:42:38,057 - INFO - evaluating now! 2023-05-16 01:42:47,158 - INFO - Epoch [135/200] (90984) train_loss: 24.5937, val_loss: 25.7922, lr: 0.000309, 130.71s 2023-05-16 01:44:48,408 - INFO - epoch complete! 2023-05-16 01:44:48,408 - INFO - evaluating now! 2023-05-16 01:44:57,510 - INFO - Epoch [136/200] (91653) train_loss: 24.5435, val_loss: 25.9668, lr: 0.000303, 130.35s 2023-05-16 01:46:58,627 - INFO - epoch complete! 2023-05-16 01:46:58,628 - INFO - evaluating now! 2023-05-16 01:47:07,687 - INFO - Epoch [137/200] (92322) train_loss: 24.5308, val_loss: 25.8127, lr: 0.000297, 130.18s 2023-05-16 01:49:08,700 - INFO - epoch complete! 2023-05-16 01:49:08,701 - INFO - evaluating now! 2023-05-16 01:49:17,814 - INFO - Epoch [138/200] (92991) train_loss: 24.4688, val_loss: 26.1293, lr: 0.000291, 130.13s 2023-05-16 01:51:18,991 - INFO - epoch complete! 2023-05-16 01:51:18,991 - INFO - evaluating now! 2023-05-16 01:51:28,097 - INFO - Epoch [139/200] (93660) train_loss: 24.4767, val_loss: 25.8552, lr: 0.000285, 130.28s 2023-05-16 01:53:29,601 - INFO - epoch complete! 2023-05-16 01:53:29,602 - INFO - evaluating now! 2023-05-16 01:53:38,705 - INFO - Epoch [140/200] (94329) train_loss: 24.4737, val_loss: 25.6744, lr: 0.000280, 130.61s 2023-05-16 01:53:38,753 - INFO - Saved model at 140 2023-05-16 01:53:38,753 - INFO - Val loss decrease from 25.7375 to 25.6744, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch140.tar 2023-05-16 01:55:39,862 - INFO - epoch complete! 2023-05-16 01:55:39,862 - INFO - evaluating now! 2023-05-16 01:55:48,906 - INFO - Epoch [141/200] (94998) train_loss: 24.4508, val_loss: 25.7005, lr: 0.000274, 130.15s 2023-05-16 01:57:50,021 - INFO - epoch complete! 2023-05-16 01:57:50,022 - INFO - evaluating now! 2023-05-16 01:57:59,114 - INFO - Epoch [142/200] (95667) train_loss: 24.4284, val_loss: 25.7961, lr: 0.000269, 130.21s 2023-05-16 02:00:00,335 - INFO - epoch complete! 2023-05-16 02:00:00,336 - INFO - evaluating now! 2023-05-16 02:00:09,412 - INFO - Epoch [143/200] (96336) train_loss: 24.3820, val_loss: 25.9582, lr: 0.000263, 130.30s 2023-05-16 02:02:10,546 - INFO - epoch complete! 2023-05-16 02:02:10,547 - INFO - evaluating now! 2023-05-16 02:02:19,668 - INFO - Epoch [144/200] (97005) train_loss: 24.3501, val_loss: 25.7167, lr: 0.000258, 130.26s 2023-05-16 02:04:20,738 - INFO - epoch complete! 2023-05-16 02:04:20,739 - INFO - evaluating now! 2023-05-16 02:04:29,809 - INFO - Epoch [145/200] (97674) train_loss: 24.3190, val_loss: 25.9623, lr: 0.000252, 130.14s 2023-05-16 02:06:30,938 - INFO - epoch complete! 2023-05-16 02:06:30,939 - INFO - evaluating now! 2023-05-16 02:06:39,975 - INFO - Epoch [146/200] (98343) train_loss: 24.3064, val_loss: 25.9253, lr: 0.000247, 130.17s 2023-05-16 02:08:41,114 - INFO - epoch complete! 2023-05-16 02:08:41,115 - INFO - evaluating now! 2023-05-16 02:08:50,154 - INFO - Epoch [147/200] (99012) train_loss: 24.2813, val_loss: 25.6168, lr: 0.000242, 130.18s 2023-05-16 02:08:50,200 - INFO - Saved model at 147 2023-05-16 02:08:50,200 - INFO - Val loss decrease from 25.6744 to 25.6168, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch147.tar 2023-05-16 02:10:51,311 - INFO - epoch complete! 2023-05-16 02:10:51,312 - INFO - evaluating now! 2023-05-16 02:11:00,381 - INFO - Epoch [148/200] (99681) train_loss: 24.2734, val_loss: 25.7784, lr: 0.000237, 130.18s 2023-05-16 02:13:01,498 - INFO - epoch complete! 2023-05-16 02:13:01,498 - INFO - evaluating now! 2023-05-16 02:13:10,584 - INFO - Epoch [149/200] (100350) train_loss: 24.2418, val_loss: 25.7225, lr: 0.000232, 130.20s 2023-05-16 02:15:11,708 - INFO - epoch complete! 2023-05-16 02:15:11,709 - INFO - evaluating now! 2023-05-16 02:15:20,815 - INFO - Epoch [150/200] (101019) train_loss: 24.2561, val_loss: 25.7978, lr: 0.000227, 130.23s 2023-05-16 02:17:22,143 - INFO - epoch complete! 2023-05-16 02:17:22,143 - INFO - evaluating now! 2023-05-16 02:17:31,275 - INFO - Epoch [151/200] (101688) train_loss: 24.2078, val_loss: 25.9403, lr: 0.000222, 130.46s 2023-05-16 02:19:32,390 - INFO - epoch complete! 2023-05-16 02:19:32,390 - INFO - evaluating now! 2023-05-16 02:19:41,442 - INFO - Epoch [152/200] (102357) train_loss: 24.1935, val_loss: 25.7476, lr: 0.000217, 130.17s 2023-05-16 02:21:42,562 - INFO - epoch complete! 2023-05-16 02:21:42,562 - INFO - evaluating now! 2023-05-16 02:21:51,618 - INFO - Epoch [153/200] (103026) train_loss: 24.1871, val_loss: 25.7341, lr: 0.000212, 130.18s 2023-05-16 02:23:52,742 - INFO - epoch complete! 2023-05-16 02:23:52,742 - INFO - evaluating now! 2023-05-16 02:24:01,790 - INFO - Epoch [154/200] (103695) train_loss: 24.1415, val_loss: 25.6559, lr: 0.000208, 130.17s 2023-05-16 02:26:02,901 - INFO - epoch complete! 2023-05-16 02:26:02,902 - INFO - evaluating now! 2023-05-16 02:26:12,002 - INFO - Epoch [155/200] (104364) train_loss: 24.0990, val_loss: 25.7197, lr: 0.000203, 130.21s 2023-05-16 02:28:13,136 - INFO - epoch complete! 2023-05-16 02:28:13,136 - INFO - evaluating now! 2023-05-16 02:28:22,247 - INFO - Epoch [156/200] (105033) train_loss: 24.1087, val_loss: 25.7969, lr: 0.000199, 130.24s 2023-05-16 02:30:25,206 - INFO - epoch complete! 2023-05-16 02:30:25,207 - INFO - evaluating now! 2023-05-16 02:30:34,287 - INFO - Epoch [157/200] (105702) train_loss: 24.0557, val_loss: 25.6883, lr: 0.000194, 132.04s 2023-05-16 02:32:35,427 - INFO - epoch complete! 2023-05-16 02:32:35,428 - INFO - evaluating now! 2023-05-16 02:32:44,538 - INFO - Epoch [158/200] (106371) train_loss: 24.0410, val_loss: 25.7369, lr: 0.000190, 130.25s 2023-05-16 02:34:45,773 - INFO - epoch complete! 2023-05-16 02:34:45,774 - INFO - evaluating now! 2023-05-16 02:34:54,919 - INFO - Epoch [159/200] (107040) train_loss: 24.0742, val_loss: 25.6552, lr: 0.000186, 130.38s 2023-05-16 02:36:56,047 - INFO - epoch complete! 2023-05-16 02:36:56,048 - INFO - evaluating now! 2023-05-16 02:37:05,182 - INFO - Epoch [160/200] (107709) train_loss: 24.0276, val_loss: 25.7409, lr: 0.000182, 130.26s 2023-05-16 02:39:06,988 - INFO - epoch complete! 2023-05-16 02:39:06,988 - INFO - evaluating now! 2023-05-16 02:39:16,150 - INFO - Epoch [161/200] (108378) train_loss: 24.0082, val_loss: 25.7639, lr: 0.000178, 130.97s 2023-05-16 02:41:17,473 - INFO - epoch complete! 2023-05-16 02:41:17,474 - INFO - evaluating now! 2023-05-16 02:41:26,608 - INFO - Epoch [162/200] (109047) train_loss: 24.0000, val_loss: 25.6198, lr: 0.000174, 130.46s 2023-05-16 02:43:27,631 - INFO - epoch complete! 2023-05-16 02:43:27,631 - INFO - evaluating now! 2023-05-16 02:43:36,751 - INFO - Epoch [163/200] (109716) train_loss: 24.0098, val_loss: 25.7175, lr: 0.000170, 130.14s 2023-05-16 02:45:37,881 - INFO - epoch complete! 2023-05-16 02:45:37,882 - INFO - evaluating now! 2023-05-16 02:45:46,941 - INFO - Epoch [164/200] (110385) train_loss: 23.9626, val_loss: 25.6838, lr: 0.000166, 130.19s 2023-05-16 02:47:48,034 - INFO - epoch complete! 2023-05-16 02:47:48,035 - INFO - evaluating now! 2023-05-16 02:47:57,104 - INFO - Epoch [165/200] (111054) train_loss: 23.9370, val_loss: 25.6295, lr: 0.000163, 130.16s 2023-05-16 02:49:58,223 - INFO - epoch complete! 2023-05-16 02:49:58,224 - INFO - evaluating now! 2023-05-16 02:50:07,336 - INFO - Epoch [166/200] (111723) train_loss: 23.9131, val_loss: 25.6873, lr: 0.000159, 130.23s 2023-05-16 02:52:08,433 - INFO - epoch complete! 2023-05-16 02:52:08,434 - INFO - evaluating now! 2023-05-16 02:52:17,529 - INFO - Epoch [167/200] (112392) train_loss: 23.9074, val_loss: 25.7225, lr: 0.000156, 130.19s 2023-05-16 02:54:18,525 - INFO - epoch complete! 2023-05-16 02:54:18,526 - INFO - evaluating now! 2023-05-16 02:54:27,618 - INFO - Epoch [168/200] (113061) train_loss: 23.9095, val_loss: 25.6056, lr: 0.000152, 130.09s 2023-05-16 02:54:27,665 - INFO - Saved model at 168 2023-05-16 02:54:27,666 - INFO - Val loss decrease from 25.6168 to 25.6056, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch168.tar 2023-05-16 02:56:28,960 - INFO - epoch complete! 2023-05-16 02:56:28,961 - INFO - evaluating now! 2023-05-16 02:56:38,042 - INFO - Epoch [169/200] (113730) train_loss: 23.9050, val_loss: 25.6591, lr: 0.000149, 130.38s 2023-05-16 02:58:39,195 - INFO - epoch complete! 2023-05-16 02:58:39,196 - INFO - evaluating now! 2023-05-16 02:58:48,269 - INFO - Epoch [170/200] (114399) train_loss: 23.9152, val_loss: 25.6050, lr: 0.000146, 130.23s 2023-05-16 02:58:48,315 - INFO - Saved model at 170 2023-05-16 02:58:48,315 - INFO - Val loss decrease from 25.6056 to 25.6050, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch170.tar 2023-05-16 03:00:49,435 - INFO - epoch complete! 2023-05-16 03:00:49,436 - INFO - evaluating now! 2023-05-16 03:00:58,483 - INFO - Epoch [171/200] (115068) train_loss: 23.8431, val_loss: 25.5844, lr: 0.000143, 130.17s 2023-05-16 03:00:58,529 - INFO - Saved model at 171 2023-05-16 03:00:58,529 - INFO - Val loss decrease from 25.6050 to 25.5844, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch171.tar 2023-05-16 03:02:59,775 - INFO - epoch complete! 2023-05-16 03:02:59,775 - INFO - evaluating now! 2023-05-16 03:03:08,869 - INFO - Epoch [172/200] (115737) train_loss: 23.8505, val_loss: 25.6208, lr: 0.000140, 130.34s 2023-05-16 03:05:09,965 - INFO - epoch complete! 2023-05-16 03:05:09,966 - INFO - evaluating now! 2023-05-16 03:05:19,018 - INFO - Epoch [173/200] (116406) train_loss: 23.8237, val_loss: 25.6906, lr: 0.000137, 130.15s 2023-05-16 03:07:20,146 - INFO - epoch complete! 2023-05-16 03:07:20,146 - INFO - evaluating now! 2023-05-16 03:07:29,204 - INFO - Epoch [174/200] (117075) train_loss: 23.8123, val_loss: 25.5942, lr: 0.000134, 130.19s 2023-05-16 03:09:30,308 - INFO - epoch complete! 2023-05-16 03:09:30,308 - INFO - evaluating now! 2023-05-16 03:09:39,362 - INFO - Epoch [175/200] (117744) train_loss: 23.8013, val_loss: 25.6407, lr: 0.000132, 130.16s 2023-05-16 03:11:40,999 - INFO - epoch complete! 2023-05-16 03:11:41,000 - INFO - evaluating now! 2023-05-16 03:11:50,133 - INFO - Epoch [176/200] (118413) train_loss: 23.8094, val_loss: 25.5675, lr: 0.000129, 130.77s 2023-05-16 03:11:50,180 - INFO - Saved model at 176 2023-05-16 03:11:50,181 - INFO - Val loss decrease from 25.5844 to 25.5675, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch176.tar 2023-05-16 03:13:51,295 - INFO - epoch complete! 2023-05-16 03:13:51,296 - INFO - evaluating now! 2023-05-16 03:14:00,347 - INFO - Epoch [177/200] (119082) train_loss: 23.7840, val_loss: 25.5798, lr: 0.000127, 130.17s 2023-05-16 03:16:01,472 - INFO - epoch complete! 2023-05-16 03:16:01,473 - INFO - evaluating now! 2023-05-16 03:16:10,545 - INFO - Epoch [178/200] (119751) train_loss: 23.7766, val_loss: 25.6596, lr: 0.000124, 130.20s 2023-05-16 03:18:11,698 - INFO - epoch complete! 2023-05-16 03:18:11,699 - INFO - evaluating now! 2023-05-16 03:18:20,762 - INFO - Epoch [179/200] (120420) train_loss: 23.7469, val_loss: 25.5890, lr: 0.000122, 130.22s 2023-05-16 03:20:21,867 - INFO - epoch complete! 2023-05-16 03:20:21,868 - INFO - evaluating now! 2023-05-16 03:20:30,927 - INFO - Epoch [180/200] (121089) train_loss: 23.7648, val_loss: 25.6002, lr: 0.000120, 130.16s 2023-05-16 03:22:32,712 - INFO - epoch complete! 2023-05-16 03:22:32,712 - INFO - evaluating now! 2023-05-16 03:22:41,742 - INFO - Epoch [181/200] (121758) train_loss: 23.7501, val_loss: 25.6632, lr: 0.000118, 130.82s 2023-05-16 03:24:43,311 - INFO - epoch complete! 2023-05-16 03:24:43,312 - INFO - evaluating now! 2023-05-16 03:24:52,373 - INFO - Epoch [182/200] (122427) train_loss: 23.7115, val_loss: 25.5730, lr: 0.000116, 130.63s 2023-05-16 03:26:53,505 - INFO - epoch complete! 2023-05-16 03:26:53,505 - INFO - evaluating now! 2023-05-16 03:27:02,573 - INFO - Epoch [183/200] (123096) train_loss: 23.7255, val_loss: 25.5833, lr: 0.000114, 130.20s 2023-05-16 03:29:03,655 - INFO - epoch complete! 2023-05-16 03:29:03,655 - INFO - evaluating now! 2023-05-16 03:29:12,722 - INFO - Epoch [184/200] (123765) train_loss: 23.7209, val_loss: 25.5907, lr: 0.000112, 130.15s 2023-05-16 03:31:14,128 - INFO - epoch complete! 2023-05-16 03:31:14,128 - INFO - evaluating now! 2023-05-16 03:31:23,213 - INFO - Epoch [185/200] (124434) train_loss: 23.7185, val_loss: 25.6445, lr: 0.000111, 130.49s 2023-05-16 03:33:24,335 - INFO - epoch complete! 2023-05-16 03:33:24,335 - INFO - evaluating now! 2023-05-16 03:33:33,456 - INFO - Epoch [186/200] (125103) train_loss: 23.6967, val_loss: 25.5615, lr: 0.000109, 130.24s 2023-05-16 03:33:33,503 - INFO - Saved model at 186 2023-05-16 03:33:33,503 - INFO - Val loss decrease from 25.5675 to 25.5615, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch186.tar 2023-05-16 03:35:34,614 - INFO - epoch complete! 2023-05-16 03:35:34,614 - INFO - evaluating now! 2023-05-16 03:35:43,698 - INFO - Epoch [187/200] (125772) train_loss: 23.6737, val_loss: 25.5817, lr: 0.000108, 130.19s 2023-05-16 03:37:44,838 - INFO - epoch complete! 2023-05-16 03:37:44,838 - INFO - evaluating now! 2023-05-16 03:37:53,893 - INFO - Epoch [188/200] (126441) train_loss: 23.6837, val_loss: 25.5882, lr: 0.000107, 130.20s 2023-05-16 03:39:55,023 - INFO - epoch complete! 2023-05-16 03:39:55,023 - INFO - evaluating now! 2023-05-16 03:40:04,065 - INFO - Epoch [189/200] (127110) train_loss: 23.6397, val_loss: 25.6509, lr: 0.000106, 130.17s 2023-05-16 03:42:05,158 - INFO - epoch complete! 2023-05-16 03:42:05,158 - INFO - evaluating now! 2023-05-16 03:42:14,222 - INFO - Epoch [190/200] (127779) train_loss: 23.6684, val_loss: 25.5988, lr: 0.000104, 130.16s 2023-05-16 03:44:13,958 - INFO - epoch complete! 2023-05-16 03:44:13,959 - INFO - evaluating now! 2023-05-16 03:44:23,042 - INFO - Epoch [191/200] (128448) train_loss: 23.6490, val_loss: 25.5992, lr: 0.000104, 128.82s 2023-05-16 03:46:25,237 - INFO - epoch complete! 2023-05-16 03:46:25,237 - INFO - evaluating now! 2023-05-16 03:46:34,364 - INFO - Epoch [192/200] (129117) train_loss: 23.6423, val_loss: 25.5679, lr: 0.000103, 131.32s 2023-05-16 03:48:36,099 - INFO - epoch complete! 2023-05-16 03:48:36,099 - INFO - evaluating now! 2023-05-16 03:48:45,179 - INFO - Epoch [193/200] (129786) train_loss: 23.6574, val_loss: 25.6093, lr: 0.000102, 130.81s 2023-05-16 03:50:46,440 - INFO - epoch complete! 2023-05-16 03:50:46,441 - INFO - evaluating now! 2023-05-16 03:50:55,500 - INFO - Epoch [194/200] (130455) train_loss: 23.6118, val_loss: 25.5810, lr: 0.000101, 130.32s 2023-05-16 03:52:59,040 - INFO - epoch complete! 2023-05-16 03:52:59,041 - INFO - evaluating now! 2023-05-16 03:53:08,137 - INFO - Epoch [195/200] (131124) train_loss: 23.6370, val_loss: 25.5791, lr: 0.000101, 132.64s 2023-05-16 03:55:10,938 - INFO - epoch complete! 2023-05-16 03:55:10,938 - INFO - evaluating now! 2023-05-16 03:55:20,048 - INFO - Epoch [196/200] (131793) train_loss: 23.6381, val_loss: 25.6610, lr: 0.000100, 131.91s 2023-05-16 03:57:22,349 - INFO - epoch complete! 2023-05-16 03:57:22,349 - INFO - evaluating now! 2023-05-16 03:57:31,410 - INFO - Epoch [197/200] (132462) train_loss: 23.6208, val_loss: 25.6042, lr: 0.000100, 131.36s 2023-05-16 03:59:33,810 - INFO - epoch complete! 2023-05-16 03:59:33,810 - INFO - evaluating now! 2023-05-16 03:59:42,884 - INFO - Epoch [198/200] (133131) train_loss: 23.6136, val_loss: 25.5789, lr: 0.000100, 131.47s 2023-05-16 04:01:44,609 - INFO - epoch complete! 2023-05-16 04:01:44,609 - INFO - evaluating now! 2023-05-16 04:01:53,709 - INFO - Epoch [199/200] (133800) train_loss: 23.6315, val_loss: 25.5475, lr: 0.000100, 130.82s 2023-05-16 04:01:53,755 - INFO - Saved model at 199 2023-05-16 04:01:53,755 - INFO - Val loss decrease from 25.5615 to 25.5475, saving to ./libcity/cache/30930/model_cache/PDFormer_PeMS08_epoch199.tar 2023-05-16 04:01:53,756 - INFO - Trained totally 200 epochs, average train time is 121.319s, average eval time is 9.083s 2023-05-16 04:01:53,843 - INFO - Loaded model at 199 2023-05-16 04:01:53,844 - INFO - Saved model at ./libcity/cache/30930/model_cache/PDFormer_PeMS08.m 2023-05-16 04:01:53,889 - INFO - Start evaluating ... 2023-05-16 04:02:11,680 - INFO - Note that you select the average mode to evaluate! 2023-05-16 04:02:11,748 - INFO - Evaluate result is saved at ./libcity/cache/30930/evaluate_cache/2023_05_16_04_02_11_PDFormer_PeMS08_average.csv 2023-05-16 04:02:11,811 - INFO - MAE MAPE RMSE masked_MAE masked_MAPE masked_RMSE 1 11.744327 inf 19.637644 11.760401 0.077948 19.529140 2 11.975752 inf 20.247381 11.992254 0.079423 20.141357 3 12.196908 inf 20.769762 12.214051 0.080845 20.666855 4 12.393086 inf 21.220171 12.410814 0.082166 21.121649 5 12.565434 inf 21.609114 12.583639 0.083352 21.512920 6 12.720485 inf 21.951965 12.739080 0.084445 21.857193 7 12.865274 inf 22.262390 12.884212 0.085499 22.168737 8 13.001018 inf 22.545931 13.020285 0.086475 22.453295 9 13.128123 inf 22.803295 13.147656 0.087415 22.711014 10 13.249768 inf 23.042545 13.269598 0.088333 22.951004 11 13.386254 inf 23.260063 13.406418 0.089319 23.169426 12 13.558510 inf 23.498171 13.579021 0.090486 23.408354 ```

PEMS08-服务器2

``` 2023-05-18 17:15:23,170 - INFO - Log directory: ./libcity/log 2023-05-18 17:15:23,171 - INFO - Begin pipeline, task=traffic_state_pred, model_name=PDFormer, dataset_name=PeMS08, exp_id=92489 2023-05-18 17:15:23,171 - INFO - {'task': 'traffic_state_pred', 'model': 'PDFormer', 'dataset': 'PeMS08', 'saved_model': True, 'train': True, 'local_rank': 0, 'gpu_id': [0], 'initial_ckpt': None, 'dataset_class': 'PDFormerDataset', 'input_window': 12, 'output_window': 12, 'train_rate': 0.6, 'eval_rate': 0.2, 'batch_size': 16, 'add_time_in_day': True, 'add_day_in_week': True, 'step_size': 2776, 'max_epoch': 200, 'bidir': True, 'far_mask_delta': 7, 'geo_num_heads': 4, 'sem_num_heads': 2, 't_num_heads': 2, 'cluster_method': 'kshape', 'cand_key_days': 21, 'seed': 1, 'type_ln': 'pre', 'set_loss': 'huber', 'huber_delta': 2, 'mode': 'average', 'executor': 'PDFormerExecutor', 'evaluator': 'TrafficStateEvaluator', 'embed_dim': 64, 'skip_dim': 256, 'mlp_ratio': 4, 'qkv_bias': True, 'drop': 0, 'attn_drop': 0, 'drop_path': 0.3, 's_attn_size': 3, 't_attn_size': 1, 'enc_depth': 6, 'type_short_path': 'hop', 'scaler': 'standard', 'load_external': True, 'normal_external': False, 'ext_scaler': 'none', 'learner': 'adamw', 'learning_rate': 0.001, 'weight_decay': 0.05, 'lr_decay': True, 'lr_scheduler': 'cosinelr', 'lr_eta_min': 0.0001, 'lr_decay_ratio': 0.1, 'lr_warmup_epoch': 5, 'lr_warmup_init': 1e-06, 'clip_grad_norm': True, 'max_grad_norm': 5, 'use_early_stop': True, 'patience': 50, 'task_level': 0, 'use_curriculum_learning': True, 'random_flip': True, 'quan_delta': 0.25, 'dtw_delta': 5, 'cache_dataset': True, 'num_workers': 0, 'pad_with_last_sample': True, 'lape_dim': 8, 'gpu': True, 'train_loss': 'none', 'epoch': 0, 'lr_epsilon': 1e-08, 'lr_beta1': 0.9, 'lr_beta2': 0.999, 'lr_alpha': 0.99, 'lr_momentum': 0, 'steps': [5, 20, 40, 70], 'lr_T_max': 30, 'lr_patience': 10, 'lr_threshold': 0.0001, 'log_level': 'INFO', 'log_every': 1, 'load_best_epoch': True, 'hyper_tune': False, 'grad_accmu_steps': 1, 'metrics': ['MAE', 'MAPE', 'RMSE', 'masked_MAE', 'masked_MAPE', 'masked_RMSE'], 'save_modes': ['csv'], 'geo': {'including_types': ['Point'], 'Point': {}}, 'rel': {'including_types': ['geo'], 'geo': {'cost': 'num'}}, 'dyna': {'including_types': ['state'], 'state': {'entity_id': 'geo_id', 'traffic_flow': 'num', 'traffic_occupancy': 'num', 'traffic_speed': 'num'}}, 'data_col': ['traffic_flow'], 'weight_col': 'cost', 'data_files': ['PeMS08'], 'geo_file': 'PeMS08', 'rel_file': 'PeMS08', 'output_dim': 1, 'time_intervals': 300, 'init_weight_inf_or_zero': 'zero', 'set_weight_link_or_dist': 'link', 'calculate_weight_adj': False, 'weight_adj_epsilon': 0.1, 'distributed': False, 'device': device(type='cuda', index=0), 'exp_id': 92489} 2023-05-18 17:15:23,622 - INFO - Loaded file PeMS08.geo, num_nodes=170 2023-05-18 17:15:23,624 - INFO - set_weight_link_or_dist: link 2023-05-18 17:15:23,624 - INFO - init_weight_inf_or_zero: zero 2023-05-18 17:15:23,627 - INFO - Loaded file PeMS08.rel, shape=(170, 170) 2023-05-18 17:15:23,628 - INFO - Max adj_mx value = 1.0 2023-05-18 17:15:32,997 - INFO - Loading file PeMS08.dyna 2023-05-18 17:15:34,576 - INFO - Loaded file PeMS08.dyna, shape=(17856, 170, 1) 2023-05-18 17:15:34,596 - INFO - Load DTW matrix from ./libcity/cache/dataset_cache/dtw_PeMS08.npy 2023-05-18 17:15:34,597 - INFO - Loading ./libcity/cache/dataset_cache/pdformer_point_based_PeMS08_12_12_0.6_1_0.2_standard_16_True_True_True_True_traffic_flow.npz 2023-05-18 17:15:40,743 - INFO - train x: (10700, 12, 170, 9), y: (10700, 12, 170, 9) 2023-05-18 17:15:40,743 - INFO - eval x: (3566, 12, 170, 9), y: (3566, 12, 170, 9) 2023-05-18 17:15:40,743 - INFO - test x: (3567, 12, 170, 9), y: (3567, 12, 170, 9) 2023-05-18 17:15:41,170 - INFO - StandardScaler mean: 229.8431355598314, std: 145.62553066568907 2023-05-18 17:15:41,171 - INFO - NoneScaler 2023-05-18 17:15:44,918 - INFO - Loaded file ./libcity/cache/dataset_cache/pattern_keys_kshape_PeMS08_21_3_16_5.npy 2023-05-18 17:15:44,934 - INFO - Use use_curriculum_learning! 2023-05-18 17:15:47,729 - INFO - PDFormer( (pattern_embeddings): ModuleList( (0): TokenEmbedding( (token_embed): Linear(in_features=3, out_features=64, bias=True) (norm): Identity() ) ) (enc_embed_layer): DataEmbedding( (value_embedding): TokenEmbedding( (token_embed): Linear(in_features=1, out_features=64, bias=True) (norm): Identity() ) (position_encoding): PositionalEncoding() (daytime_embedding): Embedding(1440, 64) (weekday_embedding): Embedding(7, 64) (spatial_embedding): LaplacianPE( (embedding_lap_pos_enc): Linear(in_features=8, out_features=64, bias=True) ) (dropout): Dropout(p=0, inplace=False) ) (encoder_blocks): ModuleList( (0): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): Identity() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (1): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (2): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (3): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (4): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (5): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) ) (skip_convs): ModuleList( (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (1): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (2): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (4): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (5): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) ) (end_conv1): Conv2d(12, 12, kernel_size=(1, 1), stride=(1, 1)) (end_conv2): Conv2d(256, 1, kernel_size=(1, 1), stride=(1, 1)) ) 2023-05-18 17:15:47,731 - INFO - pattern_embeddings.0.token_embed.weight torch.Size([64, 3]) cuda:0 True 2023-05-18 17:15:47,731 - INFO - pattern_embeddings.0.token_embed.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,731 - INFO - enc_embed_layer.value_embedding.token_embed.weight torch.Size([64, 1]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - enc_embed_layer.value_embedding.token_embed.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - enc_embed_layer.daytime_embedding.weight torch.Size([1440, 64]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - enc_embed_layer.weekday_embedding.weight torch.Size([7, 64]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - enc_embed_layer.spatial_embedding.embedding_lap_pos_enc.weight torch.Size([64, 8]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - enc_embed_layer.spatial_embedding.embedding_lap_pos_enc.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.norm1.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.norm1.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,732 - INFO - encoder_blocks.0.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.norm2.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.norm2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.0.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.norm1.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.norm1.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,733 - INFO - encoder_blocks.1.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.norm2.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.norm2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.1.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.2.norm1.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.2.norm1.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.2.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.2.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,734 - INFO - encoder_blocks.2.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.norm2.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.norm2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-18 17:15:47,735 - INFO - encoder_blocks.2.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.2.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.2.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.norm1.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.norm1.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,736 - INFO - encoder_blocks.3.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.3.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.3.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.3.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.3.norm2.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.3.norm2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.3.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.3.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.3.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.3.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.norm1.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.norm1.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,737 - INFO - encoder_blocks.4.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.norm2.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.norm2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.4.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.norm1.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.norm1.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,738 - INFO - encoder_blocks.5.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.norm2.weight torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.norm2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - encoder_blocks.5.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - skip_convs.0.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - skip_convs.0.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,739 - INFO - skip_convs.1.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - skip_convs.1.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - skip_convs.2.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - skip_convs.2.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - skip_convs.3.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - skip_convs.3.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - skip_convs.4.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - skip_convs.4.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - skip_convs.5.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - skip_convs.5.bias torch.Size([256]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - end_conv1.weight torch.Size([12, 12, 1, 1]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - end_conv1.bias torch.Size([12]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - end_conv2.weight torch.Size([1, 256, 1, 1]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - end_conv2.bias torch.Size([1]) cuda:0 True 2023-05-18 17:15:47,740 - INFO - Total parameter numbers: 531165 2023-05-18 17:15:47,740 - INFO - You select `adamw` optimizer. 2023-05-18 17:15:47,741 - INFO - You select `cosinelr` lr_scheduler. 2023-05-18 17:15:47,741 - WARNING - Received none train loss func and will use the loss func defined in the model. 2023-05-18 17:15:47,742 - INFO - Number of isolated points: 0 2023-05-18 17:15:47,756 - INFO - Start training ... 2023-05-18 17:15:47,756 - INFO - num_batches:669 2023-05-18 17:15:48,400 - INFO - Training: task_level increase from 0 to 1 2023-05-18 17:15:48,400 - INFO - Current batches_seen is 0 2023-05-18 17:17:51,247 - INFO - epoch complete! 2023-05-18 17:17:51,247 - INFO - evaluating now! 2023-05-18 17:17:57,733 - INFO - Epoch [0/200] (669) train_loss: 176.9078, val_loss: 212.7161, lr: 0.000201, 129.98s 2023-05-18 17:17:57,766 - INFO - Saved model at 0 2023-05-18 17:17:57,766 - INFO - Val loss decrease from inf to 212.7161, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch0.tar 2023-05-18 17:19:59,246 - INFO - epoch complete! 2023-05-18 17:19:59,246 - INFO - evaluating now! 2023-05-18 17:20:05,742 - INFO - Epoch [1/200] (1338) train_loss: 46.6429, val_loss: 200.3482, lr: 0.000401, 127.97s 2023-05-18 17:20:05,775 - INFO - Saved model at 1 2023-05-18 17:20:05,775 - INFO - Val loss decrease from 212.7161 to 200.3482, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch1.tar 2023-05-18 17:22:11,937 - INFO - epoch complete! 2023-05-18 17:22:11,937 - INFO - evaluating now! 2023-05-18 17:22:18,451 - INFO - Epoch [2/200] (2007) train_loss: 35.1762, val_loss: 201.7594, lr: 0.000600, 132.68s 2023-05-18 17:24:16,115 - INFO - epoch complete! 2023-05-18 17:24:16,116 - INFO - evaluating now! 2023-05-18 17:24:22,618 - INFO - Epoch [3/200] (2676) train_loss: 31.3463, val_loss: 203.3758, lr: 0.000800, 124.17s 2023-05-18 17:24:40,423 - INFO - Training: task_level increase from 1 to 2 2023-05-18 17:24:40,423 - INFO - Current batches_seen is 2776 2023-05-18 17:26:24,869 - INFO - epoch complete! 2023-05-18 17:26:24,869 - INFO - evaluating now! 2023-05-18 17:26:31,357 - INFO - Epoch [4/200] (3345) train_loss: 32.9974, val_loss: 177.3616, lr: 0.000999, 128.74s 2023-05-18 17:26:31,390 - INFO - Saved model at 4 2023-05-18 17:26:31,390 - INFO - Val loss decrease from 200.3482 to 177.3616, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch4.tar 2023-05-18 17:28:34,860 - INFO - epoch complete! 2023-05-18 17:28:34,860 - INFO - evaluating now! 2023-05-18 17:28:41,369 - INFO - Epoch [5/200] (4014) train_loss: 30.3677, val_loss: 176.7822, lr: 0.000998, 129.98s 2023-05-18 17:28:41,402 - INFO - Saved model at 5 2023-05-18 17:28:41,402 - INFO - Val loss decrease from 177.3616 to 176.7822, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch5.tar 2023-05-18 17:30:42,301 - INFO - epoch complete! 2023-05-18 17:30:42,302 - INFO - evaluating now! 2023-05-18 17:30:48,802 - INFO - Epoch [6/200] (4683) train_loss: 29.2318, val_loss: 177.3306, lr: 0.000997, 127.40s 2023-05-18 17:32:49,393 - INFO - epoch complete! 2023-05-18 17:32:49,394 - INFO - evaluating now! 2023-05-18 17:32:55,876 - INFO - Epoch [7/200] (5352) train_loss: 28.9159, val_loss: 176.4998, lr: 0.000996, 127.07s 2023-05-18 17:32:55,909 - INFO - Saved model at 7 2023-05-18 17:32:55,909 - INFO - Val loss decrease from 176.7822 to 176.4998, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch7.tar 2023-05-18 17:33:32,893 - INFO - Training: task_level increase from 2 to 3 2023-05-18 17:33:32,894 - INFO - Current batches_seen is 5552 2023-05-18 17:34:59,888 - INFO - epoch complete! 2023-05-18 17:34:59,888 - INFO - evaluating now! 2023-05-18 17:35:06,378 - INFO - Epoch [8/200] (6021) train_loss: 29.6127, val_loss: 174.1752, lr: 0.000996, 130.47s 2023-05-18 17:35:06,411 - INFO - Saved model at 8 2023-05-18 17:35:06,411 - INFO - Val loss decrease from 176.4998 to 174.1752, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch8.tar 2023-05-18 17:37:10,488 - INFO - epoch complete! 2023-05-18 17:37:10,489 - INFO - evaluating now! 2023-05-18 17:37:16,994 - INFO - Epoch [9/200] (6690) train_loss: 29.3607, val_loss: 173.5539, lr: 0.000994, 130.58s 2023-05-18 17:37:17,026 - INFO - Saved model at 9 2023-05-18 17:37:17,027 - INFO - Val loss decrease from 174.1752 to 173.5539, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch9.tar 2023-05-18 17:39:12,533 - INFO - epoch complete! 2023-05-18 17:39:12,533 - INFO - evaluating now! 2023-05-18 17:39:18,969 - INFO - Epoch [10/200] (7359) train_loss: 29.0077, val_loss: 174.4257, lr: 0.000993, 121.94s 2023-05-18 17:41:24,646 - INFO - epoch complete! 2023-05-18 17:41:24,646 - INFO - evaluating now! 2023-05-18 17:41:31,157 - INFO - Epoch [11/200] (8028) train_loss: 28.5478, val_loss: 173.5295, lr: 0.000992, 124.43s 2023-05-18 17:41:31,191 - INFO - Saved model at 11 2023-05-18 17:41:31,191 - INFO - Val loss decrease from 173.5539 to 173.5295, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch11.tar 2023-05-18 17:42:24,156 - INFO - Training: task_level increase from 3 to 4 2023-05-18 17:42:24,157 - INFO - Current batches_seen is 8328 2023-05-18 17:43:30,871 - INFO - epoch complete! 2023-05-18 17:43:30,872 - INFO - evaluating now! 2023-05-18 17:43:37,362 - INFO - Epoch [12/200] (8697) train_loss: 30.4748, val_loss: 155.6604, lr: 0.000991, 126.17s 2023-05-18 17:43:43,621 - INFO - Saved model at 12 2023-05-18 17:43:43,621 - INFO - Val loss decrease from 173.5295 to 155.6604, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch12.tar 2023-05-18 17:45:48,948 - INFO - epoch complete! 2023-05-18 17:45:48,949 - INFO - evaluating now! 2023-05-18 17:45:55,453 - INFO - Epoch [13/200] (9366) train_loss: 29.2414, val_loss: 155.3377, lr: 0.000989, 131.83s 2023-05-18 17:46:02,839 - INFO - Saved model at 13 2023-05-18 17:46:02,839 - INFO - Val loss decrease from 155.6604 to 155.3377, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch13.tar 2023-05-18 17:48:07,680 - INFO - epoch complete! 2023-05-18 17:48:07,681 - INFO - evaluating now! 2023-05-18 17:48:14,167 - INFO - Epoch [14/200] (10035) train_loss: 28.8409, val_loss: 155.8029, lr: 0.000988, 131.33s 2023-05-18 17:50:27,319 - INFO - epoch complete! 2023-05-18 17:50:27,319 - INFO - evaluating now! 2023-05-18 17:50:33,828 - INFO - Epoch [15/200] (10704) train_loss: 28.5878, val_loss: 155.5463, lr: 0.000986, 132.05s 2023-05-18 17:51:56,694 - INFO - Training: task_level increase from 4 to 5 2023-05-18 17:51:56,694 - INFO - Current batches_seen is 11104 2023-05-18 17:52:47,281 - INFO - epoch complete! 2023-05-18 17:52:47,281 - INFO - evaluating now! 2023-05-18 17:52:53,793 - INFO - Epoch [16/200] (11373) train_loss: 29.4867, val_loss: 138.3832, lr: 0.000984, 132.00s 2023-05-18 17:53:01,435 - INFO - Saved model at 16 2023-05-18 17:53:01,436 - INFO - Val loss decrease from 155.3377 to 138.3832, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch16.tar 2023-05-18 17:55:06,574 - INFO - epoch complete! 2023-05-18 17:55:06,574 - INFO - evaluating now! 2023-05-18 17:55:13,093 - INFO - Epoch [17/200] (12042) train_loss: 29.1752, val_loss: 139.0722, lr: 0.000982, 131.66s 2023-05-18 17:57:25,687 - INFO - epoch complete! 2023-05-18 17:57:25,688 - INFO - evaluating now! 2023-05-18 17:57:32,206 - INFO - Epoch [18/200] (12711) train_loss: 29.0471, val_loss: 138.5618, lr: 0.000980, 132.32s 2023-05-18 17:59:45,525 - INFO - epoch complete! 2023-05-18 17:59:45,525 - INFO - evaluating now! 2023-05-18 17:59:52,035 - INFO - Epoch [19/200] (13380) train_loss: 28.7257, val_loss: 138.9554, lr: 0.000978, 132.21s 2023-05-18 18:01:33,454 - INFO - Training: task_level increase from 5 to 6 2023-05-18 18:01:33,454 - INFO - Current batches_seen is 13880 2023-05-18 18:02:04,744 - INFO - epoch complete! 2023-05-18 18:02:04,744 - INFO - evaluating now! 2023-05-18 18:02:11,222 - INFO - Epoch [20/200] (14049) train_loss: 29.8404, val_loss: 122.2715, lr: 0.000976, 131.62s 2023-05-18 18:02:16,765 - INFO - Saved model at 20 2023-05-18 18:02:16,765 - INFO - Val loss decrease from 138.3832 to 122.2715, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch20.tar 2023-05-18 18:04:22,685 - INFO - epoch complete! 2023-05-18 18:04:22,685 - INFO - evaluating now! 2023-05-18 18:04:29,182 - INFO - Epoch [21/200] (14718) train_loss: 29.3494, val_loss: 122.5044, lr: 0.000973, 132.42s 2023-05-18 18:06:42,404 - INFO - epoch complete! 2023-05-18 18:06:42,405 - INFO - evaluating now! 2023-05-18 18:06:48,943 - INFO - Epoch [22/200] (15387) train_loss: 29.1743, val_loss: 123.0121, lr: 0.000971, 132.36s 2023-05-18 18:09:02,282 - INFO - epoch complete! 2023-05-18 18:09:02,282 - INFO - evaluating now! 2023-05-18 18:09:08,730 - INFO - Epoch [23/200] (16056) train_loss: 28.8291, val_loss: 123.1461, lr: 0.000968, 132.11s 2023-05-18 18:11:09,329 - INFO - Training: task_level increase from 6 to 7 2023-05-18 18:11:09,330 - INFO - Current batches_seen is 16656 2023-05-18 18:11:22,317 - INFO - epoch complete! 2023-05-18 18:11:23,680 - INFO - evaluating now! 2023-05-18 18:11:30,202 - INFO - Epoch [24/200] (16725) train_loss: 29.3896, val_loss: 107.0965, lr: 0.000966, 133.50s 2023-05-18 18:11:30,236 - INFO - Saved model at 24 2023-05-18 18:11:30,236 - INFO - Val loss decrease from 122.2715 to 107.0965, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch24.tar 2023-05-18 18:13:36,496 - INFO - epoch complete! 2023-05-18 18:13:36,497 - INFO - evaluating now! 2023-05-18 18:13:42,994 - INFO - Epoch [25/200] (17394) train_loss: 29.4946, val_loss: 106.3261, lr: 0.000963, 132.76s 2023-05-18 18:13:50,701 - INFO - Saved model at 25 2023-05-18 18:13:50,702 - INFO - Val loss decrease from 107.0965 to 106.3261, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch25.tar 2023-05-18 18:15:56,479 - INFO - epoch complete! 2023-05-18 18:15:56,480 - INFO - evaluating now! 2023-05-18 18:16:02,988 - INFO - Epoch [26/200] (18063) train_loss: 29.1706, val_loss: 106.3497, lr: 0.000960, 132.29s 2023-05-18 18:18:16,106 - INFO - epoch complete! 2023-05-18 18:18:16,107 - INFO - evaluating now! 2023-05-18 18:18:22,614 - INFO - Epoch [27/200] (18732) train_loss: 28.9678, val_loss: 106.8563, lr: 0.000957, 131.98s 2023-05-18 18:20:36,613 - INFO - epoch complete! 2023-05-18 18:20:36,613 - INFO - evaluating now! 2023-05-18 18:20:43,114 - INFO - Epoch [28/200] (19401) train_loss: 28.8837, val_loss: 107.0176, lr: 0.000954, 132.43s 2023-05-18 18:20:56,572 - INFO - Training: task_level increase from 7 to 8 2023-05-18 18:20:56,573 - INFO - Current batches_seen is 19432 2023-05-18 18:22:56,609 - INFO - epoch complete! 2023-05-18 18:22:56,610 - INFO - evaluating now! 2023-05-18 18:23:03,112 - INFO - Epoch [29/200] (20070) train_loss: 29.7815, val_loss: 89.8299, lr: 0.000951, 132.37s 2023-05-18 18:23:10,735 - INFO - Saved model at 29 2023-05-18 18:23:10,735 - INFO - Val loss decrease from 106.3261 to 89.8299, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch29.tar 2023-05-18 18:25:15,984 - INFO - epoch complete! 2023-05-18 18:25:15,984 - INFO - evaluating now! 2023-05-18 18:25:22,505 - INFO - Epoch [30/200] (20739) train_loss: 29.3130, val_loss: 89.9346, lr: 0.000948, 131.77s 2023-05-18 18:27:34,590 - INFO - epoch complete! 2023-05-18 18:27:34,590 - INFO - evaluating now! 2023-05-18 18:27:41,101 - INFO - Epoch [31/200] (21408) train_loss: 29.0626, val_loss: 90.2729, lr: 0.000944, 132.67s 2023-05-18 18:29:54,945 - INFO - epoch complete! 2023-05-18 18:29:54,946 - INFO - evaluating now! 2023-05-18 18:30:01,442 - INFO - Epoch [32/200] (22077) train_loss: 29.0000, val_loss: 90.3633, lr: 0.000941, 132.70s 2023-05-18 18:30:33,454 - INFO - Training: task_level increase from 8 to 9 2023-05-18 18:30:33,454 - INFO - Current batches_seen is 22208 2023-05-18 18:32:14,649 - INFO - epoch complete! 2023-05-18 18:32:14,650 - INFO - evaluating now! 2023-05-18 18:32:21,103 - INFO - Epoch [33/200] (22746) train_loss: 29.7736, val_loss: 82.6911, lr: 0.000937, 132.32s 2023-05-18 18:32:21,136 - INFO - Saved model at 33 2023-05-18 18:32:21,136 - INFO - Val loss decrease from 89.8299 to 82.6911, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch33.tar 2023-05-18 18:34:27,258 - INFO - epoch complete! 2023-05-18 18:34:27,258 - INFO - evaluating now! 2023-05-18 18:34:33,784 - INFO - Epoch [34/200] (23415) train_loss: 29.2638, val_loss: 82.4552, lr: 0.000934, 132.65s 2023-05-18 18:34:41,805 - INFO - Saved model at 34 2023-05-18 18:34:41,805 - INFO - Val loss decrease from 82.6911 to 82.4552, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch34.tar 2023-05-18 18:36:46,130 - INFO - epoch complete! 2023-05-18 18:36:46,130 - INFO - evaluating now! 2023-05-18 18:36:52,649 - INFO - Epoch [35/200] (24084) train_loss: 29.3555, val_loss: 82.5643, lr: 0.000930, 130.84s 2023-05-18 18:38:49,114 - INFO - epoch complete! 2023-05-18 18:38:49,114 - INFO - evaluating now! 2023-05-18 18:38:55,610 - INFO - Epoch [36/200] (24753) train_loss: 29.1596, val_loss: 82.9159, lr: 0.000926, 122.96s 2023-05-18 18:39:43,378 - INFO - Training: task_level increase from 9 to 10 2023-05-18 18:39:43,378 - INFO - Current batches_seen is 24984 2023-05-18 18:40:59,336 - INFO - epoch complete! 2023-05-18 18:40:59,336 - INFO - evaluating now! 2023-05-18 18:41:05,850 - INFO - Epoch [37/200] (25422) train_loss: 29.6728, val_loss: 64.9288, lr: 0.000922, 122.32s 2023-05-18 18:41:13,704 - INFO - Saved model at 37 2023-05-18 18:41:13,705 - INFO - Val loss decrease from 82.4552 to 64.9288, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch37.tar 2023-05-18 18:43:10,074 - INFO - epoch complete! 2023-05-18 18:43:10,074 - INFO - evaluating now! 2023-05-18 18:43:16,600 - INFO - Epoch [38/200] (26091) train_loss: 29.5391, val_loss: 65.1675, lr: 0.000918, 122.89s 2023-05-18 18:45:22,986 - INFO - epoch complete! 2023-05-18 18:45:22,987 - INFO - evaluating now! 2023-05-18 18:45:29,486 - INFO - Epoch [39/200] (26760) train_loss: 29.3875, val_loss: 64.9553, lr: 0.000914, 125.07s 2023-05-18 18:47:39,966 - INFO - epoch complete! 2023-05-18 18:47:39,966 - INFO - evaluating now! 2023-05-18 18:47:54,943 - INFO - Epoch [40/200] (27429) train_loss: 29.2681, val_loss: 64.5896, lr: 0.000910, 137.46s 2023-05-18 18:47:54,976 - INFO - Saved model at 40 2023-05-18 18:47:54,976 - INFO - Val loss decrease from 64.9288 to 64.5896, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch40.tar 2023-05-18 18:48:55,771 - INFO - Training: task_level increase from 10 to 11 2023-05-18 18:48:55,771 - INFO - Current batches_seen is 27760 2023-05-18 18:49:57,965 - INFO - epoch complete! 2023-05-18 18:49:57,965 - INFO - evaluating now! 2023-05-18 18:50:04,481 - INFO - Epoch [41/200] (28098) train_loss: 29.6092, val_loss: 47.5735, lr: 0.000906, 129.51s 2023-05-18 18:50:12,586 - INFO - Saved model at 41 2023-05-18 18:50:12,586 - INFO - Val loss decrease from 64.5896 to 47.5735, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch41.tar 2023-05-18 18:52:15,219 - INFO - epoch complete! 2023-05-18 18:52:15,220 - INFO - evaluating now! 2023-05-18 18:52:21,748 - INFO - Epoch [42/200] (28767) train_loss: 29.6460, val_loss: 46.8572, lr: 0.000901, 129.16s 2023-05-18 18:52:29,538 - INFO - Saved model at 42 2023-05-18 18:52:29,539 - INFO - Val loss decrease from 47.5735 to 46.8572, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch42.tar 2023-05-18 18:54:32,478 - INFO - epoch complete! 2023-05-18 18:54:32,478 - INFO - evaluating now! 2023-05-18 18:54:38,926 - INFO - Epoch [43/200] (29436) train_loss: 29.3261, val_loss: 46.3947, lr: 0.000897, 129.39s 2023-05-18 18:54:46,448 - INFO - Saved model at 43 2023-05-18 18:54:46,449 - INFO - Val loss decrease from 46.8572 to 46.3947, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch43.tar 2023-05-18 18:56:49,277 - INFO - epoch complete! 2023-05-18 18:56:49,277 - INFO - evaluating now! 2023-05-18 18:56:55,788 - INFO - Epoch [44/200] (30105) train_loss: 29.4045, val_loss: 46.8497, lr: 0.000892, 129.34s 2023-05-18 18:58:22,303 - INFO - Training: task_level increase from 11 to 12 2023-05-18 18:58:22,303 - INFO - Current batches_seen is 30536 2023-05-18 18:59:06,037 - INFO - epoch complete! 2023-05-18 18:59:06,037 - INFO - evaluating now! 2023-05-18 18:59:12,556 - INFO - Epoch [45/200] (30774) train_loss: 29.7177, val_loss: 29.8997, lr: 0.000888, 129.19s 2023-05-18 18:59:19,965 - INFO - Saved model at 45 2023-05-18 18:59:19,966 - INFO - Val loss decrease from 46.3947 to 29.8997, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch45.tar 2023-05-18 19:01:23,179 - INFO - epoch complete! 2023-05-18 19:01:23,180 - INFO - evaluating now! 2023-05-18 19:01:29,683 - INFO - Epoch [46/200] (31443) train_loss: 29.4641, val_loss: 29.7221, lr: 0.000883, 129.72s 2023-05-18 19:01:37,648 - INFO - Saved model at 46 2023-05-18 19:01:37,648 - INFO - Val loss decrease from 29.8997 to 29.7221, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch46.tar 2023-05-18 19:03:39,712 - INFO - epoch complete! 2023-05-18 19:03:39,713 - INFO - evaluating now! 2023-05-18 19:03:46,247 - INFO - Epoch [47/200] (32112) train_loss: 29.4425, val_loss: 28.8280, lr: 0.000878, 128.60s 2023-05-18 19:03:53,578 - INFO - Saved model at 47 2023-05-18 19:03:53,578 - INFO - Val loss decrease from 29.7221 to 28.8280, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch47.tar 2023-05-18 19:05:56,544 - INFO - epoch complete! 2023-05-18 19:05:56,544 - INFO - evaluating now! 2023-05-18 19:06:03,074 - INFO - Epoch [48/200] (32781) train_loss: 29.2533, val_loss: 28.7336, lr: 0.000873, 129.50s 2023-05-18 19:06:10,496 - INFO - Saved model at 48 2023-05-18 19:06:10,496 - INFO - Val loss decrease from 28.8280 to 28.7336, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch48.tar 2023-05-18 19:08:13,486 - INFO - epoch complete! 2023-05-18 19:08:13,487 - INFO - evaluating now! 2023-05-18 19:08:20,001 - INFO - Epoch [49/200] (33450) train_loss: 29.1608, val_loss: 29.0241, lr: 0.000868, 129.50s 2023-05-18 19:10:29,447 - INFO - epoch complete! 2023-05-18 19:10:29,447 - INFO - evaluating now! 2023-05-18 19:10:35,964 - INFO - Epoch [50/200] (34119) train_loss: 29.0388, val_loss: 29.3279, lr: 0.000863, 128.63s 2023-05-18 19:12:38,766 - INFO - epoch complete! 2023-05-18 19:12:38,766 - INFO - evaluating now! 2023-05-18 19:12:45,283 - INFO - Epoch [51/200] (34788) train_loss: 28.9366, val_loss: 29.3787, lr: 0.000858, 129.32s 2023-05-18 19:14:54,042 - INFO - epoch complete! 2023-05-18 19:14:54,042 - INFO - evaluating now! 2023-05-18 19:15:00,563 - INFO - Epoch [52/200] (35457) train_loss: 28.7282, val_loss: 28.6031, lr: 0.000853, 129.50s 2023-05-18 19:15:08,610 - INFO - Saved model at 52 2023-05-18 19:15:08,610 - INFO - Val loss decrease from 28.7336 to 28.6031, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch52.tar 2023-05-18 19:17:11,762 - INFO - epoch complete! 2023-05-18 19:17:11,763 - INFO - evaluating now! 2023-05-18 19:17:18,271 - INFO - Epoch [53/200] (36126) train_loss: 28.7094, val_loss: 28.6989, lr: 0.000848, 129.66s 2023-05-18 19:19:21,903 - INFO - epoch complete! 2023-05-18 19:19:21,904 - INFO - evaluating now! 2023-05-18 19:19:28,394 - INFO - Epoch [54/200] (36795) train_loss: 28.6945, val_loss: 28.0642, lr: 0.000842, 130.12s 2023-05-18 19:19:35,608 - INFO - Saved model at 54 2023-05-18 19:19:35,609 - INFO - Val loss decrease from 28.6031 to 28.0642, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch54.tar 2023-05-18 19:21:37,952 - INFO - epoch complete! 2023-05-18 19:21:37,952 - INFO - evaluating now! 2023-05-18 19:21:44,448 - INFO - Epoch [55/200] (37464) train_loss: 28.5155, val_loss: 27.9832, lr: 0.000837, 128.84s 2023-05-18 19:21:44,624 - INFO - Saved model at 55 2023-05-18 19:21:44,625 - INFO - Val loss decrease from 28.0642 to 27.9832, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch55.tar 2023-05-18 19:24:13,496 - INFO - epoch complete! 2023-05-18 19:24:13,496 - INFO - evaluating now! 2023-05-18 19:24:26,258 - INFO - Epoch [56/200] (38133) train_loss: 28.3372, val_loss: 28.2034, lr: 0.000831, 161.63s 2023-05-18 19:28:02,438 - INFO - epoch complete! 2023-05-18 19:28:02,438 - INFO - evaluating now! 2023-05-18 19:28:15,178 - INFO - Epoch [57/200] (38802) train_loss: 28.3186, val_loss: 28.3357, lr: 0.000826, 228.26s 2023-05-18 19:31:50,877 - INFO - epoch complete! 2023-05-18 19:31:50,877 - INFO - evaluating now! 2023-05-18 19:32:03,487 - INFO - Epoch [58/200] (39471) train_loss: 28.2352, val_loss: 28.2243, lr: 0.000820, 228.31s 2023-05-18 19:34:37,104 - INFO - epoch complete! 2023-05-18 19:34:37,105 - INFO - evaluating now! 2023-05-18 19:34:43,582 - INFO - Epoch [59/200] (40140) train_loss: 28.2582, val_loss: 28.1577, lr: 0.000815, 159.69s 2023-05-18 19:36:45,591 - INFO - epoch complete! 2023-05-18 19:36:45,592 - INFO - evaluating now! 2023-05-18 19:36:52,075 - INFO - Epoch [60/200] (40809) train_loss: 27.9724, val_loss: 29.0236, lr: 0.000809, 122.46s 2023-05-18 19:38:57,062 - INFO - epoch complete! 2023-05-18 19:38:57,062 - INFO - evaluating now! 2023-05-18 19:39:03,572 - INFO - Epoch [61/200] (41478) train_loss: 27.9207, val_loss: 27.9473, lr: 0.000803, 123.50s 2023-05-18 19:39:10,972 - INFO - Saved model at 61 2023-05-18 19:39:10,972 - INFO - Val loss decrease from 27.9832 to 27.9473, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch61.tar 2023-05-18 19:41:06,705 - INFO - epoch complete! 2023-05-18 19:41:06,705 - INFO - evaluating now! 2023-05-18 19:41:13,194 - INFO - Epoch [62/200] (42147) train_loss: 27.9276, val_loss: 28.0770, lr: 0.000797, 122.22s 2023-05-18 19:43:15,468 - INFO - epoch complete! 2023-05-18 19:43:15,468 - INFO - evaluating now! 2023-05-18 19:43:21,960 - INFO - Epoch [63/200] (42816) train_loss: 27.9474, val_loss: 28.8320, lr: 0.000791, 122.41s 2023-05-18 19:45:25,790 - INFO - epoch complete! 2023-05-18 19:45:25,790 - INFO - evaluating now! 2023-05-18 19:45:32,300 - INFO - Epoch [64/200] (43485) train_loss: 27.8665, val_loss: 28.5269, lr: 0.000785, 122.38s 2023-05-18 19:47:35,934 - INFO - epoch complete! 2023-05-18 19:47:35,934 - INFO - evaluating now! 2023-05-18 19:47:42,381 - INFO - Epoch [65/200] (44154) train_loss: 27.7752, val_loss: 28.5201, lr: 0.000779, 122.41s 2023-05-18 19:49:45,450 - INFO - epoch complete! 2023-05-18 19:49:45,450 - INFO - evaluating now! 2023-05-18 19:49:51,939 - INFO - Epoch [66/200] (44823) train_loss: 27.6369, val_loss: 27.6519, lr: 0.000773, 121.93s 2023-05-18 19:49:59,904 - INFO - Saved model at 66 2023-05-18 19:49:59,904 - INFO - Val loss decrease from 27.9473 to 27.6519, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch66.tar 2023-05-18 19:52:00,610 - INFO - epoch complete! 2023-05-18 19:52:00,610 - INFO - evaluating now! 2023-05-18 19:52:07,084 - INFO - Epoch [67/200] (45492) train_loss: 27.5542, val_loss: 27.5029, lr: 0.000767, 127.18s 2023-05-18 19:52:13,906 - INFO - Saved model at 67 2023-05-18 19:52:13,906 - INFO - Val loss decrease from 27.6519 to 27.5029, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch67.tar 2023-05-18 19:54:21,399 - INFO - epoch complete! 2023-05-18 19:54:21,399 - INFO - evaluating now! 2023-05-18 19:54:27,895 - INFO - Epoch [68/200] (46161) train_loss: 27.5131, val_loss: 27.5721, lr: 0.000761, 133.99s 2023-05-18 19:56:39,452 - INFO - epoch complete! 2023-05-18 19:56:39,453 - INFO - evaluating now! 2023-05-18 19:56:45,957 - INFO - Epoch [69/200] (46830) train_loss: 27.3696, val_loss: 27.1869, lr: 0.000754, 130.16s 2023-05-18 19:56:48,934 - INFO - Saved model at 69 2023-05-18 19:56:48,934 - INFO - Val loss decrease from 27.5029 to 27.1869, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch69.tar 2023-05-18 19:58:46,660 - INFO - epoch complete! 2023-05-18 19:58:46,661 - INFO - evaluating now! 2023-05-18 19:58:53,146 - INFO - Epoch [70/200] (47499) train_loss: 27.2898, val_loss: 27.3964, lr: 0.000748, 124.21s 2023-05-18 20:00:57,524 - INFO - epoch complete! 2023-05-18 20:00:57,525 - INFO - evaluating now! 2023-05-18 20:01:04,034 - INFO - Epoch [71/200] (48168) train_loss: 27.2266, val_loss: 27.2394, lr: 0.000742, 123.06s 2023-05-18 20:03:08,267 - INFO - epoch complete! 2023-05-18 20:03:08,267 - INFO - evaluating now! 2023-05-18 20:03:14,723 - INFO - Epoch [72/200] (48837) train_loss: 27.1856, val_loss: 27.3881, lr: 0.000735, 122.87s 2023-05-18 20:05:19,728 - INFO - epoch complete! 2023-05-18 20:05:19,729 - INFO - evaluating now! 2023-05-18 20:05:26,224 - INFO - Epoch [73/200] (49506) train_loss: 27.0089, val_loss: 27.7995, lr: 0.000729, 123.37s 2023-05-18 20:07:30,513 - INFO - epoch complete! 2023-05-18 20:07:30,513 - INFO - evaluating now! 2023-05-18 20:07:37,006 - INFO - Epoch [74/200] (50175) train_loss: 27.0379, val_loss: 27.3382, lr: 0.000722, 123.03s 2023-05-18 20:09:40,718 - INFO - epoch complete! 2023-05-18 20:09:40,718 - INFO - evaluating now! 2023-05-18 20:09:47,256 - INFO - Epoch [75/200] (50844) train_loss: 27.0188, val_loss: 27.2520, lr: 0.000716, 122.44s 2023-05-18 20:11:43,917 - INFO - epoch complete! 2023-05-18 20:11:43,917 - INFO - evaluating now! 2023-05-18 20:11:50,379 - INFO - Epoch [76/200] (51513) train_loss: 26.8299, val_loss: 27.1550, lr: 0.000709, 123.12s 2023-05-18 20:11:57,435 - INFO - Saved model at 76 2023-05-18 20:11:57,436 - INFO - Val loss decrease from 27.1869 to 27.1550, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch76.tar 2023-05-18 20:13:53,889 - INFO - epoch complete! 2023-05-18 20:13:53,889 - INFO - evaluating now! 2023-05-18 20:14:00,381 - INFO - Epoch [77/200] (52182) train_loss: 26.7814, val_loss: 27.3376, lr: 0.000702, 122.95s 2023-05-18 20:16:04,502 - INFO - epoch complete! 2023-05-18 20:16:04,502 - INFO - evaluating now! 2023-05-18 20:16:10,994 - INFO - Epoch [78/200] (52851) train_loss: 26.8454, val_loss: 27.0744, lr: 0.000696, 122.99s 2023-05-18 20:16:18,725 - INFO - Saved model at 78 2023-05-18 20:16:18,725 - INFO - Val loss decrease from 27.1550 to 27.0744, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch78.tar 2023-05-18 20:18:16,306 - INFO - epoch complete! 2023-05-18 20:18:16,306 - INFO - evaluating now! 2023-05-18 20:18:22,754 - INFO - Epoch [79/200] (53520) train_loss: 26.6146, val_loss: 26.6362, lr: 0.000689, 124.03s 2023-05-18 20:18:30,997 - INFO - Saved model at 79 2023-05-18 20:18:30,998 - INFO - Val loss decrease from 27.0744 to 26.6362, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch79.tar 2023-05-18 20:20:28,217 - INFO - epoch complete! 2023-05-18 20:20:28,217 - INFO - evaluating now! 2023-05-18 20:20:34,686 - INFO - Epoch [80/200] (54189) train_loss: 26.6399, val_loss: 27.2145, lr: 0.000682, 123.69s 2023-05-18 20:22:39,002 - INFO - epoch complete! 2023-05-18 20:22:39,002 - INFO - evaluating now! 2023-05-18 20:22:45,476 - INFO - Epoch [81/200] (54858) train_loss: 26.6846, val_loss: 26.9573, lr: 0.000676, 123.53s 2023-05-18 20:24:50,066 - INFO - epoch complete! 2023-05-18 20:24:50,066 - INFO - evaluating now! 2023-05-18 20:24:56,535 - INFO - Epoch [82/200] (55527) train_loss: 26.5716, val_loss: 27.0624, lr: 0.000669, 123.76s 2023-05-18 20:27:00,529 - INFO - epoch complete! 2023-05-18 20:27:00,529 - INFO - evaluating now! 2023-05-18 20:27:07,026 - INFO - Epoch [83/200] (56196) train_loss: 26.5406, val_loss: 26.7956, lr: 0.000662, 123.16s 2023-05-18 20:29:12,844 - INFO - epoch complete! 2023-05-18 20:29:12,845 - INFO - evaluating now! 2023-05-18 20:29:19,332 - INFO - Epoch [84/200] (56865) train_loss: 26.4666, val_loss: 27.3208, lr: 0.000655, 124.36s 2023-05-18 20:31:23,754 - INFO - epoch complete! 2023-05-18 20:31:23,755 - INFO - evaluating now! 2023-05-18 20:31:30,249 - INFO - Epoch [85/200] (57534) train_loss: 26.3920, val_loss: 26.7446, lr: 0.000648, 123.43s 2023-05-18 20:33:33,682 - INFO - epoch complete! 2023-05-18 20:33:33,682 - INFO - evaluating now! 2023-05-18 20:33:40,157 - INFO - Epoch [86/200] (58203) train_loss: 26.3767, val_loss: 27.0418, lr: 0.000641, 122.55s 2023-05-18 20:35:36,253 - INFO - epoch complete! 2023-05-18 20:35:36,254 - INFO - evaluating now! 2023-05-18 20:35:42,748 - INFO - Epoch [87/200] (58872) train_loss: 26.3097, val_loss: 26.7135, lr: 0.000634, 122.59s 2023-05-18 20:37:46,429 - INFO - epoch complete! 2023-05-18 20:37:46,429 - INFO - evaluating now! 2023-05-18 20:37:52,922 - INFO - Epoch [88/200] (59541) train_loss: 26.2918, val_loss: 26.4553, lr: 0.000627, 122.40s 2023-05-18 20:38:00,646 - INFO - Saved model at 88 2023-05-18 20:38:00,646 - INFO - Val loss decrease from 26.6362 to 26.4553, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch88.tar 2023-05-18 20:39:56,259 - INFO - epoch complete! 2023-05-18 20:39:56,259 - INFO - evaluating now! 2023-05-18 20:40:02,757 - INFO - Epoch [89/200] (60210) train_loss: 26.1616, val_loss: 26.5203, lr: 0.000620, 122.11s 2023-05-18 20:42:09,693 - INFO - epoch complete! 2023-05-18 20:42:09,693 - INFO - evaluating now! 2023-05-18 20:42:16,176 - INFO - Epoch [90/200] (60879) train_loss: 26.0847, val_loss: 26.4594, lr: 0.000613, 125.53s 2023-05-18 20:44:31,341 - INFO - epoch complete! 2023-05-18 20:44:31,342 - INFO - evaluating now! 2023-05-18 20:44:37,831 - INFO - Epoch [91/200] (61548) train_loss: 26.0539, val_loss: 26.7182, lr: 0.000606, 134.05s 2023-05-18 20:46:53,032 - INFO - epoch complete! 2023-05-18 20:46:53,032 - INFO - evaluating now! 2023-05-18 20:46:59,535 - INFO - Epoch [92/200] (62217) train_loss: 26.0279, val_loss: 26.8633, lr: 0.000599, 133.96s 2023-05-18 20:49:14,226 - INFO - epoch complete! 2023-05-18 20:49:14,226 - INFO - evaluating now! 2023-05-18 20:49:20,714 - INFO - Epoch [93/200] (62886) train_loss: 25.9613, val_loss: 27.5453, lr: 0.000592, 133.07s 2023-05-18 20:51:36,028 - INFO - epoch complete! 2023-05-18 20:51:36,028 - INFO - evaluating now! 2023-05-18 20:51:42,516 - INFO - Epoch [94/200] (63555) train_loss: 25.9569, val_loss: 27.0839, lr: 0.000585, 134.06s 2023-05-18 20:53:58,135 - INFO - epoch complete! 2023-05-18 20:53:58,135 - INFO - evaluating now! 2023-05-18 20:54:04,639 - INFO - Epoch [95/200] (64224) train_loss: 25.8947, val_loss: 27.2637, lr: 0.000578, 134.08s 2023-05-18 20:56:12,422 - INFO - epoch complete! 2023-05-18 20:56:12,423 - INFO - evaluating now! 2023-05-18 20:56:18,921 - INFO - Epoch [96/200] (64893) train_loss: 25.9560, val_loss: 26.5491, lr: 0.000571, 126.28s 2023-05-18 20:58:15,923 - INFO - epoch complete! 2023-05-18 20:58:15,923 - INFO - evaluating now! 2023-05-18 20:58:22,421 - INFO - Epoch [97/200] (65562) train_loss: 25.9200, val_loss: 26.4634, lr: 0.000564, 122.07s 2023-05-18 21:00:28,144 - INFO - epoch complete! 2023-05-18 21:00:28,145 - INFO - evaluating now! 2023-05-18 21:00:34,639 - INFO - Epoch [98/200] (66231) train_loss: 25.8446, val_loss: 26.9649, lr: 0.000557, 124.73s 2023-05-18 21:02:40,811 - INFO - epoch complete! 2023-05-18 21:02:40,812 - INFO - evaluating now! 2023-05-18 21:02:47,324 - INFO - Epoch [99/200] (66900) train_loss: 25.7224, val_loss: 26.4782, lr: 0.000550, 125.34s 2023-05-18 21:04:53,094 - INFO - epoch complete! 2023-05-18 21:04:53,095 - INFO - evaluating now! 2023-05-18 21:04:59,596 - INFO - Epoch [100/200] (67569) train_loss: 25.7055, val_loss: 26.3822, lr: 0.000543, 124.77s 2023-05-18 21:05:07,012 - INFO - Saved model at 100 2023-05-18 21:05:07,013 - INFO - Val loss decrease from 26.4553 to 26.3822, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch100.tar 2023-05-18 21:07:06,292 - INFO - epoch complete! 2023-05-18 21:07:06,292 - INFO - evaluating now! 2023-05-18 21:07:12,788 - INFO - Epoch [101/200] (68238) train_loss: 25.6545, val_loss: 27.7576, lr: 0.000536, 125.78s 2023-05-18 21:09:19,535 - INFO - epoch complete! 2023-05-18 21:09:19,535 - INFO - evaluating now! 2023-05-18 21:09:26,047 - INFO - Epoch [102/200] (68907) train_loss: 25.6738, val_loss: 26.2734, lr: 0.000529, 125.20s 2023-05-18 21:09:34,037 - INFO - Saved model at 102 2023-05-18 21:09:34,037 - INFO - Val loss decrease from 26.3822 to 26.2734, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch102.tar 2023-05-18 21:11:31,651 - INFO - epoch complete! 2023-05-18 21:11:31,652 - INFO - evaluating now! 2023-05-18 21:11:38,146 - INFO - Epoch [103/200] (69576) train_loss: 25.6535, val_loss: 26.4370, lr: 0.000522, 124.11s 2023-05-18 21:13:43,341 - INFO - epoch complete! 2023-05-18 21:13:43,342 - INFO - evaluating now! 2023-05-18 21:13:49,839 - INFO - Epoch [104/200] (70245) train_loss: 25.5455, val_loss: 26.5089, lr: 0.000515, 125.20s 2023-05-18 21:15:56,878 - INFO - epoch complete! 2023-05-18 21:15:56,878 - INFO - evaluating now! 2023-05-18 21:16:03,394 - INFO - Epoch [105/200] (70914) train_loss: 25.4752, val_loss: 26.3768, lr: 0.000508, 125.56s 2023-05-18 21:18:09,686 - INFO - epoch complete! 2023-05-18 21:18:09,686 - INFO - evaluating now! 2023-05-18 21:18:16,122 - INFO - Epoch [106/200] (71583) train_loss: 25.3940, val_loss: 26.4809, lr: 0.000501, 125.41s 2023-05-18 21:20:22,786 - INFO - epoch complete! 2023-05-18 21:20:22,787 - INFO - evaluating now! 2023-05-18 21:20:29,296 - INFO - Epoch [107/200] (72252) train_loss: 25.3908, val_loss: 26.3609, lr: 0.000494, 125.40s 2023-05-18 21:22:35,716 - INFO - epoch complete! 2023-05-18 21:22:35,716 - INFO - evaluating now! 2023-05-18 21:22:42,220 - INFO - Epoch [108/200] (72921) train_loss: 25.4090, val_loss: 26.3733, lr: 0.000487, 125.49s 2023-05-18 21:24:49,180 - INFO - epoch complete! 2023-05-18 21:24:49,181 - INFO - evaluating now! 2023-05-18 21:24:55,690 - INFO - Epoch [109/200] (73590) train_loss: 25.3574, val_loss: 26.8599, lr: 0.000480, 125.83s 2023-05-18 21:27:01,231 - INFO - epoch complete! 2023-05-18 21:27:01,231 - INFO - evaluating now! 2023-05-18 21:27:07,732 - INFO - Epoch [110/200] (74259) train_loss: 25.2854, val_loss: 26.2657, lr: 0.000473, 124.74s 2023-05-18 21:27:15,519 - INFO - Saved model at 110 2023-05-18 21:27:15,519 - INFO - Val loss decrease from 26.2734 to 26.2657, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch110.tar 2023-05-18 21:29:14,212 - INFO - epoch complete! 2023-05-18 21:29:14,212 - INFO - evaluating now! 2023-05-18 21:29:20,720 - INFO - Epoch [111/200] (74928) train_loss: 25.2460, val_loss: 26.2048, lr: 0.000466, 125.20s 2023-05-18 21:29:28,002 - INFO - Saved model at 111 2023-05-18 21:29:28,002 - INFO - Val loss decrease from 26.2657 to 26.2048, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch111.tar 2023-05-18 21:31:26,683 - INFO - epoch complete! 2023-05-18 21:31:26,683 - INFO - evaluating now! 2023-05-18 21:31:33,193 - INFO - Epoch [112/200] (75597) train_loss: 25.1974, val_loss: 26.5378, lr: 0.000459, 125.19s 2023-05-18 21:33:39,268 - INFO - epoch complete! 2023-05-18 21:33:39,268 - INFO - evaluating now! 2023-05-18 21:33:45,711 - INFO - Epoch [113/200] (76266) train_loss: 25.2543, val_loss: 25.9488, lr: 0.000452, 124.90s 2023-05-18 21:33:54,067 - INFO - Saved model at 113 2023-05-18 21:33:54,068 - INFO - Val loss decrease from 26.2048 to 25.9488, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch113.tar 2023-05-18 21:35:53,062 - INFO - epoch complete! 2023-05-18 21:35:53,062 - INFO - evaluating now! 2023-05-18 21:35:59,549 - INFO - Epoch [114/200] (76935) train_loss: 25.2304, val_loss: 26.8456, lr: 0.000445, 125.48s 2023-05-18 21:38:04,520 - INFO - epoch complete! 2023-05-18 21:38:04,520 - INFO - evaluating now! 2023-05-18 21:38:11,032 - INFO - Epoch [115/200] (77604) train_loss: 25.1433, val_loss: 26.0422, lr: 0.000438, 124.27s 2023-05-18 21:40:15,948 - INFO - epoch complete! 2023-05-18 21:40:15,948 - INFO - evaluating now! 2023-05-18 21:40:22,459 - INFO - Epoch [116/200] (78273) train_loss: 25.1426, val_loss: 26.0609, lr: 0.000431, 123.87s 2023-05-18 21:42:32,192 - INFO - epoch complete! 2023-05-18 21:42:32,193 - INFO - evaluating now! 2023-05-18 21:42:38,687 - INFO - Epoch [117/200] (78942) train_loss: 25.0847, val_loss: 26.2325, lr: 0.000424, 129.00s 2023-05-18 21:44:54,565 - INFO - epoch complete! 2023-05-18 21:44:54,565 - INFO - evaluating now! 2023-05-18 21:45:01,067 - INFO - Epoch [118/200] (79611) train_loss: 25.0379, val_loss: 25.9974, lr: 0.000418, 134.40s 2023-05-18 21:47:10,206 - INFO - epoch complete! 2023-05-18 21:47:10,206 - INFO - evaluating now! 2023-05-18 21:47:16,734 - INFO - Epoch [119/200] (80280) train_loss: 25.0094, val_loss: 26.0190, lr: 0.000411, 128.06s 2023-05-18 21:49:21,359 - INFO - epoch complete! 2023-05-18 21:49:21,360 - INFO - evaluating now! 2023-05-18 21:49:27,876 - INFO - Epoch [120/200] (80949) train_loss: 24.9617, val_loss: 26.5139, lr: 0.000404, 123.96s 2023-05-18 21:51:31,962 - INFO - epoch complete! 2023-05-18 21:51:31,962 - INFO - evaluating now! 2023-05-18 21:51:38,462 - INFO - Epoch [121/200] (81618) train_loss: 24.8938, val_loss: 26.5887, lr: 0.000398, 126.28s 2023-05-18 21:53:43,435 - INFO - epoch complete! 2023-05-18 21:53:43,436 - INFO - evaluating now! 2023-05-18 21:53:49,944 - INFO - Epoch [122/200] (82287) train_loss: 24.9367, val_loss: 25.9748, lr: 0.000391, 124.09s 2023-05-18 21:55:55,025 - INFO - epoch complete! 2023-05-18 21:55:55,025 - INFO - evaluating now! 2023-05-18 21:56:01,476 - INFO - Epoch [123/200] (82956) train_loss: 24.8575, val_loss: 26.0674, lr: 0.000384, 123.62s 2023-05-18 21:58:05,881 - INFO - epoch complete! 2023-05-18 21:58:05,881 - INFO - evaluating now! 2023-05-18 21:58:12,378 - INFO - Epoch [124/200] (83625) train_loss: 24.8442, val_loss: 26.0170, lr: 0.000378, 123.38s 2023-05-18 22:00:17,343 - INFO - epoch complete! 2023-05-18 22:00:17,343 - INFO - evaluating now! 2023-05-18 22:00:23,844 - INFO - Epoch [125/200] (84294) train_loss: 24.8164, val_loss: 26.0439, lr: 0.000371, 124.08s 2023-05-18 22:02:30,916 - INFO - epoch complete! 2023-05-18 22:02:30,917 - INFO - evaluating now! 2023-05-18 22:02:37,422 - INFO - Epoch [126/200] (84963) train_loss: 24.7793, val_loss: 26.0075, lr: 0.000365, 125.56s 2023-05-18 22:04:43,591 - INFO - epoch complete! 2023-05-18 22:04:43,591 - INFO - evaluating now! 2023-05-18 22:04:50,077 - INFO - Epoch [127/200] (85632) train_loss: 24.7612, val_loss: 26.1055, lr: 0.000358, 125.37s 2023-05-18 22:06:54,743 - INFO - epoch complete! 2023-05-18 22:06:54,743 - INFO - evaluating now! 2023-05-18 22:07:01,239 - INFO - Epoch [128/200] (86301) train_loss: 24.7516, val_loss: 26.0758, lr: 0.000352, 124.12s 2023-05-18 22:09:06,539 - INFO - epoch complete! 2023-05-18 22:09:06,540 - INFO - evaluating now! 2023-05-18 22:09:13,071 - INFO - Epoch [129/200] (86970) train_loss: 24.6933, val_loss: 26.0670, lr: 0.000346, 124.24s 2023-05-18 22:11:18,392 - INFO - epoch complete! 2023-05-18 22:11:18,392 - INFO - evaluating now! 2023-05-18 22:11:24,852 - INFO - Epoch [130/200] (87639) train_loss: 24.7349, val_loss: 25.9908, lr: 0.000339, 124.05s 2023-05-18 22:13:30,889 - INFO - epoch complete! 2023-05-18 22:13:30,889 - INFO - evaluating now! 2023-05-18 22:13:37,411 - INFO - Epoch [131/200] (88308) train_loss: 24.6569, val_loss: 25.9608, lr: 0.000333, 124.54s 2023-05-18 22:15:40,981 - INFO - epoch complete! 2023-05-18 22:15:40,981 - INFO - evaluating now! 2023-05-18 22:15:47,479 - INFO - Epoch [132/200] (88977) train_loss: 24.6440, val_loss: 25.8684, lr: 0.000327, 122.78s 2023-05-18 22:15:55,639 - INFO - Saved model at 132 2023-05-18 22:15:55,640 - INFO - Val loss decrease from 25.9488 to 25.8684, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch132.tar 2023-05-18 22:17:51,515 - INFO - epoch complete! 2023-05-18 22:17:51,516 - INFO - evaluating now! 2023-05-18 22:17:58,025 - INFO - Epoch [133/200] (89646) train_loss: 24.5964, val_loss: 26.0186, lr: 0.000321, 122.38s 2023-05-18 22:20:00,823 - INFO - epoch complete! 2023-05-18 22:20:00,824 - INFO - evaluating now! 2023-05-18 22:20:07,304 - INFO - Epoch [134/200] (90315) train_loss: 24.5867, val_loss: 26.1364, lr: 0.000315, 121.76s 2023-05-18 22:22:10,927 - INFO - epoch complete! 2023-05-18 22:22:10,927 - INFO - evaluating now! 2023-05-18 22:22:17,442 - INFO - Epoch [135/200] (90984) train_loss: 24.5534, val_loss: 26.0429, lr: 0.000309, 122.81s 2023-05-18 22:24:21,046 - INFO - epoch complete! 2023-05-18 22:24:21,046 - INFO - evaluating now! 2023-05-18 22:24:27,550 - INFO - Epoch [136/200] (91653) train_loss: 24.5066, val_loss: 25.9355, lr: 0.000303, 122.87s 2023-05-18 22:26:31,142 - INFO - epoch complete! 2023-05-18 22:26:31,142 - INFO - evaluating now! 2023-05-18 22:26:37,635 - INFO - Epoch [137/200] (92322) train_loss: 24.4733, val_loss: 25.9090, lr: 0.000297, 122.13s 2023-05-18 22:28:40,697 - INFO - epoch complete! 2023-05-18 22:28:40,698 - INFO - evaluating now! 2023-05-18 22:28:47,210 - INFO - Epoch [138/200] (92991) train_loss: 24.4231, val_loss: 25.7759, lr: 0.000291, 122.52s 2023-05-18 22:28:54,548 - INFO - Saved model at 138 2023-05-18 22:28:54,548 - INFO - Val loss decrease from 25.8684 to 25.7759, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch138.tar 2023-05-18 22:30:50,737 - INFO - epoch complete! 2023-05-18 22:30:50,738 - INFO - evaluating now! 2023-05-18 22:30:57,235 - INFO - Epoch [139/200] (93660) train_loss: 24.4376, val_loss: 25.9484, lr: 0.000285, 122.69s 2023-05-18 22:33:01,017 - INFO - epoch complete! 2023-05-18 22:33:01,018 - INFO - evaluating now! 2023-05-18 22:33:07,536 - INFO - Epoch [140/200] (94329) train_loss: 24.3929, val_loss: 26.4034, lr: 0.000280, 122.92s 2023-05-18 22:35:22,432 - INFO - epoch complete! 2023-05-18 22:35:22,433 - INFO - evaluating now! 2023-05-18 22:35:28,968 - INFO - Epoch [141/200] (94998) train_loss: 24.4274, val_loss: 26.1211, lr: 0.000274, 133.34s 2023-05-18 22:37:45,027 - INFO - epoch complete! 2023-05-18 22:37:45,027 - INFO - evaluating now! 2023-05-18 22:37:51,542 - INFO - Epoch [142/200] (95667) train_loss: 24.3795, val_loss: 25.8665, lr: 0.000269, 134.58s 2023-05-18 22:40:07,144 - INFO - epoch complete! 2023-05-18 22:40:07,145 - INFO - evaluating now! 2023-05-18 22:40:13,673 - INFO - Epoch [143/200] (96336) train_loss: 24.3613, val_loss: 25.9292, lr: 0.000263, 134.63s 2023-05-18 22:42:28,079 - INFO - epoch complete! 2023-05-18 22:42:28,080 - INFO - evaluating now! 2023-05-18 22:42:34,585 - INFO - Epoch [144/200] (97005) train_loss: 24.3367, val_loss: 25.8588, lr: 0.000258, 133.75s 2023-05-18 22:44:49,430 - INFO - epoch complete! 2023-05-18 22:44:49,431 - INFO - evaluating now! 2023-05-18 22:44:55,946 - INFO - Epoch [145/200] (97674) train_loss: 24.2705, val_loss: 26.1762, lr: 0.000252, 134.59s 2023-05-18 22:47:10,892 - INFO - epoch complete! 2023-05-18 22:47:10,893 - INFO - evaluating now! 2023-05-18 22:47:17,422 - INFO - Epoch [146/200] (98343) train_loss: 24.2590, val_loss: 25.8975, lr: 0.000247, 133.42s 2023-05-18 22:49:23,160 - INFO - epoch complete! 2023-05-18 22:49:23,161 - INFO - evaluating now! 2023-05-18 22:49:29,676 - INFO - Epoch [147/200] (99012) train_loss: 24.2406, val_loss: 25.7713, lr: 0.000242, 124.58s 2023-05-18 22:49:29,708 - INFO - Saved model at 147 2023-05-18 22:49:29,709 - INFO - Val loss decrease from 25.7759 to 25.7713, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch147.tar 2023-05-18 22:51:27,981 - INFO - epoch complete! 2023-05-18 22:51:27,981 - INFO - evaluating now! 2023-05-18 22:51:34,471 - INFO - Epoch [148/200] (99681) train_loss: 24.2180, val_loss: 25.8702, lr: 0.000237, 124.76s 2023-05-18 22:53:39,838 - INFO - epoch complete! 2023-05-18 22:53:39,838 - INFO - evaluating now! 2023-05-18 22:53:46,335 - INFO - Epoch [149/200] (100350) train_loss: 24.2123, val_loss: 26.1271, lr: 0.000232, 124.49s 2023-05-18 22:55:52,216 - INFO - epoch complete! 2023-05-18 22:55:52,217 - INFO - evaluating now! 2023-05-18 22:55:58,736 - INFO - Epoch [150/200] (101019) train_loss: 24.1954, val_loss: 25.8835, lr: 0.000227, 124.82s 2023-05-18 22:58:04,500 - INFO - epoch complete! 2023-05-18 22:58:04,501 - INFO - evaluating now! 2023-05-18 22:58:10,989 - INFO - Epoch [151/200] (101688) train_loss: 24.1559, val_loss: 25.9077, lr: 0.000222, 125.00s 2023-05-18 23:00:17,592 - INFO - epoch complete! 2023-05-18 23:00:17,593 - INFO - evaluating now! 2023-05-18 23:00:24,098 - INFO - Epoch [152/200] (102357) train_loss: 24.1083, val_loss: 25.8235, lr: 0.000217, 125.22s 2023-05-18 23:02:31,429 - INFO - epoch complete! 2023-05-18 23:02:31,429 - INFO - evaluating now! 2023-05-18 23:02:37,934 - INFO - Epoch [153/200] (103026) train_loss: 24.1403, val_loss: 25.7456, lr: 0.000212, 125.90s 2023-05-18 23:02:45,976 - INFO - Saved model at 153 2023-05-18 23:02:45,976 - INFO - Val loss decrease from 25.7713 to 25.7456, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch153.tar 2023-05-18 23:04:44,637 - INFO - epoch complete! 2023-05-18 23:04:44,637 - INFO - evaluating now! 2023-05-18 23:04:51,164 - INFO - Epoch [154/200] (103695) train_loss: 24.1058, val_loss: 25.7303, lr: 0.000208, 125.19s 2023-05-18 23:04:57,813 - INFO - Saved model at 154 2023-05-18 23:04:57,813 - INFO - Val loss decrease from 25.7456 to 25.7303, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch154.tar 2023-05-18 23:06:56,428 - INFO - epoch complete! 2023-05-18 23:06:56,429 - INFO - evaluating now! 2023-05-18 23:07:02,933 - INFO - Epoch [155/200] (104364) train_loss: 24.0618, val_loss: 25.6400, lr: 0.000203, 125.12s 2023-05-18 23:07:10,945 - INFO - Saved model at 155 2023-05-18 23:07:10,945 - INFO - Val loss decrease from 25.7303 to 25.6400, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch155.tar 2023-05-18 23:09:07,871 - INFO - epoch complete! 2023-05-18 23:09:07,871 - INFO - evaluating now! 2023-05-18 23:09:14,359 - INFO - Epoch [156/200] (105033) train_loss: 24.0799, val_loss: 25.7588, lr: 0.000199, 123.41s 2023-05-18 23:11:17,917 - INFO - epoch complete! 2023-05-18 23:11:17,917 - INFO - evaluating now! 2023-05-18 23:11:24,359 - INFO - Epoch [157/200] (105702) train_loss: 24.0136, val_loss: 25.8863, lr: 0.000194, 122.40s 2023-05-18 23:13:27,919 - INFO - epoch complete! 2023-05-18 23:13:27,920 - INFO - evaluating now! 2023-05-18 23:13:34,431 - INFO - Epoch [158/200] (106371) train_loss: 24.0029, val_loss: 25.7054, lr: 0.000190, 122.44s 2023-05-18 23:15:37,977 - INFO - epoch complete! 2023-05-18 23:15:37,977 - INFO - evaluating now! 2023-05-18 23:15:44,465 - INFO - Epoch [159/200] (107040) train_loss: 24.0139, val_loss: 25.8116, lr: 0.000186, 122.43s 2023-05-18 23:17:47,960 - INFO - epoch complete! 2023-05-18 23:17:47,960 - INFO - evaluating now! 2023-05-18 23:17:54,463 - INFO - Epoch [160/200] (107709) train_loss: 23.9933, val_loss: 25.8728, lr: 0.000182, 122.68s 2023-05-18 23:19:57,265 - INFO - epoch complete! 2023-05-18 23:19:57,266 - INFO - evaluating now! 2023-05-18 23:20:03,741 - INFO - Epoch [161/200] (108378) train_loss: 23.9616, val_loss: 25.7841, lr: 0.000178, 121.90s 2023-05-18 23:22:05,821 - INFO - epoch complete! 2023-05-18 23:22:05,821 - INFO - evaluating now! 2023-05-18 23:22:12,326 - INFO - Epoch [162/200] (109047) train_loss: 23.9515, val_loss: 25.7350, lr: 0.000174, 122.45s 2023-05-18 23:24:15,835 - INFO - epoch complete! 2023-05-18 23:24:15,836 - INFO - evaluating now! 2023-05-18 23:24:22,343 - INFO - Epoch [163/200] (109716) train_loss: 23.9551, val_loss: 25.7131, lr: 0.000170, 122.43s 2023-05-18 23:26:26,328 - INFO - epoch complete! 2023-05-18 23:26:26,328 - INFO - evaluating now! 2023-05-18 23:26:32,776 - INFO - Epoch [164/200] (110385) train_loss: 23.9088, val_loss: 25.7826, lr: 0.000166, 122.81s 2023-05-18 23:28:37,145 - INFO - epoch complete! 2023-05-18 23:28:37,146 - INFO - evaluating now! 2023-05-18 23:28:43,646 - INFO - Epoch [165/200] (111054) train_loss: 23.9178, val_loss: 25.7174, lr: 0.000163, 122.62s 2023-05-18 23:30:47,985 - INFO - epoch complete! 2023-05-18 23:30:47,985 - INFO - evaluating now! 2023-05-18 23:30:54,496 - INFO - Epoch [166/200] (111723) train_loss: 23.8599, val_loss: 25.8035, lr: 0.000159, 123.69s 2023-05-18 23:32:58,545 - INFO - epoch complete! 2023-05-18 23:32:58,546 - INFO - evaluating now! 2023-05-18 23:33:05,058 - INFO - Epoch [167/200] (112392) train_loss: 23.8515, val_loss: 25.7809, lr: 0.000156, 123.14s 2023-05-18 23:35:08,756 - INFO - epoch complete! 2023-05-18 23:35:08,756 - INFO - evaluating now! 2023-05-18 23:35:15,236 - INFO - Epoch [168/200] (113061) train_loss: 23.8762, val_loss: 25.7283, lr: 0.000152, 122.28s 2023-05-18 23:37:18,264 - INFO - epoch complete! 2023-05-18 23:37:18,264 - INFO - evaluating now! 2023-05-18 23:37:24,772 - INFO - Epoch [169/200] (113730) train_loss: 23.8694, val_loss: 25.7519, lr: 0.000149, 122.60s 2023-05-18 23:39:28,661 - INFO - epoch complete! 2023-05-18 23:39:28,661 - INFO - evaluating now! 2023-05-18 23:39:35,167 - INFO - Epoch [170/200] (114399) train_loss: 23.8603, val_loss: 25.7121, lr: 0.000146, 123.11s 2023-05-18 23:41:39,192 - INFO - epoch complete! 2023-05-18 23:41:39,193 - INFO - evaluating now! 2023-05-18 23:41:45,650 - INFO - Epoch [171/200] (115068) train_loss: 23.8086, val_loss: 25.7424, lr: 0.000143, 122.81s 2023-05-18 23:43:49,471 - INFO - epoch complete! 2023-05-18 23:43:49,471 - INFO - evaluating now! 2023-05-18 23:43:55,957 - INFO - Epoch [172/200] (115737) train_loss: 23.7889, val_loss: 25.7843, lr: 0.000140, 122.99s 2023-05-18 23:46:00,435 - INFO - epoch complete! 2023-05-18 23:46:00,436 - INFO - evaluating now! 2023-05-18 23:46:06,943 - INFO - Epoch [173/200] (116406) train_loss: 23.7816, val_loss: 25.7672, lr: 0.000137, 122.95s 2023-05-18 23:48:11,224 - INFO - epoch complete! 2023-05-18 23:48:11,225 - INFO - evaluating now! 2023-05-18 23:48:17,731 - INFO - Epoch [174/200] (117075) train_loss: 23.7650, val_loss: 25.7364, lr: 0.000134, 122.98s 2023-05-18 23:50:22,328 - INFO - epoch complete! 2023-05-18 23:50:22,329 - INFO - evaluating now! 2023-05-18 23:50:28,824 - INFO - Epoch [175/200] (117744) train_loss: 23.7702, val_loss: 25.7401, lr: 0.000132, 124.03s 2023-05-18 23:52:32,169 - INFO - epoch complete! 2023-05-18 23:52:32,170 - INFO - evaluating now! 2023-05-18 23:52:38,676 - INFO - Epoch [176/200] (118413) train_loss: 23.7646, val_loss: 25.6338, lr: 0.000129, 122.23s 2023-05-18 23:52:46,816 - INFO - Saved model at 176 2023-05-18 23:52:46,816 - INFO - Val loss decrease from 25.6400 to 25.6338, saving to ./libcity/cache/92489/model_cache/PDFormer_PeMS08_epoch176.tar 2023-05-18 23:54:41,806 - INFO - epoch complete! 2023-05-18 23:54:41,806 - INFO - evaluating now! 2023-05-18 23:54:48,322 - INFO - Epoch [177/200] (119082) train_loss: 23.7292, val_loss: 25.6880, lr: 0.000127, 121.51s 2023-05-18 23:56:50,663 - INFO - epoch complete! 2023-05-18 23:56:50,663 - INFO - evaluating now! 2023-05-18 23:56:57,111 - INFO - Epoch [178/200] (119751) train_loss: 23.7316, val_loss: 25.7614, lr: 0.000124, 121.30s 2023-05-18 23:58:59,750 - INFO - epoch complete! 2023-05-18 23:58:59,751 - INFO - evaluating now! 2023-05-18 23:59:06,259 - INFO - Epoch [179/200] (120420) train_loss: 23.7005, val_loss: 25.7899, lr: 0.000122, 121.45s 2023-05-19 00:01:08,933 - INFO - epoch complete! 2023-05-19 00:01:08,933 - INFO - evaluating now! 2023-05-19 00:01:15,450 - INFO - Epoch [180/200] (121089) train_loss: 23.7289, val_loss: 25.8446, lr: 0.000120, 121.65s 2023-05-19 00:03:17,630 - INFO - epoch complete! 2023-05-19 00:03:17,630 - INFO - evaluating now! 2023-05-19 00:03:24,154 - INFO - Epoch [181/200] (121758) train_loss: 23.7038, val_loss: 25.6973, lr: 0.000118, 121.56s 2023-05-19 00:05:26,062 - INFO - epoch complete! 2023-05-19 00:05:26,062 - INFO - evaluating now! 2023-05-19 00:05:32,567 - INFO - Epoch [182/200] (122427) train_loss: 23.6619, val_loss: 25.6786, lr: 0.000116, 120.97s 2023-05-19 00:07:35,367 - INFO - epoch complete! 2023-05-19 00:07:35,367 - INFO - evaluating now! 2023-05-19 00:07:41,862 - INFO - Epoch [183/200] (123096) train_loss: 23.6711, val_loss: 25.7469, lr: 0.000114, 121.47s 2023-05-19 00:09:44,603 - INFO - epoch complete! 2023-05-19 00:09:44,603 - INFO - evaluating now! 2023-05-19 00:09:51,144 - INFO - Epoch [184/200] (123765) train_loss: 23.6722, val_loss: 25.7611, lr: 0.000112, 121.50s 2023-05-19 00:11:55,288 - INFO - epoch complete! 2023-05-19 00:11:55,288 - INFO - evaluating now! 2023-05-19 00:12:01,737 - INFO - Epoch [185/200] (124434) train_loss: 23.6685, val_loss: 25.7274, lr: 0.000111, 123.06s 2023-05-19 00:14:06,475 - INFO - epoch complete! 2023-05-19 00:14:06,476 - INFO - evaluating now! 2023-05-19 00:14:12,975 - INFO - Epoch [186/200] (125103) train_loss: 23.6586, val_loss: 25.7116, lr: 0.000109, 123.26s 2023-05-19 00:16:19,015 - INFO - epoch complete! 2023-05-19 00:16:19,016 - INFO - evaluating now! 2023-05-19 00:16:25,524 - INFO - Epoch [187/200] (125772) train_loss: 23.6300, val_loss: 25.7483, lr: 0.000108, 124.96s 2023-05-19 00:18:30,042 - INFO - epoch complete! 2023-05-19 00:18:30,043 - INFO - evaluating now! 2023-05-19 00:18:36,570 - INFO - Epoch [188/200] (126441) train_loss: 23.6355, val_loss: 25.7200, lr: 0.000107, 123.90s 2023-05-19 00:20:39,750 - INFO - epoch complete! 2023-05-19 00:20:39,750 - INFO - evaluating now! 2023-05-19 00:20:46,261 - INFO - Epoch [189/200] (127110) train_loss: 23.5939, val_loss: 25.7019, lr: 0.000106, 122.59s 2023-05-19 00:22:50,707 - INFO - epoch complete! 2023-05-19 00:22:50,707 - INFO - evaluating now! 2023-05-19 00:22:57,214 - INFO - Epoch [190/200] (127779) train_loss: 23.6320, val_loss: 25.7699, lr: 0.000104, 123.45s 2023-05-19 00:25:01,442 - INFO - epoch complete! 2023-05-19 00:25:01,443 - INFO - evaluating now! 2023-05-19 00:25:07,962 - INFO - Epoch [191/200] (128448) train_loss: 23.6038, val_loss: 25.6920, lr: 0.000104, 123.34s 2023-05-19 00:27:11,539 - INFO - epoch complete! 2023-05-19 00:27:11,540 - INFO - evaluating now! 2023-05-19 00:27:18,030 - INFO - Epoch [192/200] (129117) train_loss: 23.5933, val_loss: 25.7102, lr: 0.000103, 122.38s 2023-05-19 00:29:22,375 - INFO - epoch complete! 2023-05-19 00:29:22,375 - INFO - evaluating now! 2023-05-19 00:29:28,880 - INFO - Epoch [193/200] (129786) train_loss: 23.6043, val_loss: 25.6707, lr: 0.000102, 122.92s 2023-05-19 00:31:33,395 - INFO - epoch complete! 2023-05-19 00:31:33,395 - INFO - evaluating now! 2023-05-19 00:31:39,915 - INFO - Epoch [194/200] (130455) train_loss: 23.5690, val_loss: 25.7523, lr: 0.000101, 123.13s 2023-05-19 00:33:44,299 - INFO - epoch complete! 2023-05-19 00:33:44,300 - INFO - evaluating now! 2023-05-19 00:33:50,818 - INFO - Epoch [195/200] (131124) train_loss: 23.5973, val_loss: 25.7384, lr: 0.000101, 123.22s 2023-05-19 00:35:55,015 - INFO - epoch complete! 2023-05-19 00:35:55,015 - INFO - evaluating now! 2023-05-19 00:36:01,524 - INFO - Epoch [196/200] (131793) train_loss: 23.5894, val_loss: 25.7409, lr: 0.000100, 122.81s 2023-05-19 00:38:05,709 - INFO - epoch complete! 2023-05-19 00:38:05,709 - INFO - evaluating now! 2023-05-19 00:38:12,211 - INFO - Epoch [197/200] (132462) train_loss: 23.5717, val_loss: 25.7026, lr: 0.000100, 123.45s 2023-05-19 00:40:17,102 - INFO - epoch complete! 2023-05-19 00:40:17,102 - INFO - evaluating now! 2023-05-19 00:40:23,617 - INFO - Epoch [198/200] (133131) train_loss: 23.5651, val_loss: 25.7071, lr: 0.000100, 123.82s 2023-05-19 00:42:27,362 - INFO - epoch complete! 2023-05-19 00:42:27,362 - INFO - evaluating now! 2023-05-19 00:42:33,888 - INFO - Epoch [199/200] (133800) train_loss: 23.5760, val_loss: 25.7220, lr: 0.000100, 122.25s 2023-05-19 00:42:33,888 - INFO - Trained totally 200 epochs, average train time is 120.845s, average eval time is 6.592s 2023-05-19 00:42:33,923 - INFO - Loaded model at 176 2023-05-19 00:42:33,924 - INFO - Saved model at ./libcity/cache/92489/model_cache/PDFormer_PeMS08.m 2023-05-19 00:42:33,956 - INFO - Start evaluating ... 2023-05-19 00:42:46,726 - INFO - Note that you select the average mode to evaluate! 2023-05-19 00:42:46,729 - INFO - Evaluate result is saved at ./libcity/cache/92489/evaluate_cache/2023_05_19_00_42_46_PDFormer_PeMS08_average.csv 2023-05-19 00:42:46,735 - INFO - MAE MAPE RMSE masked_MAE masked_MAPE masked_RMSE 1 11.807277 inf 19.672308 11.823156 0.078038 19.558163 2 12.032962 inf 20.286419 12.049309 0.079441 20.176619 3 12.252482 inf 20.817410 12.269487 0.080858 20.713125 4 12.448793 inf 21.282396 12.466464 0.082153 21.183153 5 12.624876 inf 21.685398 12.643200 0.083311 21.590067 6 12.784041 inf 22.039949 12.802795 0.084481 21.947374 7 12.932391 inf 22.358686 12.951554 0.085542 22.268145 8 13.071898 inf 22.650635 13.091407 0.086552 22.561703 9 13.202084 inf 22.917021 13.221920 0.087506 22.829172 10 13.329255 inf 23.166691 13.349422 0.088458 23.079922 11 13.472559 inf 23.394999 13.493119 0.089432 23.309145 12 13.643632 inf 23.640915 13.664682 0.090525 23.555864 ```

PEMS04

``` 2023-05-15 21:58:02,489 - INFO - Log directory: ./libcity/log 2023-05-15 21:58:02,490 - INFO - Begin pipeline, task=traffic_state_pred, model_name=PDFormer, dataset_name=PeMS04, exp_id=97069 2023-05-15 21:58:02,490 - INFO - {'task': 'traffic_state_pred', 'model': 'PDFormer', 'dataset': 'PeMS04', 'saved_model': True, 'train': True, 'local_rank': 0, 'gpu_id': [3], 'initial_ckpt': None, 'dataset_class': 'PDFormerDataset', 'input_window': 12, 'output_window': 12, 'train_rate': 0.6, 'eval_rate': 0.2, 'batch_size': 16, 'add_time_in_day': True, 'add_day_in_week': True, 'step_size': 1274, 'max_epoch': 200, 'bidir': True, 'far_mask_delta': 7, 'geo_num_heads': 4, 'sem_num_heads': 2, 't_num_heads': 2, 'cluster_method': 'kshape', 'cand_key_days': 14, 'seed': 1, 'type_ln': 'pre', 'set_loss': 'huber', 'huber_delta': 2, 'mode': 'average', 'executor': 'PDFormerExecutor', 'evaluator': 'TrafficStateEvaluator', 'embed_dim': 64, 'skip_dim': 256, 'mlp_ratio': 4, 'qkv_bias': True, 'drop': 0, 'attn_drop': 0, 'drop_path': 0.3, 's_attn_size': 3, 't_attn_size': 1, 'enc_depth': 6, 'type_short_path': 'hop', 'scaler': 'standard', 'load_external': True, 'normal_external': False, 'ext_scaler': 'none', 'learner': 'adamw', 'learning_rate': 0.001, 'weight_decay': 0.05, 'lr_decay': True, 'lr_scheduler': 'cosinelr', 'lr_eta_min': 0.0001, 'lr_decay_ratio': 0.1, 'lr_warmup_epoch': 5, 'lr_warmup_init': 1e-06, 'clip_grad_norm': True, 'max_grad_norm': 5, 'use_early_stop': True, 'patience': 50, 'task_level': 0, 'use_curriculum_learning': True, 'random_flip': True, 'quan_delta': 0.25, 'dtw_delta': 5, 'cache_dataset': True, 'num_workers': 0, 'pad_with_last_sample': True, 'lape_dim': 8, 'gpu': True, 'train_loss': 'none', 'epoch': 0, 'lr_epsilon': 1e-08, 'lr_beta1': 0.9, 'lr_beta2': 0.999, 'lr_alpha': 0.99, 'lr_momentum': 0, 'steps': [5, 20, 40, 70], 'lr_T_max': 30, 'lr_patience': 10, 'lr_threshold': 0.0001, 'log_level': 'INFO', 'log_every': 1, 'load_best_epoch': True, 'hyper_tune': False, 'grad_accmu_steps': 1, 'metrics': ['MAE', 'MAPE', 'RMSE', 'masked_MAE', 'masked_MAPE', 'masked_RMSE'], 'save_modes': ['csv'], 'geo': {'including_types': ['Point'], 'Point': {}}, 'rel': {'including_types': ['geo'], 'geo': {'cost': 'num'}}, 'dyna': {'including_types': ['state'], 'state': {'entity_id': 'geo_id', 'traffic_flow': 'num', 'traffic_occupancy': 'num', 'traffic_speed': 'num'}}, 'data_col': ['traffic_flow'], 'weight_col': 'cost', 'data_files': ['PeMS04'], 'geo_file': 'PeMS04', 'rel_file': 'PeMS04', 'output_dim': 1, 'time_intervals': 300, 'init_weight_inf_or_zero': 'zero', 'set_weight_link_or_dist': 'link', 'calculate_weight_adj': False, 'weight_adj_epsilon': 0, 'distributed': False, 'device': device(type='cuda', index=0), 'exp_id': 97069} 2023-05-15 21:58:02,853 - INFO - Loaded file PeMS04.geo, num_nodes=307 2023-05-15 21:58:02,855 - INFO - set_weight_link_or_dist: link 2023-05-15 21:58:02,855 - INFO - init_weight_inf_or_zero: zero 2023-05-15 21:58:02,858 - INFO - Loaded file PeMS04.rel, shape=(307, 307) 2023-05-15 21:58:02,859 - INFO - Max adj_mx value = 1.0 2023-05-15 21:59:27,050 - INFO - Loading file PeMS04.dyna 2023-05-15 21:59:30,670 - INFO - Loaded file PeMS04.dyna, shape=(16992, 307, 1) 2023-05-15 21:59:30,702 - INFO - Load DTW matrix from ./libcity/cache/dataset_cache/dtw_PeMS04.npy 2023-05-15 21:59:30,702 - INFO - Loading file PeMS04.dyna 2023-05-15 21:59:34,293 - INFO - Loaded file PeMS04.dyna, shape=(16992, 307, 1) 2023-05-15 21:59:45,470 - INFO - Dataset created 2023-05-15 21:59:45,470 - INFO - x shape: (16969, 12, 307, 9), y shape: (16969, 12, 307, 9) 2023-05-15 21:59:45,493 - INFO - train x: (10181, 12, 307, 9), y: (10181, 12, 307, 9) 2023-05-15 21:59:45,493 - INFO - eval x: (3394, 12, 307, 9), y: (3394, 12, 307, 9) 2023-05-15 21:59:45,493 - INFO - test x: (3394, 12, 307, 9), y: (3394, 12, 307, 9) 2023-05-15 22:00:59,928 - INFO - Saved at ./libcity/cache/dataset_cache/pdformer_point_based_PeMS04_12_12_0.6_1_0.2_standard_16_True_True_True_True_traffic_flow.npz 2023-05-15 22:01:01,248 - INFO - StandardScaler mean: 207.22733840505313, std: 156.47765518492758 2023-05-15 22:01:01,248 - INFO - NoneScaler 2023-05-15 22:01:09,145 - INFO - Loaded file ./libcity/cache/dataset_cache/pattern_keys_kshape_PeMS04_14_3_16_5.npy 2023-05-15 22:01:09,158 - INFO - Use use_curriculum_learning! 2023-05-15 22:01:12,134 - INFO - PDFormer( (pattern_embeddings): ModuleList( (0): TokenEmbedding( (token_embed): Linear(in_features=3, out_features=64, bias=True) (norm): Identity() ) ) (enc_embed_layer): DataEmbedding( (value_embedding): TokenEmbedding( (token_embed): Linear(in_features=1, out_features=64, bias=True) (norm): Identity() ) (position_encoding): PositionalEncoding() (daytime_embedding): Embedding(1440, 64) (weekday_embedding): Embedding(7, 64) (spatial_embedding): LaplacianPE( (embedding_lap_pos_enc): Linear(in_features=8, out_features=64, bias=True) ) (dropout): Dropout(p=0, inplace=False) ) (encoder_blocks): ModuleList( (0): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): Identity() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (1): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (2): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (3): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (4): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (5): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) ) (skip_convs): ModuleList( (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (1): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (2): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (4): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (5): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) ) (end_conv1): Conv2d(12, 12, kernel_size=(1, 1), stride=(1, 1)) (end_conv2): Conv2d(256, 1, kernel_size=(1, 1), stride=(1, 1)) ) 2023-05-15 22:01:12,136 - INFO - pattern_embeddings.0.token_embed.weight torch.Size([64, 3]) cuda:0 True 2023-05-15 22:01:12,136 - INFO - pattern_embeddings.0.token_embed.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,136 - INFO - enc_embed_layer.value_embedding.token_embed.weight torch.Size([64, 1]) cuda:0 True 2023-05-15 22:01:12,136 - INFO - enc_embed_layer.value_embedding.token_embed.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,136 - INFO - enc_embed_layer.daytime_embedding.weight torch.Size([1440, 64]) cuda:0 True 2023-05-15 22:01:12,136 - INFO - enc_embed_layer.weekday_embedding.weight torch.Size([7, 64]) cuda:0 True 2023-05-15 22:01:12,136 - INFO - enc_embed_layer.spatial_embedding.embedding_lap_pos_enc.weight torch.Size([64, 8]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - enc_embed_layer.spatial_embedding.embedding_lap_pos_enc.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,137 - INFO - encoder_blocks.0.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.0.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,138 - INFO - encoder_blocks.1.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 22:01:12,139 - INFO - encoder_blocks.1.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,140 - INFO - encoder_blocks.2.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.2.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,141 - INFO - encoder_blocks.3.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.3.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,142 - INFO - encoder_blocks.4.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,143 - INFO - encoder_blocks.4.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.4.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.norm1.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.norm1.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,144 - INFO - encoder_blocks.5.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.norm2.weight torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.norm2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - encoder_blocks.5.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - skip_convs.0.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,145 - INFO - skip_convs.0.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.1.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.1.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.2.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.2.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.3.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.3.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.4.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.4.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.5.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - skip_convs.5.bias torch.Size([256]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - end_conv1.weight torch.Size([12, 12, 1, 1]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - end_conv1.bias torch.Size([12]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - end_conv2.weight torch.Size([1, 256, 1, 1]) cuda:0 True 2023-05-15 22:01:12,146 - INFO - end_conv2.bias torch.Size([1]) cuda:0 True 2023-05-15 22:01:12,147 - INFO - Total parameter numbers: 531165 2023-05-15 22:01:12,147 - INFO - You select `adamw` optimizer. 2023-05-15 22:01:12,148 - INFO - You select `cosinelr` lr_scheduler. 2023-05-15 22:01:12,148 - WARNING - Received none train loss func and will use the loss func defined in the model. 2023-05-15 22:01:12,149 - INFO - Number of isolated points: 0 2023-05-15 22:01:12,259 - INFO - Start training ... 2023-05-15 22:01:12,259 - INFO - num_batches:637 2023-05-15 22:01:12,312 - INFO - Training: task_level increase from 0 to 1 2023-05-15 22:01:12,312 - INFO - Current batches_seen is 0 2023-05-15 22:04:44,260 - INFO - epoch complete! 2023-05-15 22:04:44,261 - INFO - evaluating now! 2023-05-15 22:05:04,868 - INFO - Epoch [0/200] (637) train_loss: 190.6860, val_loss: 233.8297, lr: 0.000201, 232.61s 2023-05-15 22:05:04,915 - INFO - Saved model at 0 2023-05-15 22:05:04,916 - INFO - Val loss decrease from inf to 233.8297, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch0.tar 2023-05-15 22:08:36,990 - INFO - epoch complete! 2023-05-15 22:08:36,991 - INFO - evaluating now! 2023-05-15 22:08:57,652 - INFO - Epoch [1/200] (1274) train_loss: 55.3242, val_loss: 220.0484, lr: 0.000401, 232.74s 2023-05-15 22:08:57,698 - INFO - Saved model at 1 2023-05-15 22:08:57,698 - INFO - Val loss decrease from 233.8297 to 220.0484, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch1.tar 2023-05-15 22:08:57,720 - INFO - Training: task_level increase from 1 to 2 2023-05-15 22:08:57,720 - INFO - Current batches_seen is 1274 2023-05-15 22:12:30,653 - INFO - epoch complete! 2023-05-15 22:12:30,653 - INFO - evaluating now! 2023-05-15 22:12:51,455 - INFO - Epoch [2/200] (1911) train_loss: 47.8835, val_loss: 200.7080, lr: 0.000600, 233.76s 2023-05-15 22:12:51,502 - INFO - Saved model at 2 2023-05-15 22:12:51,502 - INFO - Val loss decrease from 220.0484 to 200.7080, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch2.tar 2023-05-15 22:16:24,711 - INFO - epoch complete! 2023-05-15 22:16:24,712 - INFO - evaluating now! 2023-05-15 22:16:45,426 - INFO - Epoch [3/200] (2548) train_loss: 40.4374, val_loss: 203.1034, lr: 0.000800, 233.92s 2023-05-15 22:16:45,447 - INFO - Training: task_level increase from 2 to 3 2023-05-15 22:16:45,448 - INFO - Current batches_seen is 2548 2023-05-15 22:20:18,275 - INFO - epoch complete! 2023-05-15 22:20:18,276 - INFO - evaluating now! 2023-05-15 22:20:38,985 - INFO - Epoch [4/200] (3185) train_loss: 39.8593, val_loss: 194.7218, lr: 0.000999, 233.56s 2023-05-15 22:20:39,032 - INFO - Saved model at 4 2023-05-15 22:20:39,032 - INFO - Val loss decrease from 200.7080 to 194.7218, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch4.tar 2023-05-15 22:24:11,515 - INFO - epoch complete! 2023-05-15 22:24:11,516 - INFO - evaluating now! 2023-05-15 22:24:32,202 - INFO - Epoch [5/200] (3822) train_loss: 38.6492, val_loss: 194.7309, lr: 0.000998, 233.17s 2023-05-15 22:24:32,223 - INFO - Training: task_level increase from 3 to 4 2023-05-15 22:24:32,223 - INFO - Current batches_seen is 3822 2023-05-15 22:28:05,298 - INFO - epoch complete! 2023-05-15 22:28:05,299 - INFO - evaluating now! 2023-05-15 22:28:26,152 - INFO - Epoch [6/200] (4459) train_loss: 41.0695, val_loss: 175.0038, lr: 0.000997, 233.95s 2023-05-15 22:28:26,198 - INFO - Saved model at 6 2023-05-15 22:28:26,198 - INFO - Val loss decrease from 194.7218 to 175.0038, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch6.tar 2023-05-15 22:31:58,975 - INFO - epoch complete! 2023-05-15 22:31:58,975 - INFO - evaluating now! 2023-05-15 22:32:19,624 - INFO - Epoch [7/200] (5096) train_loss: 38.2783, val_loss: 173.4990, lr: 0.000996, 233.43s 2023-05-15 22:32:19,671 - INFO - Saved model at 7 2023-05-15 22:32:19,671 - INFO - Val loss decrease from 175.0038 to 173.4990, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch7.tar 2023-05-15 22:32:19,692 - INFO - Training: task_level increase from 4 to 5 2023-05-15 22:32:19,692 - INFO - Current batches_seen is 5096 2023-05-15 22:35:57,613 - INFO - epoch complete! 2023-05-15 22:35:57,613 - INFO - evaluating now! 2023-05-15 22:36:18,621 - INFO - Epoch [8/200] (5733) train_loss: 39.9574, val_loss: 154.6023, lr: 0.000996, 238.95s 2023-05-15 22:36:18,667 - INFO - Saved model at 8 2023-05-15 22:36:18,668 - INFO - Val loss decrease from 173.4990 to 154.6023, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch8.tar 2023-05-15 22:39:53,554 - INFO - epoch complete! 2023-05-15 22:39:53,573 - INFO - evaluating now! 2023-05-15 22:40:14,394 - INFO - Epoch [9/200] (6370) train_loss: 38.6530, val_loss: 154.0868, lr: 0.000994, 235.73s 2023-05-15 22:40:14,440 - INFO - Saved model at 9 2023-05-15 22:40:14,441 - INFO - Val loss decrease from 154.6023 to 154.0868, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch9.tar 2023-05-15 22:40:14,462 - INFO - Training: task_level increase from 5 to 6 2023-05-15 22:40:14,462 - INFO - Current batches_seen is 6370 2023-05-15 22:43:47,900 - INFO - epoch complete! 2023-05-15 22:43:47,901 - INFO - evaluating now! 2023-05-15 22:44:08,664 - INFO - Epoch [10/200] (7007) train_loss: 40.4667, val_loss: 134.9352, lr: 0.000993, 234.22s 2023-05-15 22:44:08,710 - INFO - Saved model at 10 2023-05-15 22:44:08,710 - INFO - Val loss decrease from 154.0868 to 134.9352, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch10.tar 2023-05-15 22:47:43,134 - INFO - epoch complete! 2023-05-15 22:47:43,134 - INFO - evaluating now! 2023-05-15 22:48:03,791 - INFO - Epoch [11/200] (7644) train_loss: 38.6955, val_loss: 135.4914, lr: 0.000992, 235.08s 2023-05-15 22:48:03,812 - INFO - Training: task_level increase from 6 to 7 2023-05-15 22:48:03,812 - INFO - Current batches_seen is 7644 2023-05-15 22:51:36,488 - INFO - epoch complete! 2023-05-15 22:51:36,488 - INFO - evaluating now! 2023-05-15 22:51:57,194 - INFO - Epoch [12/200] (8281) train_loss: 39.9268, val_loss: 117.0623, lr: 0.000991, 233.40s 2023-05-15 22:51:57,241 - INFO - Saved model at 12 2023-05-15 22:51:57,241 - INFO - Val loss decrease from 134.9352 to 117.0623, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch12.tar 2023-05-15 22:55:29,952 - INFO - epoch complete! 2023-05-15 22:55:29,953 - INFO - evaluating now! 2023-05-15 22:55:50,710 - INFO - Epoch [13/200] (8918) train_loss: 39.0473, val_loss: 117.2023, lr: 0.000989, 233.47s 2023-05-15 22:55:50,731 - INFO - Training: task_level increase from 7 to 8 2023-05-15 22:55:50,731 - INFO - Current batches_seen is 8918 2023-05-15 22:59:23,837 - INFO - epoch complete! 2023-05-15 22:59:23,838 - INFO - evaluating now! 2023-05-15 22:59:44,525 - INFO - Epoch [14/200] (9555) train_loss: 39.8649, val_loss: 100.0900, lr: 0.000988, 233.81s 2023-05-15 22:59:44,571 - INFO - Saved model at 14 2023-05-15 22:59:44,571 - INFO - Val loss decrease from 117.0623 to 100.0900, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch14.tar 2023-05-15 23:03:17,234 - INFO - epoch complete! 2023-05-15 23:03:17,235 - INFO - evaluating now! 2023-05-15 23:03:37,943 - INFO - Epoch [15/200] (10192) train_loss: 39.2263, val_loss: 99.8353, lr: 0.000986, 233.37s 2023-05-15 23:03:37,990 - INFO - Saved model at 15 2023-05-15 23:03:37,990 - INFO - Val loss decrease from 100.0900 to 99.8353, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch15.tar 2023-05-15 23:03:38,011 - INFO - Training: task_level increase from 8 to 9 2023-05-15 23:03:38,011 - INFO - Current batches_seen is 10192 2023-05-15 23:07:10,371 - INFO - epoch complete! 2023-05-15 23:07:10,371 - INFO - evaluating now! 2023-05-15 23:07:31,065 - INFO - Epoch [16/200] (10829) train_loss: 40.0309, val_loss: 96.8797, lr: 0.000984, 233.08s 2023-05-15 23:07:31,112 - INFO - Saved model at 16 2023-05-15 23:07:31,112 - INFO - Val loss decrease from 99.8353 to 96.8797, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch16.tar 2023-05-15 23:10:58,054 - INFO - epoch complete! 2023-05-15 23:10:58,054 - INFO - evaluating now! 2023-05-15 23:11:18,943 - INFO - Epoch [17/200] (11466) train_loss: 39.6614, val_loss: 96.2000, lr: 0.000982, 227.83s 2023-05-15 23:11:18,990 - INFO - Saved model at 17 2023-05-15 23:11:18,991 - INFO - Val loss decrease from 96.8797 to 96.2000, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch17.tar 2023-05-15 23:11:19,012 - INFO - Training: task_level increase from 9 to 10 2023-05-15 23:11:19,012 - INFO - Current batches_seen is 11466 2023-05-15 23:14:51,891 - INFO - epoch complete! 2023-05-15 23:14:51,892 - INFO - evaluating now! 2023-05-15 23:15:12,571 - INFO - Epoch [18/200] (12103) train_loss: 40.5259, val_loss: 78.2893, lr: 0.000980, 233.58s 2023-05-15 23:15:12,618 - INFO - Saved model at 18 2023-05-15 23:15:12,618 - INFO - Val loss decrease from 96.2000 to 78.2893, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch18.tar 2023-05-15 23:18:45,352 - INFO - epoch complete! 2023-05-15 23:18:45,352 - INFO - evaluating now! 2023-05-15 23:19:06,043 - INFO - Epoch [19/200] (12740) train_loss: 39.8636, val_loss: 78.4497, lr: 0.000978, 233.42s 2023-05-15 23:19:06,065 - INFO - Training: task_level increase from 10 to 11 2023-05-15 23:19:06,065 - INFO - Current batches_seen is 12740 2023-05-15 23:22:38,461 - INFO - epoch complete! 2023-05-15 23:22:38,461 - INFO - evaluating now! 2023-05-15 23:22:59,123 - INFO - Epoch [20/200] (13377) train_loss: 40.7370, val_loss: 60.1571, lr: 0.000976, 233.08s 2023-05-15 23:22:59,169 - INFO - Saved model at 20 2023-05-15 23:22:59,169 - INFO - Val loss decrease from 78.2893 to 60.1571, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch20.tar 2023-05-15 23:26:32,734 - INFO - epoch complete! 2023-05-15 23:26:32,735 - INFO - evaluating now! 2023-05-15 23:26:53,508 - INFO - Epoch [21/200] (14014) train_loss: 39.9656, val_loss: 60.1613, lr: 0.000973, 234.34s 2023-05-15 23:26:53,528 - INFO - Training: task_level increase from 11 to 12 2023-05-15 23:26:53,528 - INFO - Current batches_seen is 14014 2023-05-15 23:30:26,244 - INFO - epoch complete! 2023-05-15 23:30:26,244 - INFO - evaluating now! 2023-05-15 23:30:46,947 - INFO - Epoch [22/200] (14651) train_loss: 41.0042, val_loss: 41.7099, lr: 0.000971, 233.44s 2023-05-15 23:30:46,994 - INFO - Saved model at 22 2023-05-15 23:30:46,994 - INFO - Val loss decrease from 60.1571 to 41.7099, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch22.tar 2023-05-15 23:34:19,012 - INFO - epoch complete! 2023-05-15 23:34:19,013 - INFO - evaluating now! 2023-05-15 23:34:39,767 - INFO - Epoch [23/200] (15288) train_loss: 40.0304, val_loss: 40.7426, lr: 0.000968, 232.77s 2023-05-15 23:34:39,814 - INFO - Saved model at 23 2023-05-15 23:34:39,814 - INFO - Val loss decrease from 41.7099 to 40.7426, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch23.tar 2023-05-15 23:38:11,998 - INFO - epoch complete! 2023-05-15 23:38:11,999 - INFO - evaluating now! 2023-05-15 23:38:32,777 - INFO - Epoch [24/200] (15925) train_loss: 39.5405, val_loss: 40.6566, lr: 0.000966, 232.96s 2023-05-15 23:38:32,823 - INFO - Saved model at 24 2023-05-15 23:38:32,823 - INFO - Val loss decrease from 40.7426 to 40.6566, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch24.tar 2023-05-15 23:42:06,325 - INFO - epoch complete! 2023-05-15 23:42:06,326 - INFO - evaluating now! 2023-05-15 23:42:26,991 - INFO - Epoch [25/200] (16562) train_loss: 39.4095, val_loss: 42.5698, lr: 0.000963, 234.17s 2023-05-15 23:45:59,173 - INFO - epoch complete! 2023-05-15 23:45:59,174 - INFO - evaluating now! 2023-05-15 23:46:19,702 - INFO - Epoch [26/200] (17199) train_loss: 39.0831, val_loss: 42.4368, lr: 0.000960, 232.71s 2023-05-15 23:49:51,606 - INFO - epoch complete! 2023-05-15 23:49:51,607 - INFO - evaluating now! 2023-05-15 23:50:12,107 - INFO - Epoch [27/200] (17836) train_loss: 38.8841, val_loss: 39.8081, lr: 0.000957, 232.40s 2023-05-15 23:50:12,153 - INFO - Saved model at 27 2023-05-15 23:50:12,153 - INFO - Val loss decrease from 40.6566 to 39.8081, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch27.tar 2023-05-15 23:53:43,821 - INFO - epoch complete! 2023-05-15 23:53:43,822 - INFO - evaluating now! 2023-05-15 23:54:04,318 - INFO - Epoch [28/200] (18473) train_loss: 38.7181, val_loss: 40.2153, lr: 0.000954, 232.17s 2023-05-15 23:57:35,816 - INFO - epoch complete! 2023-05-15 23:57:35,816 - INFO - evaluating now! 2023-05-15 23:57:56,357 - INFO - Epoch [29/200] (19110) train_loss: 38.3767, val_loss: 39.4696, lr: 0.000951, 232.04s 2023-05-15 23:57:56,404 - INFO - Saved model at 29 2023-05-15 23:57:56,404 - INFO - Val loss decrease from 39.8081 to 39.4696, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch29.tar 2023-05-16 00:01:28,723 - INFO - epoch complete! 2023-05-16 00:01:28,723 - INFO - evaluating now! 2023-05-16 00:01:49,320 - INFO - Epoch [30/200] (19747) train_loss: 38.0242, val_loss: 40.8705, lr: 0.000948, 232.92s 2023-05-16 00:05:21,792 - INFO - epoch complete! 2023-05-16 00:05:21,792 - INFO - evaluating now! 2023-05-16 00:05:42,291 - INFO - Epoch [31/200] (20384) train_loss: 37.9677, val_loss: 41.5935, lr: 0.000944, 232.97s 2023-05-16 00:09:13,930 - INFO - epoch complete! 2023-05-16 00:09:13,930 - INFO - evaluating now! 2023-05-16 00:09:34,491 - INFO - Epoch [32/200] (21021) train_loss: 37.7497, val_loss: 40.0221, lr: 0.000941, 232.20s 2023-05-16 00:13:01,364 - INFO - epoch complete! 2023-05-16 00:13:01,365 - INFO - evaluating now! 2023-05-16 00:13:21,997 - INFO - Epoch [33/200] (21658) train_loss: 37.4746, val_loss: 38.5635, lr: 0.000937, 227.51s 2023-05-16 00:13:22,043 - INFO - Saved model at 33 2023-05-16 00:13:22,043 - INFO - Val loss decrease from 39.4696 to 38.5635, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch33.tar 2023-05-16 00:16:55,631 - INFO - epoch complete! 2023-05-16 00:16:55,631 - INFO - evaluating now! 2023-05-16 00:17:16,326 - INFO - Epoch [34/200] (22295) train_loss: 37.4229, val_loss: 38.4000, lr: 0.000934, 234.28s 2023-05-16 00:17:16,372 - INFO - Saved model at 34 2023-05-16 00:17:16,373 - INFO - Val loss decrease from 38.5635 to 38.4000, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch34.tar 2023-05-16 00:20:49,046 - INFO - epoch complete! 2023-05-16 00:20:49,047 - INFO - evaluating now! 2023-05-16 00:21:09,528 - INFO - Epoch [35/200] (22932) train_loss: 37.3600, val_loss: 37.9439, lr: 0.000930, 233.15s 2023-05-16 00:21:09,574 - INFO - Saved model at 35 2023-05-16 00:21:09,574 - INFO - Val loss decrease from 38.4000 to 37.9439, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch35.tar 2023-05-16 00:24:41,384 - INFO - epoch complete! 2023-05-16 00:24:41,384 - INFO - evaluating now! 2023-05-16 00:25:01,971 - INFO - Epoch [36/200] (23569) train_loss: 37.1545, val_loss: 37.7214, lr: 0.000926, 232.40s 2023-05-16 00:25:02,017 - INFO - Saved model at 36 2023-05-16 00:25:02,017 - INFO - Val loss decrease from 37.9439 to 37.7214, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch36.tar 2023-05-16 00:28:33,971 - INFO - epoch complete! 2023-05-16 00:28:33,972 - INFO - evaluating now! 2023-05-16 00:28:54,634 - INFO - Epoch [37/200] (24206) train_loss: 37.0147, val_loss: 38.5960, lr: 0.000922, 232.62s 2023-05-16 00:32:27,582 - INFO - epoch complete! 2023-05-16 00:32:27,582 - INFO - evaluating now! 2023-05-16 00:32:48,128 - INFO - Epoch [38/200] (24843) train_loss: 36.6856, val_loss: 37.4206, lr: 0.000918, 233.49s 2023-05-16 00:32:48,174 - INFO - Saved model at 38 2023-05-16 00:32:48,174 - INFO - Val loss decrease from 37.7214 to 37.4206, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch38.tar 2023-05-16 00:36:20,015 - INFO - epoch complete! 2023-05-16 00:36:20,016 - INFO - evaluating now! 2023-05-16 00:36:40,549 - INFO - Epoch [39/200] (25480) train_loss: 36.6811, val_loss: 37.9376, lr: 0.000914, 232.37s 2023-05-16 00:40:12,581 - INFO - epoch complete! 2023-05-16 00:40:12,582 - INFO - evaluating now! 2023-05-16 00:40:33,181 - INFO - Epoch [40/200] (26117) train_loss: 36.3933, val_loss: 37.6585, lr: 0.000910, 232.63s 2023-05-16 00:44:06,431 - INFO - epoch complete! 2023-05-16 00:44:06,432 - INFO - evaluating now! 2023-05-16 00:44:27,168 - INFO - Epoch [41/200] (26754) train_loss: 36.5197, val_loss: 37.4161, lr: 0.000906, 233.99s 2023-05-16 00:44:27,215 - INFO - Saved model at 41 2023-05-16 00:44:27,215 - INFO - Val loss decrease from 37.4206 to 37.4161, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch41.tar 2023-05-16 00:48:00,166 - INFO - epoch complete! 2023-05-16 00:48:00,166 - INFO - evaluating now! 2023-05-16 00:48:20,718 - INFO - Epoch [42/200] (27391) train_loss: 36.0876, val_loss: 39.4699, lr: 0.000901, 233.50s 2023-05-16 00:51:52,683 - INFO - epoch complete! 2023-05-16 00:51:52,684 - INFO - evaluating now! 2023-05-16 00:52:13,233 - INFO - Epoch [43/200] (28028) train_loss: 36.3674, val_loss: 37.7405, lr: 0.000897, 232.51s 2023-05-16 00:55:46,510 - INFO - epoch complete! 2023-05-16 00:55:46,510 - INFO - evaluating now! 2023-05-16 00:56:07,205 - INFO - Epoch [44/200] (28665) train_loss: 36.0672, val_loss: 37.6780, lr: 0.000892, 233.97s 2023-05-16 00:59:40,424 - INFO - epoch complete! 2023-05-16 00:59:40,424 - INFO - evaluating now! 2023-05-16 01:00:01,131 - INFO - Epoch [45/200] (29302) train_loss: 36.0118, val_loss: 37.1479, lr: 0.000888, 233.92s 2023-05-16 01:00:01,177 - INFO - Saved model at 45 2023-05-16 01:00:01,177 - INFO - Val loss decrease from 37.4161 to 37.1479, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch45.tar 2023-05-16 01:03:33,498 - INFO - epoch complete! 2023-05-16 01:03:33,499 - INFO - evaluating now! 2023-05-16 01:03:54,040 - INFO - Epoch [46/200] (29939) train_loss: 35.8536, val_loss: 36.8656, lr: 0.000883, 232.86s 2023-05-16 01:03:54,086 - INFO - Saved model at 46 2023-05-16 01:03:54,086 - INFO - Val loss decrease from 37.1479 to 36.8656, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch46.tar 2023-05-16 01:07:25,925 - INFO - epoch complete! 2023-05-16 01:07:25,926 - INFO - evaluating now! 2023-05-16 01:07:46,418 - INFO - Epoch [47/200] (30576) train_loss: 35.6886, val_loss: 36.5024, lr: 0.000878, 232.33s 2023-05-16 01:07:46,465 - INFO - Saved model at 47 2023-05-16 01:07:46,465 - INFO - Val loss decrease from 36.8656 to 36.5024, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch47.tar 2023-05-16 01:11:18,530 - INFO - epoch complete! 2023-05-16 01:11:18,531 - INFO - evaluating now! 2023-05-16 01:11:39,029 - INFO - Epoch [48/200] (31213) train_loss: 35.6547, val_loss: 36.4990, lr: 0.000873, 232.56s 2023-05-16 01:11:39,076 - INFO - Saved model at 48 2023-05-16 01:11:39,076 - INFO - Val loss decrease from 36.5024 to 36.4990, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch48.tar 2023-05-16 01:15:11,413 - INFO - epoch complete! 2023-05-16 01:15:11,414 - INFO - evaluating now! 2023-05-16 01:15:32,086 - INFO - Epoch [49/200] (31850) train_loss: 35.7806, val_loss: 36.3210, lr: 0.000868, 233.01s 2023-05-16 01:15:32,132 - INFO - Saved model at 49 2023-05-16 01:15:32,132 - INFO - Val loss decrease from 36.4990 to 36.3210, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch49.tar 2023-05-16 01:19:04,437 - INFO - epoch complete! 2023-05-16 01:19:04,438 - INFO - evaluating now! 2023-05-16 01:19:24,886 - INFO - Epoch [50/200] (32487) train_loss: 35.5808, val_loss: 37.1835, lr: 0.000863, 232.75s 2023-05-16 01:22:56,526 - INFO - epoch complete! 2023-05-16 01:22:56,527 - INFO - evaluating now! 2023-05-16 01:23:17,028 - INFO - Epoch [51/200] (33124) train_loss: 35.5545, val_loss: 36.3213, lr: 0.000858, 232.14s 2023-05-16 01:26:48,991 - INFO - epoch complete! 2023-05-16 01:26:48,992 - INFO - evaluating now! 2023-05-16 01:27:09,600 - INFO - Epoch [52/200] (33761) train_loss: 35.2853, val_loss: 36.6876, lr: 0.000853, 232.57s 2023-05-16 01:30:42,925 - INFO - epoch complete! 2023-05-16 01:30:42,926 - INFO - evaluating now! 2023-05-16 01:31:03,570 - INFO - Epoch [53/200] (34398) train_loss: 35.3043, val_loss: 36.3334, lr: 0.000848, 233.97s 2023-05-16 01:34:36,425 - INFO - epoch complete! 2023-05-16 01:34:36,426 - INFO - evaluating now! 2023-05-16 01:34:56,969 - INFO - Epoch [54/200] (35035) train_loss: 35.2114, val_loss: 36.6414, lr: 0.000842, 233.40s 2023-05-16 01:38:29,045 - INFO - epoch complete! 2023-05-16 01:38:29,045 - INFO - evaluating now! 2023-05-16 01:38:49,606 - INFO - Epoch [55/200] (35672) train_loss: 35.2438, val_loss: 36.4290, lr: 0.000837, 232.64s 2023-05-16 01:42:21,889 - INFO - epoch complete! 2023-05-16 01:42:21,889 - INFO - evaluating now! 2023-05-16 01:42:42,596 - INFO - Epoch [56/200] (36309) train_loss: 35.0732, val_loss: 36.5684, lr: 0.000831, 232.99s 2023-05-16 01:46:16,117 - INFO - epoch complete! 2023-05-16 01:46:16,118 - INFO - evaluating now! 2023-05-16 01:46:36,737 - INFO - Epoch [57/200] (36946) train_loss: 35.0888, val_loss: 36.1566, lr: 0.000826, 234.14s 2023-05-16 01:46:36,784 - INFO - Saved model at 57 2023-05-16 01:46:36,784 - INFO - Val loss decrease from 36.3210 to 36.1566, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch57.tar 2023-05-16 01:50:09,623 - INFO - epoch complete! 2023-05-16 01:50:09,624 - INFO - evaluating now! 2023-05-16 01:50:30,177 - INFO - Epoch [58/200] (37583) train_loss: 34.9984, val_loss: 35.8672, lr: 0.000820, 233.39s 2023-05-16 01:50:30,223 - INFO - Saved model at 58 2023-05-16 01:50:30,224 - INFO - Val loss decrease from 36.1566 to 35.8672, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch58.tar 2023-05-16 01:54:02,544 - INFO - epoch complete! 2023-05-16 01:54:02,544 - INFO - evaluating now! 2023-05-16 01:54:23,103 - INFO - Epoch [59/200] (38220) train_loss: 34.9681, val_loss: 36.4592, lr: 0.000815, 232.88s 2023-05-16 01:57:56,608 - INFO - epoch complete! 2023-05-16 01:57:56,609 - INFO - evaluating now! 2023-05-16 01:58:17,343 - INFO - Epoch [60/200] (38857) train_loss: 34.8242, val_loss: 36.1180, lr: 0.000809, 234.24s 2023-05-16 02:01:50,347 - INFO - epoch complete! 2023-05-16 02:01:50,348 - INFO - evaluating now! 2023-05-16 02:02:10,950 - INFO - Epoch [61/200] (39494) train_loss: 34.7306, val_loss: 35.8666, lr: 0.000803, 233.61s 2023-05-16 02:02:10,996 - INFO - Saved model at 61 2023-05-16 02:02:10,996 - INFO - Val loss decrease from 35.8672 to 35.8666, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch61.tar 2023-05-16 02:05:43,688 - INFO - epoch complete! 2023-05-16 02:05:43,689 - INFO - evaluating now! 2023-05-16 02:06:04,235 - INFO - Epoch [62/200] (40131) train_loss: 34.6009, val_loss: 36.6596, lr: 0.000797, 233.24s 2023-05-16 02:09:36,983 - INFO - epoch complete! 2023-05-16 02:09:36,984 - INFO - evaluating now! 2023-05-16 02:09:57,557 - INFO - Epoch [63/200] (40768) train_loss: 35.0177, val_loss: 36.5754, lr: 0.000791, 233.32s 2023-05-16 02:13:30,407 - INFO - epoch complete! 2023-05-16 02:13:30,408 - INFO - evaluating now! 2023-05-16 02:13:51,055 - INFO - Epoch [64/200] (41405) train_loss: 34.6391, val_loss: 36.1403, lr: 0.000785, 233.50s 2023-05-16 02:17:23,459 - INFO - epoch complete! 2023-05-16 02:17:23,459 - INFO - evaluating now! 2023-05-16 02:17:44,087 - INFO - Epoch [65/200] (42042) train_loss: 34.6354, val_loss: 36.2977, lr: 0.000779, 233.03s 2023-05-16 02:21:16,544 - INFO - epoch complete! 2023-05-16 02:21:16,545 - INFO - evaluating now! 2023-05-16 02:21:37,101 - INFO - Epoch [66/200] (42679) train_loss: 34.5720, val_loss: 36.6058, lr: 0.000773, 233.01s 2023-05-16 02:25:03,917 - INFO - epoch complete! 2023-05-16 02:25:03,918 - INFO - evaluating now! 2023-05-16 02:25:24,614 - INFO - Epoch [67/200] (43316) train_loss: 34.4319, val_loss: 35.8115, lr: 0.000767, 227.51s 2023-05-16 02:25:24,660 - INFO - Saved model at 67 2023-05-16 02:25:24,660 - INFO - Val loss decrease from 35.8666 to 35.8115, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch67.tar 2023-05-16 02:28:57,789 - INFO - epoch complete! 2023-05-16 02:28:57,790 - INFO - evaluating now! 2023-05-16 02:29:18,263 - INFO - Epoch [68/200] (43953) train_loss: 34.4204, val_loss: 35.7838, lr: 0.000761, 233.60s 2023-05-16 02:29:18,309 - INFO - Saved model at 68 2023-05-16 02:29:18,309 - INFO - Val loss decrease from 35.8115 to 35.7838, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch68.tar 2023-05-16 02:32:50,644 - INFO - epoch complete! 2023-05-16 02:32:50,644 - INFO - evaluating now! 2023-05-16 02:33:11,269 - INFO - Epoch [69/200] (44590) train_loss: 34.3371, val_loss: 35.6327, lr: 0.000754, 232.96s 2023-05-16 02:33:11,316 - INFO - Saved model at 69 2023-05-16 02:33:11,316 - INFO - Val loss decrease from 35.7838 to 35.6327, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch69.tar 2023-05-16 02:36:43,727 - INFO - epoch complete! 2023-05-16 02:36:43,728 - INFO - evaluating now! 2023-05-16 02:37:04,265 - INFO - Epoch [70/200] (45227) train_loss: 34.3363, val_loss: 35.5812, lr: 0.000748, 232.95s 2023-05-16 02:37:04,312 - INFO - Saved model at 70 2023-05-16 02:37:04,312 - INFO - Val loss decrease from 35.6327 to 35.5812, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch70.tar 2023-05-16 02:40:36,901 - INFO - epoch complete! 2023-05-16 02:40:36,902 - INFO - evaluating now! 2023-05-16 02:40:57,599 - INFO - Epoch [71/200] (45864) train_loss: 34.3784, val_loss: 36.6550, lr: 0.000742, 233.29s 2023-05-16 02:44:30,160 - INFO - epoch complete! 2023-05-16 02:44:30,160 - INFO - evaluating now! 2023-05-16 02:44:50,727 - INFO - Epoch [72/200] (46501) train_loss: 34.1990, val_loss: 36.0503, lr: 0.000735, 233.13s 2023-05-16 02:48:23,276 - INFO - epoch complete! 2023-05-16 02:48:23,276 - INFO - evaluating now! 2023-05-16 02:48:43,847 - INFO - Epoch [73/200] (47138) train_loss: 34.2948, val_loss: 35.4997, lr: 0.000729, 233.12s 2023-05-16 02:48:43,893 - INFO - Saved model at 73 2023-05-16 02:48:43,893 - INFO - Val loss decrease from 35.5812 to 35.4997, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch73.tar 2023-05-16 02:52:15,806 - INFO - epoch complete! 2023-05-16 02:52:15,806 - INFO - evaluating now! 2023-05-16 02:52:36,323 - INFO - Epoch [74/200] (47775) train_loss: 34.2155, val_loss: 35.8947, lr: 0.000722, 232.43s 2023-05-16 02:56:07,904 - INFO - epoch complete! 2023-05-16 02:56:07,905 - INFO - evaluating now! 2023-05-16 02:56:28,393 - INFO - Epoch [75/200] (48412) train_loss: 34.1795, val_loss: 36.5635, lr: 0.000716, 232.07s 2023-05-16 03:00:00,126 - INFO - epoch complete! 2023-05-16 03:00:00,126 - INFO - evaluating now! 2023-05-16 03:00:20,641 - INFO - Epoch [76/200] (49049) train_loss: 34.1175, val_loss: 35.5326, lr: 0.000709, 232.25s 2023-05-16 03:03:53,252 - INFO - epoch complete! 2023-05-16 03:03:53,252 - INFO - evaluating now! 2023-05-16 03:04:13,897 - INFO - Epoch [77/200] (49686) train_loss: 34.2035, val_loss: 36.2536, lr: 0.000702, 233.26s 2023-05-16 03:07:45,972 - INFO - epoch complete! 2023-05-16 03:07:45,972 - INFO - evaluating now! 2023-05-16 03:08:06,445 - INFO - Epoch [78/200] (50323) train_loss: 33.9663, val_loss: 35.9406, lr: 0.000696, 232.55s 2023-05-16 03:11:38,162 - INFO - epoch complete! 2023-05-16 03:11:38,163 - INFO - evaluating now! 2023-05-16 03:11:58,718 - INFO - Epoch [79/200] (50960) train_loss: 33.9979, val_loss: 35.9547, lr: 0.000689, 232.27s 2023-05-16 03:15:30,772 - INFO - epoch complete! 2023-05-16 03:15:30,772 - INFO - evaluating now! 2023-05-16 03:15:51,428 - INFO - Epoch [80/200] (51597) train_loss: 34.0802, val_loss: 36.6045, lr: 0.000682, 232.71s 2023-05-16 03:19:22,095 - INFO - epoch complete! 2023-05-16 03:19:22,095 - INFO - evaluating now! 2023-05-16 03:19:42,708 - INFO - Epoch [81/200] (52234) train_loss: 33.9805, val_loss: 35.6185, lr: 0.000676, 231.28s 2023-05-16 03:23:12,502 - INFO - epoch complete! 2023-05-16 03:23:12,502 - INFO - evaluating now! 2023-05-16 03:23:33,007 - INFO - Epoch [82/200] (52871) train_loss: 33.7727, val_loss: 35.5096, lr: 0.000669, 230.30s 2023-05-16 03:27:03,321 - INFO - epoch complete! 2023-05-16 03:27:03,321 - INFO - evaluating now! 2023-05-16 03:27:23,843 - INFO - Epoch [83/200] (53508) train_loss: 33.7644, val_loss: 36.1928, lr: 0.000662, 230.84s 2023-05-16 03:30:51,000 - INFO - epoch complete! 2023-05-16 03:30:51,001 - INFO - evaluating now! 2023-05-16 03:31:11,772 - INFO - Epoch [84/200] (54145) train_loss: 33.8737, val_loss: 35.4657, lr: 0.000655, 227.93s 2023-05-16 03:31:11,819 - INFO - Saved model at 84 2023-05-16 03:31:11,819 - INFO - Val loss decrease from 35.4997 to 35.4657, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch84.tar 2023-05-16 03:34:44,209 - INFO - epoch complete! 2023-05-16 03:34:44,209 - INFO - evaluating now! 2023-05-16 03:35:04,697 - INFO - Epoch [85/200] (54782) train_loss: 33.7330, val_loss: 35.4420, lr: 0.000648, 232.88s 2023-05-16 03:35:04,744 - INFO - Saved model at 85 2023-05-16 03:35:04,744 - INFO - Val loss decrease from 35.4657 to 35.4420, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch85.tar 2023-05-16 03:38:36,265 - INFO - epoch complete! 2023-05-16 03:38:36,266 - INFO - evaluating now! 2023-05-16 03:38:56,748 - INFO - Epoch [86/200] (55419) train_loss: 33.7071, val_loss: 35.6451, lr: 0.000641, 232.00s 2023-05-16 03:42:28,476 - INFO - epoch complete! 2023-05-16 03:42:28,477 - INFO - evaluating now! 2023-05-16 03:42:49,041 - INFO - Epoch [87/200] (56056) train_loss: 33.6250, val_loss: 35.2865, lr: 0.000634, 232.29s 2023-05-16 03:42:49,192 - INFO - Saved model at 87 2023-05-16 03:42:49,192 - INFO - Val loss decrease from 35.4420 to 35.2865, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch87.tar 2023-05-16 03:46:22,383 - INFO - epoch complete! 2023-05-16 03:46:22,384 - INFO - evaluating now! 2023-05-16 03:46:43,183 - INFO - Epoch [88/200] (56693) train_loss: 33.5341, val_loss: 35.2259, lr: 0.000627, 233.99s 2023-05-16 03:46:43,229 - INFO - Saved model at 88 2023-05-16 03:46:43,229 - INFO - Val loss decrease from 35.2865 to 35.2259, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch88.tar 2023-05-16 03:50:15,940 - INFO - epoch complete! 2023-05-16 03:50:15,941 - INFO - evaluating now! 2023-05-16 03:50:36,472 - INFO - Epoch [89/200] (57330) train_loss: 33.6945, val_loss: 36.5319, lr: 0.000620, 233.24s 2023-05-16 03:54:08,462 - INFO - epoch complete! 2023-05-16 03:54:08,463 - INFO - evaluating now! 2023-05-16 03:54:28,892 - INFO - Epoch [90/200] (57967) train_loss: 33.6121, val_loss: 35.7925, lr: 0.000613, 232.42s 2023-05-16 03:58:01,039 - INFO - epoch complete! 2023-05-16 03:58:01,039 - INFO - evaluating now! 2023-05-16 03:58:21,588 - INFO - Epoch [91/200] (58604) train_loss: 33.5036, val_loss: 36.3580, lr: 0.000606, 232.70s 2023-05-16 04:01:55,124 - INFO - epoch complete! 2023-05-16 04:01:55,125 - INFO - evaluating now! 2023-05-16 04:02:15,734 - INFO - Epoch [92/200] (59241) train_loss: 33.6220, val_loss: 36.1210, lr: 0.000599, 234.15s 2023-05-16 04:05:47,327 - INFO - epoch complete! 2023-05-16 04:05:47,328 - INFO - evaluating now! 2023-05-16 04:06:07,467 - INFO - Epoch [93/200] (59878) train_loss: 33.4230, val_loss: 35.5442, lr: 0.000592, 231.73s 2023-05-16 04:09:38,892 - INFO - epoch complete! 2023-05-16 04:09:38,893 - INFO - evaluating now! 2023-05-16 04:09:58,888 - INFO - Epoch [94/200] (60515) train_loss: 33.4384, val_loss: 35.2176, lr: 0.000585, 231.42s 2023-05-16 04:09:58,934 - INFO - Saved model at 94 2023-05-16 04:09:58,934 - INFO - Val loss decrease from 35.2259 to 35.2176, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch94.tar 2023-05-16 04:13:30,398 - INFO - epoch complete! 2023-05-16 04:13:30,399 - INFO - evaluating now! 2023-05-16 04:13:50,407 - INFO - Epoch [95/200] (61152) train_loss: 33.3585, val_loss: 35.5733, lr: 0.000578, 231.47s 2023-05-16 04:17:21,843 - INFO - epoch complete! 2023-05-16 04:17:21,844 - INFO - evaluating now! 2023-05-16 04:17:41,843 - INFO - Epoch [96/200] (61789) train_loss: 33.3694, val_loss: 35.3455, lr: 0.000571, 231.44s 2023-05-16 04:21:13,264 - INFO - epoch complete! 2023-05-16 04:21:13,265 - INFO - evaluating now! 2023-05-16 04:21:33,266 - INFO - Epoch [97/200] (62426) train_loss: 33.3768, val_loss: 35.9885, lr: 0.000564, 231.42s 2023-05-16 04:25:04,652 - INFO - epoch complete! 2023-05-16 04:25:04,652 - INFO - evaluating now! 2023-05-16 04:25:24,646 - INFO - Epoch [98/200] (63063) train_loss: 33.3607, val_loss: 35.3067, lr: 0.000557, 231.38s 2023-05-16 04:28:56,061 - INFO - epoch complete! 2023-05-16 04:28:56,062 - INFO - evaluating now! 2023-05-16 04:29:16,034 - INFO - Epoch [99/200] (63700) train_loss: 33.2482, val_loss: 35.6506, lr: 0.000550, 231.39s 2023-05-16 04:32:47,430 - INFO - epoch complete! 2023-05-16 04:32:47,431 - INFO - evaluating now! 2023-05-16 04:33:07,388 - INFO - Epoch [100/200] (64337) train_loss: 33.2356, val_loss: 35.5839, lr: 0.000543, 231.35s 2023-05-16 04:36:38,841 - INFO - epoch complete! 2023-05-16 04:36:38,842 - INFO - evaluating now! 2023-05-16 04:36:58,790 - INFO - Epoch [101/200] (64974) train_loss: 33.3313, val_loss: 35.5321, lr: 0.000536, 231.40s 2023-05-16 04:40:30,195 - INFO - epoch complete! 2023-05-16 04:40:30,196 - INFO - evaluating now! 2023-05-16 04:40:50,156 - INFO - Epoch [102/200] (65611) train_loss: 33.2929, val_loss: 35.3021, lr: 0.000529, 231.36s 2023-05-16 04:44:21,592 - INFO - epoch complete! 2023-05-16 04:44:21,593 - INFO - evaluating now! 2023-05-16 04:44:41,563 - INFO - Epoch [103/200] (66248) train_loss: 33.1762, val_loss: 35.4362, lr: 0.000522, 231.41s 2023-05-16 04:48:13,007 - INFO - epoch complete! 2023-05-16 04:48:13,008 - INFO - evaluating now! 2023-05-16 04:48:32,982 - INFO - Epoch [104/200] (66885) train_loss: 33.1268, val_loss: 35.2132, lr: 0.000515, 231.42s 2023-05-16 04:48:33,027 - INFO - Saved model at 104 2023-05-16 04:48:33,028 - INFO - Val loss decrease from 35.2176 to 35.2132, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch104.tar 2023-05-16 04:52:04,447 - INFO - epoch complete! 2023-05-16 04:52:04,448 - INFO - evaluating now! 2023-05-16 04:52:24,390 - INFO - Epoch [105/200] (67522) train_loss: 33.1263, val_loss: 35.1183, lr: 0.000508, 231.36s 2023-05-16 04:52:24,435 - INFO - Saved model at 105 2023-05-16 04:52:24,436 - INFO - Val loss decrease from 35.2132 to 35.1183, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch105.tar 2023-05-16 04:55:55,938 - INFO - epoch complete! 2023-05-16 04:55:55,938 - INFO - evaluating now! 2023-05-16 04:56:15,884 - INFO - Epoch [106/200] (68159) train_loss: 33.0436, val_loss: 35.2858, lr: 0.000501, 231.45s 2023-05-16 04:59:47,325 - INFO - epoch complete! 2023-05-16 04:59:47,326 - INFO - evaluating now! 2023-05-16 05:00:07,311 - INFO - Epoch [107/200] (68796) train_loss: 33.1267, val_loss: 35.5378, lr: 0.000494, 231.43s 2023-05-16 05:03:38,772 - INFO - epoch complete! 2023-05-16 05:03:38,773 - INFO - evaluating now! 2023-05-16 05:03:58,722 - INFO - Epoch [108/200] (69433) train_loss: 33.0286, val_loss: 34.9249, lr: 0.000487, 231.41s 2023-05-16 05:03:58,768 - INFO - Saved model at 108 2023-05-16 05:03:58,768 - INFO - Val loss decrease from 35.1183 to 34.9249, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch108.tar 2023-05-16 05:07:30,184 - INFO - epoch complete! 2023-05-16 05:07:30,185 - INFO - evaluating now! 2023-05-16 05:07:50,126 - INFO - Epoch [109/200] (70070) train_loss: 33.0006, val_loss: 35.4918, lr: 0.000480, 231.36s 2023-05-16 05:11:21,537 - INFO - epoch complete! 2023-05-16 05:11:21,538 - INFO - evaluating now! 2023-05-16 05:11:41,514 - INFO - Epoch [110/200] (70707) train_loss: 32.9544, val_loss: 35.5459, lr: 0.000473, 231.39s 2023-05-16 05:15:12,921 - INFO - epoch complete! 2023-05-16 05:15:12,922 - INFO - evaluating now! 2023-05-16 05:15:32,875 - INFO - Epoch [111/200] (71344) train_loss: 32.9411, val_loss: 35.0717, lr: 0.000466, 231.36s 2023-05-16 05:19:04,349 - INFO - epoch complete! 2023-05-16 05:19:04,350 - INFO - evaluating now! 2023-05-16 05:19:24,291 - INFO - Epoch [112/200] (71981) train_loss: 32.9342, val_loss: 35.1832, lr: 0.000459, 231.42s 2023-05-16 05:22:55,713 - INFO - epoch complete! 2023-05-16 05:22:55,714 - INFO - evaluating now! 2023-05-16 05:23:15,637 - INFO - Epoch [113/200] (72618) train_loss: 32.9489, val_loss: 35.7243, lr: 0.000452, 231.34s 2023-05-16 05:26:47,066 - INFO - epoch complete! 2023-05-16 05:26:47,067 - INFO - evaluating now! 2023-05-16 05:27:07,022 - INFO - Epoch [114/200] (73255) train_loss: 32.8464, val_loss: 34.9797, lr: 0.000445, 231.38s 2023-05-16 05:30:38,451 - INFO - epoch complete! 2023-05-16 05:30:38,451 - INFO - evaluating now! 2023-05-16 05:30:58,411 - INFO - Epoch [115/200] (73892) train_loss: 32.8874, val_loss: 35.3142, lr: 0.000438, 231.39s 2023-05-16 05:34:29,895 - INFO - epoch complete! 2023-05-16 05:34:29,896 - INFO - evaluating now! 2023-05-16 05:34:49,831 - INFO - Epoch [116/200] (74529) train_loss: 32.8219, val_loss: 35.1231, lr: 0.000431, 231.42s 2023-05-16 05:38:21,286 - INFO - epoch complete! 2023-05-16 05:38:21,287 - INFO - evaluating now! 2023-05-16 05:38:41,237 - INFO - Epoch [117/200] (75166) train_loss: 32.8106, val_loss: 35.0698, lr: 0.000424, 231.41s 2023-05-16 05:42:12,647 - INFO - epoch complete! 2023-05-16 05:42:12,648 - INFO - evaluating now! 2023-05-16 05:42:32,606 - INFO - Epoch [118/200] (75803) train_loss: 32.7773, val_loss: 35.1048, lr: 0.000418, 231.37s 2023-05-16 05:46:04,020 - INFO - epoch complete! 2023-05-16 05:46:04,021 - INFO - evaluating now! 2023-05-16 05:46:23,968 - INFO - Epoch [119/200] (76440) train_loss: 32.7244, val_loss: 36.1552, lr: 0.000411, 231.36s 2023-05-16 05:49:55,414 - INFO - epoch complete! 2023-05-16 05:49:55,414 - INFO - evaluating now! 2023-05-16 05:50:15,351 - INFO - Epoch [120/200] (77077) train_loss: 32.7067, val_loss: 35.1053, lr: 0.000404, 231.38s 2023-05-16 05:53:46,752 - INFO - epoch complete! 2023-05-16 05:53:46,753 - INFO - evaluating now! 2023-05-16 05:54:06,667 - INFO - Epoch [121/200] (77714) train_loss: 32.7113, val_loss: 35.1538, lr: 0.000398, 231.31s 2023-05-16 05:57:38,084 - INFO - epoch complete! 2023-05-16 05:57:38,084 - INFO - evaluating now! 2023-05-16 05:57:57,977 - INFO - Epoch [122/200] (78351) train_loss: 32.6609, val_loss: 35.3961, lr: 0.000391, 231.31s 2023-05-16 06:01:29,399 - INFO - epoch complete! 2023-05-16 06:01:29,400 - INFO - evaluating now! 2023-05-16 06:01:49,339 - INFO - Epoch [123/200] (78988) train_loss: 32.6214, val_loss: 35.0836, lr: 0.000384, 231.36s 2023-05-16 06:05:20,795 - INFO - epoch complete! 2023-05-16 06:05:20,795 - INFO - evaluating now! 2023-05-16 06:05:40,742 - INFO - Epoch [124/200] (79625) train_loss: 32.6203, val_loss: 35.1729, lr: 0.000378, 231.40s 2023-05-16 06:09:12,150 - INFO - epoch complete! 2023-05-16 06:09:12,150 - INFO - evaluating now! 2023-05-16 06:09:32,066 - INFO - Epoch [125/200] (80262) train_loss: 32.6000, val_loss: 34.9363, lr: 0.000371, 231.32s 2023-05-16 06:13:03,462 - INFO - epoch complete! 2023-05-16 06:13:03,462 - INFO - evaluating now! 2023-05-16 06:13:23,419 - INFO - Epoch [126/200] (80899) train_loss: 32.5327, val_loss: 35.3524, lr: 0.000365, 231.35s 2023-05-16 06:16:54,853 - INFO - epoch complete! 2023-05-16 06:16:54,854 - INFO - evaluating now! 2023-05-16 06:17:14,828 - INFO - Epoch [127/200] (81536) train_loss: 32.5241, val_loss: 35.2280, lr: 0.000358, 231.41s 2023-05-16 06:20:46,229 - INFO - epoch complete! 2023-05-16 06:20:46,229 - INFO - evaluating now! 2023-05-16 06:21:06,170 - INFO - Epoch [128/200] (82173) train_loss: 32.5162, val_loss: 35.1354, lr: 0.000352, 231.34s 2023-05-16 06:24:37,578 - INFO - epoch complete! 2023-05-16 06:24:37,579 - INFO - evaluating now! 2023-05-16 06:24:57,512 - INFO - Epoch [129/200] (82810) train_loss: 32.4828, val_loss: 35.6410, lr: 0.000346, 231.34s 2023-05-16 06:28:28,919 - INFO - epoch complete! 2023-05-16 06:28:28,919 - INFO - evaluating now! 2023-05-16 06:28:48,869 - INFO - Epoch [130/200] (83447) train_loss: 32.4525, val_loss: 34.9361, lr: 0.000339, 231.36s 2023-05-16 06:32:20,295 - INFO - epoch complete! 2023-05-16 06:32:20,295 - INFO - evaluating now! 2023-05-16 06:32:40,254 - INFO - Epoch [131/200] (84084) train_loss: 32.5024, val_loss: 34.9665, lr: 0.000333, 231.38s 2023-05-16 06:36:11,699 - INFO - epoch complete! 2023-05-16 06:36:11,700 - INFO - evaluating now! 2023-05-16 06:36:31,630 - INFO - Epoch [132/200] (84721) train_loss: 32.4410, val_loss: 34.9290, lr: 0.000327, 231.37s 2023-05-16 06:40:03,027 - INFO - epoch complete! 2023-05-16 06:40:03,028 - INFO - evaluating now! 2023-05-16 06:40:22,991 - INFO - Epoch [133/200] (85358) train_loss: 32.4330, val_loss: 35.1353, lr: 0.000321, 231.36s 2023-05-16 06:43:54,413 - INFO - epoch complete! 2023-05-16 06:43:54,413 - INFO - evaluating now! 2023-05-16 06:44:14,367 - INFO - Epoch [134/200] (85995) train_loss: 32.3666, val_loss: 35.0968, lr: 0.000315, 231.37s 2023-05-16 06:47:45,805 - INFO - epoch complete! 2023-05-16 06:47:45,805 - INFO - evaluating now! 2023-05-16 06:48:05,763 - INFO - Epoch [135/200] (86632) train_loss: 32.3684, val_loss: 35.0613, lr: 0.000309, 231.40s 2023-05-16 06:51:37,172 - INFO - epoch complete! 2023-05-16 06:51:37,172 - INFO - evaluating now! 2023-05-16 06:51:57,088 - INFO - Epoch [136/200] (87269) train_loss: 32.2898, val_loss: 35.1139, lr: 0.000303, 231.32s 2023-05-16 06:55:28,510 - INFO - epoch complete! 2023-05-16 06:55:28,511 - INFO - evaluating now! 2023-05-16 06:55:48,446 - INFO - Epoch [137/200] (87906) train_loss: 32.3393, val_loss: 34.9416, lr: 0.000297, 231.36s 2023-05-16 06:59:19,878 - INFO - epoch complete! 2023-05-16 06:59:19,878 - INFO - evaluating now! 2023-05-16 06:59:39,848 - INFO - Epoch [138/200] (88543) train_loss: 32.3048, val_loss: 35.0818, lr: 0.000291, 231.40s 2023-05-16 07:03:11,217 - INFO - epoch complete! 2023-05-16 07:03:11,217 - INFO - evaluating now! 2023-05-16 07:03:31,166 - INFO - Epoch [139/200] (89180) train_loss: 32.2779, val_loss: 35.0730, lr: 0.000285, 231.32s 2023-05-16 07:07:02,589 - INFO - epoch complete! 2023-05-16 07:07:02,589 - INFO - evaluating now! 2023-05-16 07:07:22,526 - INFO - Epoch [140/200] (89817) train_loss: 32.2569, val_loss: 35.0447, lr: 0.000280, 231.36s 2023-05-16 07:10:53,940 - INFO - epoch complete! 2023-05-16 07:10:53,940 - INFO - evaluating now! 2023-05-16 07:11:13,855 - INFO - Epoch [141/200] (90454) train_loss: 32.2180, val_loss: 34.8588, lr: 0.000274, 231.33s 2023-05-16 07:11:13,901 - INFO - Saved model at 141 2023-05-16 07:11:13,901 - INFO - Val loss decrease from 34.9249 to 34.8588, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch141.tar 2023-05-16 07:14:45,374 - INFO - epoch complete! 2023-05-16 07:14:45,375 - INFO - evaluating now! 2023-05-16 07:15:05,302 - INFO - Epoch [142/200] (91091) train_loss: 32.2068, val_loss: 34.9933, lr: 0.000269, 231.40s 2023-05-16 07:18:36,741 - INFO - epoch complete! 2023-05-16 07:18:36,742 - INFO - evaluating now! 2023-05-16 07:18:56,685 - INFO - Epoch [143/200] (91728) train_loss: 32.1657, val_loss: 35.2475, lr: 0.000263, 231.38s 2023-05-16 07:22:28,156 - INFO - epoch complete! 2023-05-16 07:22:28,157 - INFO - evaluating now! 2023-05-16 07:22:48,096 - INFO - Epoch [144/200] (92365) train_loss: 32.1282, val_loss: 35.0671, lr: 0.000258, 231.41s 2023-05-16 07:26:19,482 - INFO - epoch complete! 2023-05-16 07:26:19,483 - INFO - evaluating now! 2023-05-16 07:26:39,411 - INFO - Epoch [145/200] (93002) train_loss: 32.1275, val_loss: 35.0762, lr: 0.000252, 231.31s 2023-05-16 07:30:10,852 - INFO - epoch complete! 2023-05-16 07:30:10,853 - INFO - evaluating now! 2023-05-16 07:30:30,803 - INFO - Epoch [146/200] (93639) train_loss: 32.0714, val_loss: 35.0282, lr: 0.000247, 231.39s 2023-05-16 07:34:02,221 - INFO - epoch complete! 2023-05-16 07:34:02,221 - INFO - evaluating now! 2023-05-16 07:34:22,147 - INFO - Epoch [147/200] (94276) train_loss: 32.1068, val_loss: 35.0172, lr: 0.000242, 231.34s 2023-05-16 07:37:53,559 - INFO - epoch complete! 2023-05-16 07:37:53,560 - INFO - evaluating now! 2023-05-16 07:38:13,483 - INFO - Epoch [148/200] (94913) train_loss: 32.0572, val_loss: 34.8900, lr: 0.000237, 231.33s 2023-05-16 07:41:44,939 - INFO - epoch complete! 2023-05-16 07:41:44,939 - INFO - evaluating now! 2023-05-16 07:42:04,872 - INFO - Epoch [149/200] (95550) train_loss: 32.0617, val_loss: 34.9826, lr: 0.000232, 231.39s 2023-05-16 07:45:36,323 - INFO - epoch complete! 2023-05-16 07:45:36,324 - INFO - evaluating now! 2023-05-16 07:45:56,275 - INFO - Epoch [150/200] (96187) train_loss: 32.0467, val_loss: 34.9218, lr: 0.000227, 231.40s 2023-05-16 07:49:27,711 - INFO - epoch complete! 2023-05-16 07:49:27,712 - INFO - evaluating now! 2023-05-16 07:49:47,640 - INFO - Epoch [151/200] (96824) train_loss: 32.0206, val_loss: 34.8047, lr: 0.000222, 231.36s 2023-05-16 07:49:47,686 - INFO - Saved model at 151 2023-05-16 07:49:47,686 - INFO - Val loss decrease from 34.8588 to 34.8047, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch151.tar 2023-05-16 07:53:19,138 - INFO - epoch complete! 2023-05-16 07:53:19,139 - INFO - evaluating now! 2023-05-16 07:53:39,072 - INFO - Epoch [152/200] (97461) train_loss: 31.9491, val_loss: 34.8506, lr: 0.000217, 231.39s 2023-05-16 07:57:10,476 - INFO - epoch complete! 2023-05-16 07:57:10,476 - INFO - evaluating now! 2023-05-16 07:57:30,402 - INFO - Epoch [153/200] (98098) train_loss: 31.9698, val_loss: 34.9655, lr: 0.000212, 231.33s 2023-05-16 08:01:01,822 - INFO - epoch complete! 2023-05-16 08:01:01,823 - INFO - evaluating now! 2023-05-16 08:01:21,806 - INFO - Epoch [154/200] (98735) train_loss: 31.9404, val_loss: 34.9211, lr: 0.000208, 231.40s 2023-05-16 08:04:53,223 - INFO - epoch complete! 2023-05-16 08:04:53,224 - INFO - evaluating now! 2023-05-16 08:05:13,163 - INFO - Epoch [155/200] (99372) train_loss: 31.9617, val_loss: 35.0056, lr: 0.000203, 231.36s 2023-05-16 08:08:44,603 - INFO - epoch complete! 2023-05-16 08:08:44,603 - INFO - evaluating now! 2023-05-16 08:09:04,533 - INFO - Epoch [156/200] (100009) train_loss: 31.9445, val_loss: 34.9930, lr: 0.000199, 231.37s 2023-05-16 08:12:35,974 - INFO - epoch complete! 2023-05-16 08:12:35,975 - INFO - evaluating now! 2023-05-16 08:12:55,927 - INFO - Epoch [157/200] (100646) train_loss: 31.9302, val_loss: 34.9550, lr: 0.000194, 231.39s 2023-05-16 08:16:27,346 - INFO - epoch complete! 2023-05-16 08:16:27,346 - INFO - evaluating now! 2023-05-16 08:16:47,297 - INFO - Epoch [158/200] (101283) train_loss: 31.8859, val_loss: 34.8538, lr: 0.000190, 231.37s 2023-05-16 08:20:18,728 - INFO - epoch complete! 2023-05-16 08:20:18,729 - INFO - evaluating now! 2023-05-16 08:20:38,653 - INFO - Epoch [159/200] (101920) train_loss: 31.8896, val_loss: 34.9968, lr: 0.000186, 231.36s 2023-05-16 08:24:10,049 - INFO - epoch complete! 2023-05-16 08:24:10,049 - INFO - evaluating now! 2023-05-16 08:24:29,977 - INFO - Epoch [160/200] (102557) train_loss: 31.8451, val_loss: 35.3568, lr: 0.000182, 231.32s 2023-05-16 08:28:01,446 - INFO - epoch complete! 2023-05-16 08:28:01,447 - INFO - evaluating now! 2023-05-16 08:28:21,401 - INFO - Epoch [161/200] (103194) train_loss: 31.8616, val_loss: 35.0493, lr: 0.000178, 231.42s 2023-05-16 08:31:52,829 - INFO - epoch complete! 2023-05-16 08:31:52,829 - INFO - evaluating now! 2023-05-16 08:32:12,772 - INFO - Epoch [162/200] (103831) train_loss: 31.8756, val_loss: 34.9831, lr: 0.000174, 231.37s 2023-05-16 08:35:44,177 - INFO - epoch complete! 2023-05-16 08:35:44,177 - INFO - evaluating now! 2023-05-16 08:36:04,113 - INFO - Epoch [163/200] (104468) train_loss: 31.8151, val_loss: 34.8731, lr: 0.000170, 231.34s 2023-05-16 08:39:35,532 - INFO - epoch complete! 2023-05-16 08:39:35,533 - INFO - evaluating now! 2023-05-16 08:39:55,470 - INFO - Epoch [164/200] (105105) train_loss: 31.8227, val_loss: 35.1309, lr: 0.000166, 231.36s 2023-05-16 08:43:26,898 - INFO - epoch complete! 2023-05-16 08:43:26,899 - INFO - evaluating now! 2023-05-16 08:43:46,855 - INFO - Epoch [165/200] (105742) train_loss: 31.7626, val_loss: 34.8372, lr: 0.000163, 231.38s 2023-05-16 08:47:18,255 - INFO - epoch complete! 2023-05-16 08:47:18,255 - INFO - evaluating now! 2023-05-16 08:47:38,192 - INFO - Epoch [166/200] (106379) train_loss: 31.7669, val_loss: 34.9134, lr: 0.000159, 231.34s 2023-05-16 08:51:09,581 - INFO - epoch complete! 2023-05-16 08:51:09,581 - INFO - evaluating now! 2023-05-16 08:51:29,509 - INFO - Epoch [167/200] (107016) train_loss: 31.7532, val_loss: 34.9049, lr: 0.000156, 231.32s 2023-05-16 08:55:00,960 - INFO - epoch complete! 2023-05-16 08:55:00,960 - INFO - evaluating now! 2023-05-16 08:55:20,864 - INFO - Epoch [168/200] (107653) train_loss: 31.7620, val_loss: 34.8509, lr: 0.000152, 231.35s 2023-05-16 08:58:52,317 - INFO - epoch complete! 2023-05-16 08:58:52,318 - INFO - evaluating now! 2023-05-16 08:59:12,204 - INFO - Epoch [169/200] (108290) train_loss: 31.7213, val_loss: 34.9646, lr: 0.000149, 231.34s 2023-05-16 09:02:43,621 - INFO - epoch complete! 2023-05-16 09:02:43,622 - INFO - evaluating now! 2023-05-16 09:03:03,533 - INFO - Epoch [170/200] (108927) train_loss: 31.7205, val_loss: 34.8408, lr: 0.000146, 231.33s 2023-05-16 09:06:34,945 - INFO - epoch complete! 2023-05-16 09:06:34,945 - INFO - evaluating now! 2023-05-16 09:06:54,886 - INFO - Epoch [171/200] (109564) train_loss: 31.7330, val_loss: 35.0244, lr: 0.000143, 231.35s 2023-05-16 09:10:26,317 - INFO - epoch complete! 2023-05-16 09:10:26,317 - INFO - evaluating now! 2023-05-16 09:10:46,208 - INFO - Epoch [172/200] (110201) train_loss: 31.6756, val_loss: 34.8079, lr: 0.000140, 231.32s 2023-05-16 09:14:17,628 - INFO - epoch complete! 2023-05-16 09:14:17,629 - INFO - evaluating now! 2023-05-16 09:14:37,548 - INFO - Epoch [173/200] (110838) train_loss: 31.6860, val_loss: 34.8861, lr: 0.000137, 231.34s 2023-05-16 09:18:08,150 - INFO - epoch complete! 2023-05-16 09:18:08,150 - INFO - evaluating now! 2023-05-16 09:18:28,102 - INFO - Epoch [174/200] (111475) train_loss: 31.6711, val_loss: 35.1095, lr: 0.000134, 230.55s 2023-05-16 09:21:59,509 - INFO - epoch complete! 2023-05-16 09:21:59,510 - INFO - evaluating now! 2023-05-16 09:22:19,450 - INFO - Epoch [175/200] (112112) train_loss: 31.6574, val_loss: 34.8242, lr: 0.000132, 231.35s 2023-05-16 09:25:50,858 - INFO - epoch complete! 2023-05-16 09:25:50,859 - INFO - evaluating now! 2023-05-16 09:26:10,775 - INFO - Epoch [176/200] (112749) train_loss: 31.6195, val_loss: 34.9364, lr: 0.000129, 231.32s 2023-05-16 09:29:42,184 - INFO - epoch complete! 2023-05-16 09:29:42,184 - INFO - evaluating now! 2023-05-16 09:30:02,138 - INFO - Epoch [177/200] (113386) train_loss: 31.6289, val_loss: 34.8208, lr: 0.000127, 231.36s 2023-05-16 09:33:33,537 - INFO - epoch complete! 2023-05-16 09:33:33,537 - INFO - evaluating now! 2023-05-16 09:33:53,489 - INFO - Epoch [178/200] (114023) train_loss: 31.6310, val_loss: 34.8457, lr: 0.000124, 231.35s 2023-05-16 09:37:24,909 - INFO - epoch complete! 2023-05-16 09:37:24,910 - INFO - evaluating now! 2023-05-16 09:37:44,823 - INFO - Epoch [179/200] (114660) train_loss: 31.6585, val_loss: 34.8074, lr: 0.000122, 231.33s 2023-05-16 09:41:16,205 - INFO - epoch complete! 2023-05-16 09:41:16,206 - INFO - evaluating now! 2023-05-16 09:41:36,139 - INFO - Epoch [180/200] (115297) train_loss: 31.6088, val_loss: 34.9527, lr: 0.000120, 231.31s 2023-05-16 09:45:07,576 - INFO - epoch complete! 2023-05-16 09:45:07,577 - INFO - evaluating now! 2023-05-16 09:45:27,516 - INFO - Epoch [181/200] (115934) train_loss: 31.5935, val_loss: 34.7946, lr: 0.000118, 231.38s 2023-05-16 09:45:27,562 - INFO - Saved model at 181 2023-05-16 09:45:27,562 - INFO - Val loss decrease from 34.8047 to 34.7946, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch181.tar 2023-05-16 09:48:58,985 - INFO - epoch complete! 2023-05-16 09:48:58,986 - INFO - evaluating now! 2023-05-16 09:49:18,937 - INFO - Epoch [182/200] (116571) train_loss: 31.5793, val_loss: 34.8513, lr: 0.000116, 231.37s 2023-05-16 09:52:50,340 - INFO - epoch complete! 2023-05-16 09:52:50,341 - INFO - evaluating now! 2023-05-16 09:53:10,241 - INFO - Epoch [183/200] (117208) train_loss: 31.5492, val_loss: 34.7962, lr: 0.000114, 231.30s 2023-05-16 09:56:41,568 - INFO - epoch complete! 2023-05-16 09:56:41,569 - INFO - evaluating now! 2023-05-16 09:57:01,499 - INFO - Epoch [184/200] (117845) train_loss: 31.5745, val_loss: 34.9237, lr: 0.000112, 231.26s 2023-05-16 10:00:32,882 - INFO - epoch complete! 2023-05-16 10:00:32,882 - INFO - evaluating now! 2023-05-16 10:00:52,817 - INFO - Epoch [185/200] (118482) train_loss: 31.5642, val_loss: 34.8873, lr: 0.000111, 231.32s 2023-05-16 10:04:24,278 - INFO - epoch complete! 2023-05-16 10:04:24,278 - INFO - evaluating now! 2023-05-16 10:04:44,210 - INFO - Epoch [186/200] (119119) train_loss: 31.5483, val_loss: 34.8142, lr: 0.000109, 231.39s 2023-05-16 10:08:15,635 - INFO - epoch complete! 2023-05-16 10:08:15,636 - INFO - evaluating now! 2023-05-16 10:08:35,539 - INFO - Epoch [187/200] (119756) train_loss: 31.5623, val_loss: 34.9446, lr: 0.000108, 231.33s 2023-05-16 10:12:06,940 - INFO - epoch complete! 2023-05-16 10:12:06,941 - INFO - evaluating now! 2023-05-16 10:12:26,843 - INFO - Epoch [188/200] (120393) train_loss: 31.5480, val_loss: 34.8367, lr: 0.000107, 231.30s 2023-05-16 10:15:58,307 - INFO - epoch complete! 2023-05-16 10:15:58,308 - INFO - evaluating now! 2023-05-16 10:16:18,216 - INFO - Epoch [189/200] (121030) train_loss: 31.5483, val_loss: 35.0609, lr: 0.000106, 231.37s 2023-05-16 10:19:49,636 - INFO - epoch complete! 2023-05-16 10:19:49,637 - INFO - evaluating now! 2023-05-16 10:20:09,567 - INFO - Epoch [190/200] (121667) train_loss: 31.5342, val_loss: 34.8103, lr: 0.000104, 231.35s 2023-05-16 10:23:41,007 - INFO - epoch complete! 2023-05-16 10:23:41,007 - INFO - evaluating now! 2023-05-16 10:24:00,918 - INFO - Epoch [191/200] (122304) train_loss: 31.5067, val_loss: 34.7660, lr: 0.000104, 231.35s 2023-05-16 10:24:00,963 - INFO - Saved model at 191 2023-05-16 10:24:00,964 - INFO - Val loss decrease from 34.7946 to 34.7660, saving to ./libcity/cache/97069/model_cache/PDFormer_PeMS04_epoch191.tar 2023-05-16 10:27:32,374 - INFO - epoch complete! 2023-05-16 10:27:32,375 - INFO - evaluating now! 2023-05-16 10:27:52,307 - INFO - Epoch [192/200] (122941) train_loss: 31.5534, val_loss: 34.9183, lr: 0.000103, 231.34s 2023-05-16 10:31:23,702 - INFO - epoch complete! 2023-05-16 10:31:23,702 - INFO - evaluating now! 2023-05-16 10:31:43,659 - INFO - Epoch [193/200] (123578) train_loss: 31.5329, val_loss: 34.8209, lr: 0.000102, 231.35s 2023-05-16 10:35:15,082 - INFO - epoch complete! 2023-05-16 10:35:15,082 - INFO - evaluating now! 2023-05-16 10:35:35,021 - INFO - Epoch [194/200] (124215) train_loss: 31.4737, val_loss: 34.9052, lr: 0.000101, 231.36s 2023-05-16 10:39:06,409 - INFO - epoch complete! 2023-05-16 10:39:06,410 - INFO - evaluating now! 2023-05-16 10:39:26,339 - INFO - Epoch [195/200] (124852) train_loss: 31.5284, val_loss: 34.8728, lr: 0.000101, 231.32s 2023-05-16 10:42:57,728 - INFO - epoch complete! 2023-05-16 10:42:57,729 - INFO - evaluating now! 2023-05-16 10:43:17,745 - INFO - Epoch [196/200] (125489) train_loss: 31.4981, val_loss: 34.8923, lr: 0.000100, 231.41s 2023-05-16 10:46:49,174 - INFO - epoch complete! 2023-05-16 10:46:49,175 - INFO - evaluating now! 2023-05-16 10:47:09,126 - INFO - Epoch [197/200] (126126) train_loss: 31.5013, val_loss: 34.8772, lr: 0.000100, 231.38s 2023-05-16 10:50:40,523 - INFO - epoch complete! 2023-05-16 10:50:40,523 - INFO - evaluating now! 2023-05-16 10:51:00,516 - INFO - Epoch [198/200] (126763) train_loss: 31.4992, val_loss: 35.0476, lr: 0.000100, 231.39s 2023-05-16 10:54:31,941 - INFO - epoch complete! 2023-05-16 10:54:31,942 - INFO - evaluating now! 2023-05-16 10:54:51,906 - INFO - Epoch [199/200] (127400) train_loss: 31.4942, val_loss: 34.9323, lr: 0.000100, 231.39s 2023-05-16 10:54:51,907 - INFO - Trained totally 200 epochs, average train time is 211.824s, average eval time is 20.260s 2023-05-16 10:54:51,958 - INFO - Loaded model at 191 2023-05-16 10:54:51,958 - INFO - Saved model at ./libcity/cache/97069/model_cache/PDFormer_PeMS04.m 2023-05-16 10:54:52,003 - INFO - Start evaluating ... 2023-05-16 10:55:25,803 - INFO - Note that you select the average mode to evaluate! 2023-05-16 10:55:25,808 - INFO - Evaluate result is saved at ./libcity/cache/97069/evaluate_cache/2023_05_16_10_55_25_PDFormer_PeMS04_average.csv 2023-05-16 10:55:25,817 - INFO - MAE MAPE RMSE masked_MAE masked_MAPE masked_RMSE 1 16.488237 inf 27.033958 16.616440 0.109217 26.958183 2 16.749134 inf 27.546349 16.873692 0.110845 27.452517 3 16.980698 inf 27.983900 17.102612 0.112179 27.875938 4 17.177589 inf 28.347172 17.297112 0.113359 28.227507 5 17.348188 inf 28.657040 17.466003 0.114348 28.527237 6 17.499729 inf 28.929235 17.615850 0.115272 28.789883 7 17.641754 inf 29.181208 17.756535 0.116133 29.032999 8 17.773027 inf 29.412554 17.886360 0.116933 29.255606 9 17.896318 inf 29.628168 18.008062 0.117671 29.462416 10 18.012926 inf 29.829189 18.122938 0.118415 29.654417 11 18.128349 inf 30.023409 18.236618 0.119193 29.839643 12 18.251507 inf 30.222811 18.357948 0.120025 30.030327 ```

PEMSBAY

``` 2023-05-16 10:55:22,561 - INFO - Log directory: ./libcity/log 2023-05-16 10:55:22,561 - INFO - Begin pipeline, task=traffic_state_pred, model_name=PDFormer, dataset_name=PEMSBAY, exp_id=11302 2023-05-16 10:55:22,561 - INFO - {'task': 'traffic_state_pred', 'model': 'PDFormer', 'dataset': 'PEMSBAY', 'saved_model': True, 'train': True, 'local_rank': 0, 'gpu_id': [7], 'initial_ckpt': None, 'dataset_class': 'PDFormerDataset', 'input_window': 12, 'output_window': 12, 'train_rate': 0.7, 'eval_rate': 0.1, 'batch_size': 16, 'add_time_in_day': True, 'add_day_in_week': True, 'step_size': 2500, 'max_epoch': 200, 'bidir': True, 'far_mask_delta': 7, 'geo_num_heads': 4, 'sem_num_heads': 2, 't_num_heads': 2, 'cluster_method': 'kshape', 'cand_key_days': 21, 'seed': 1, 'type_ln': 'pre', 'set_loss': 'huber', 'huber_delta': 2, 'mode': 'average', 'executor': 'PDFormerExecutor', 'evaluator': 'TrafficStateEvaluator', 'embed_dim': 64, 'skip_dim': 256, 'mlp_ratio': 4, 'qkv_bias': True, 'drop': 0, 'attn_drop': 0, 'drop_path': 0.3, 's_attn_size': 3, 't_attn_size': 1, 'enc_depth': 6, 'type_short_path': 'hop', 'scaler': 'standard', 'load_external': True, 'normal_external': False, 'ext_scaler': 'none', 'learner': 'adamw', 'learning_rate': 0.001, 'weight_decay': 0.05, 'lr_decay': True, 'lr_scheduler': 'cosinelr', 'lr_eta_min': 0.0001, 'lr_decay_ratio': 0.1, 'lr_warmup_epoch': 5, 'lr_warmup_init': 1e-06, 'clip_grad_norm': True, 'max_grad_norm': 5, 'use_early_stop': True, 'patience': 50, 'task_level': 0, 'use_curriculum_learning': True, 'random_flip': True, 'quan_delta': 0.25, 'dtw_delta': 5, 'cache_dataset': True, 'num_workers': 0, 'pad_with_last_sample': True, 'lape_dim': 8, 'gpu': True, 'train_loss': 'none', 'epoch': 0, 'lr_epsilon': 1e-08, 'lr_beta1': 0.9, 'lr_beta2': 0.999, 'lr_alpha': 0.99, 'lr_momentum': 0, 'steps': [5, 20, 40, 70], 'lr_T_max': 30, 'lr_patience': 10, 'lr_threshold': 0.0001, 'log_level': 'INFO', 'log_every': 1, 'load_best_epoch': True, 'hyper_tune': False, 'grad_accmu_steps': 1, 'metrics': ['MAE', 'MAPE', 'RMSE', 'masked_MAE', 'masked_MAPE', 'masked_RMSE'], 'save_modes': ['csv'], 'geo': {'including_types': ['Point'], 'Point': {}}, 'rel': {'including_types': ['geo'], 'geo': {'cost': 'num'}}, 'dyna': {'including_types': ['state'], 'state': {'entity_id': 'geo_id', 'traffic_speed': 'num'}}, 'data_col': ['traffic_speed'], 'weight_col': 'cost', 'data_files': ['PEMSBAY'], 'geo_file': 'PEMSBAY', 'rel_file': 'PEMSBAY', 'output_dim': 1, 'time_intervals': 300, 'init_weight_inf_or_zero': 'inf', 'set_weight_link_or_dist': 'dist', 'calculate_weight_adj': True, 'weight_adj_epsilon': 0.1, 'distributed': False, 'device': device(type='cuda', index=0), 'exp_id': 11302} 2023-05-16 10:55:22,899 - INFO - Loaded file PEMSBAY.geo, num_nodes=325 2023-05-16 10:55:22,906 - INFO - set_weight_link_or_dist: dist 2023-05-16 10:55:22,906 - INFO - init_weight_inf_or_zero: inf 2023-05-16 10:55:22,936 - INFO - Loaded file PEMSBAY.rel, shape=(325, 325) 2023-05-16 10:55:22,937 - INFO - Start Calculate the weight by Gauss kernel! 2023-05-16 10:55:22,939 - INFO - Max adj_mx value = 1.0 2023-05-16 10:56:53,494 - INFO - Loading file PEMSBAY.dyna 2023-05-16 10:57:05,367 - INFO - Loaded file PEMSBAY.dyna, shape=(52116, 325, 1) 2023-05-16 15:34:53,221 - INFO - Load DTW matrix from ./libcity/cache/dataset_cache/dtw_PEMSBAY.npy 2023-05-16 15:34:53,221 - INFO - Loading file PEMSBAY.dyna 2023-05-16 15:35:05,117 - INFO - Loaded file PEMSBAY.dyna, shape=(52116, 325, 1) 2023-05-16 15:35:47,962 - INFO - Dataset created 2023-05-16 15:35:47,962 - INFO - x shape: (52093, 12, 325, 9), y shape: (52093, 12, 325, 9) 2023-05-16 15:35:48,047 - INFO - train x: (36465, 12, 325, 9), y: (36465, 12, 325, 9) 2023-05-16 15:35:48,047 - INFO - eval x: (5209, 12, 325, 9), y: (5209, 12, 325, 9) 2023-05-16 15:35:48,047 - INFO - test x: (10419, 12, 325, 9), y: (10419, 12, 325, 9) 2023-05-16 15:42:01,121 - INFO - Saved at ./libcity/cache/dataset_cache/pdformer_point_based_PEMSBAY_12_12_0.7_1_0.1_standard_16_True_True_True_True_traffic_speed.npz 2023-05-16 15:42:08,301 - INFO - StandardScaler mean: 62.73567651594296, std: 9.438173664359947 2023-05-16 15:42:08,301 - INFO - NoneScaler 2023-05-16 15:42:45,996 - INFO - Clustering... 2023-05-16 18:01:11,749 - INFO - Saved at file ./libcity/cache/dataset_cache/pattern_keys_kshape_PEMSBAY_21_3_16_5.npy 2023-05-16 18:01:11,816 - INFO - Use use_curriculum_learning! 2023-05-16 18:01:15,603 - INFO - PDFormer( (pattern_embeddings): ModuleList( (0): TokenEmbedding( (token_embed): Linear(in_features=3, out_features=64, bias=True) (norm): Identity() ) ) (enc_embed_layer): DataEmbedding( (value_embedding): TokenEmbedding( (token_embed): Linear(in_features=1, out_features=64, bias=True) (norm): Identity() ) (position_encoding): PositionalEncoding() (daytime_embedding): Embedding(1440, 64) (weekday_embedding): Embedding(7, 64) (spatial_embedding): LaplacianPE( (embedding_lap_pos_enc): Linear(in_features=8, out_features=64, bias=True) ) (dropout): Dropout(p=0, inplace=False) ) (encoder_blocks): ModuleList( (0): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): Identity() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (1): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (2): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (3): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (4): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) (5): STEncoderBlock( (norm1): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (st_attn): STSelfAttention( (pattern_q_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_k_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (pattern_v_linears): ModuleList( (0): Linear(in_features=64, out_features=32, bias=True) ) (geo_q_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_k_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_v_conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1)) (geo_attn_drop): Dropout(p=0, inplace=False) (sem_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (sem_attn_drop): Dropout(p=0, inplace=False) (t_q_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_k_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_v_conv): Conv2d(64, 16, kernel_size=(1, 1), stride=(1, 1)) (t_attn_drop): Dropout(p=0, inplace=False) (proj): Linear(in_features=64, out_features=64, bias=True) (proj_drop): Dropout(p=0, inplace=False) ) (drop_path): DropPath() (norm2): LayerNorm((64,), eps=1e-06, elementwise_affine=True) (mlp): Mlp( (fc1): Linear(in_features=64, out_features=256, bias=True) (act): GELU() (fc2): Linear(in_features=256, out_features=64, bias=True) (drop): Dropout(p=0, inplace=False) ) ) ) (skip_convs): ModuleList( (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (1): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (2): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (3): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (4): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) (5): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1)) ) (end_conv1): Conv2d(12, 12, kernel_size=(1, 1), stride=(1, 1)) (end_conv2): Conv2d(256, 1, kernel_size=(1, 1), stride=(1, 1)) ) 2023-05-16 18:01:15,608 - INFO - pattern_embeddings.0.token_embed.weight torch.Size([64, 3]) cuda:0 True 2023-05-16 18:01:15,608 - INFO - pattern_embeddings.0.token_embed.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,609 - INFO - enc_embed_layer.value_embedding.token_embed.weight torch.Size([64, 1]) cuda:0 True 2023-05-16 18:01:15,609 - INFO - enc_embed_layer.value_embedding.token_embed.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,609 - INFO - enc_embed_layer.daytime_embedding.weight torch.Size([1440, 64]) cuda:0 True 2023-05-16 18:01:15,609 - INFO - enc_embed_layer.weekday_embedding.weight torch.Size([7, 64]) cuda:0 True 2023-05-16 18:01:15,609 - INFO - enc_embed_layer.spatial_embedding.embedding_lap_pos_enc.weight torch.Size([64, 8]) cuda:0 True 2023-05-16 18:01:15,609 - INFO - enc_embed_layer.spatial_embedding.embedding_lap_pos_enc.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,609 - INFO - encoder_blocks.0.norm1.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,609 - INFO - encoder_blocks.0.norm1.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,610 - INFO - encoder_blocks.0.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,610 - INFO - encoder_blocks.0.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,610 - INFO - encoder_blocks.0.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,610 - INFO - encoder_blocks.0.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,610 - INFO - encoder_blocks.0.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,610 - INFO - encoder_blocks.0.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,610 - INFO - encoder_blocks.0.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,610 - INFO - encoder_blocks.0.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,611 - INFO - encoder_blocks.0.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,611 - INFO - encoder_blocks.0.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,611 - INFO - encoder_blocks.0.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,611 - INFO - encoder_blocks.0.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,611 - INFO - encoder_blocks.0.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,611 - INFO - encoder_blocks.0.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,611 - INFO - encoder_blocks.0.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,611 - INFO - encoder_blocks.0.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,611 - INFO - encoder_blocks.0.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,612 - INFO - encoder_blocks.0.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,612 - INFO - encoder_blocks.0.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,612 - INFO - encoder_blocks.0.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,612 - INFO - encoder_blocks.0.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,612 - INFO - encoder_blocks.0.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,612 - INFO - encoder_blocks.0.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,612 - INFO - encoder_blocks.0.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,612 - INFO - encoder_blocks.0.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-16 18:01:15,612 - INFO - encoder_blocks.0.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.0.norm2.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.0.norm2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.0.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.0.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.0.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.0.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.1.norm1.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.1.norm1.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.1.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.1.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,613 - INFO - encoder_blocks.1.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,614 - INFO - encoder_blocks.1.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.norm2.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.norm2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,615 - INFO - encoder_blocks.1.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.1.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.1.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.1.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.2.norm1.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.2.norm1.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.2.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.2.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.2.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.2.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.2.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.2.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,616 - INFO - encoder_blocks.2.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,617 - INFO - encoder_blocks.2.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.norm2.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.norm2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-16 18:01:15,618 - INFO - encoder_blocks.2.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.norm1.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.norm1.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,619 - INFO - encoder_blocks.3.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,620 - INFO - encoder_blocks.3.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.3.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.3.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.3.norm2.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.3.norm2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.3.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.3.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.3.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.3.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.4.norm1.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.4.norm1.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.4.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,621 - INFO - encoder_blocks.4.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,622 - INFO - encoder_blocks.4.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.norm2.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.norm2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-16 18:01:15,623 - INFO - encoder_blocks.4.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.norm1.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.norm1.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.pattern_q_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.pattern_q_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.pattern_k_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.pattern_k_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.pattern_v_linears.0.weight torch.Size([32, 64]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.pattern_v_linears.0.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.geo_q_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.geo_q_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.geo_k_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.geo_k_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.geo_v_conv.weight torch.Size([32, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.geo_v_conv.bias torch.Size([32]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.sem_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.sem_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.sem_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.sem_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,624 - INFO - encoder_blocks.5.st_attn.sem_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.st_attn.sem_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.st_attn.t_q_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.st_attn.t_q_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.st_attn.t_k_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.st_attn.t_k_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.st_attn.t_v_conv.weight torch.Size([16, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.st_attn.t_v_conv.bias torch.Size([16]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.st_attn.proj.weight torch.Size([64, 64]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.st_attn.proj.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.norm2.weight torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.norm2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.mlp.fc1.weight torch.Size([256, 64]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.mlp.fc1.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.mlp.fc2.weight torch.Size([64, 256]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - encoder_blocks.5.mlp.fc2.bias torch.Size([64]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - skip_convs.0.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - skip_convs.0.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - skip_convs.1.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,625 - INFO - skip_convs.1.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - skip_convs.2.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - skip_convs.2.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - skip_convs.3.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - skip_convs.3.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - skip_convs.4.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - skip_convs.4.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - skip_convs.5.weight torch.Size([256, 64, 1, 1]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - skip_convs.5.bias torch.Size([256]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - end_conv1.weight torch.Size([12, 12, 1, 1]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - end_conv1.bias torch.Size([12]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - end_conv2.weight torch.Size([1, 256, 1, 1]) cuda:0 True 2023-05-16 18:01:15,626 - INFO - end_conv2.bias torch.Size([1]) cuda:0 True 2023-05-16 18:01:15,627 - INFO - Total parameter numbers: 531165 2023-05-16 18:01:15,627 - INFO - You select `adamw` optimizer. 2023-05-16 18:01:15,628 - INFO - You select `cosinelr` lr_scheduler. 2023-05-16 18:01:15,628 - WARNING - Received none train loss func and will use the loss func defined in the model. 2023-05-16 18:01:15,630 - INFO - Number of isolated points: 0 2023-05-16 18:01:15,696 - INFO - Start training ... 2023-05-16 18:01:15,696 - INFO - num_batches:2280 2023-05-16 18:01:15,760 - INFO - Training: task_level increase from 0 to 1 2023-05-16 18:01:15,760 - INFO - Current batches_seen is 0 2023-05-16 18:15:09,024 - INFO - epoch complete! 2023-05-16 18:15:09,026 - INFO - evaluating now! 2023-05-16 18:15:45,550 - INFO - Epoch [0/200] (2280) train_loss: 6.1420, val_loss: 9.5395, lr: 0.000201, 869.85s 2023-05-16 18:15:45,619 - INFO - Saved model at 0 2023-05-16 18:15:45,620 - INFO - Val loss decrease from inf to 9.5395, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch0.tar 2023-05-16 18:17:06,321 - INFO - Training: task_level increase from 1 to 2 2023-05-16 18:17:06,321 - INFO - Current batches_seen is 2500 2023-05-16 18:29:40,487 - INFO - epoch complete! 2023-05-16 18:29:40,487 - INFO - evaluating now! 2023-05-16 18:30:16,108 - INFO - Epoch [1/200] (4560) train_loss: 1.6815, val_loss: 9.8495, lr: 0.000401, 870.49s 2023-05-16 18:32:57,297 - INFO - Training: task_level increase from 2 to 3 2023-05-16 18:32:57,298 - INFO - Current batches_seen is 5000 2023-05-16 18:44:10,141 - INFO - epoch complete! 2023-05-16 18:44:10,142 - INFO - evaluating now! 2023-05-16 18:44:46,358 - INFO - Epoch [2/200] (6840) train_loss: 1.3811, val_loss: 8.8201, lr: 0.000600, 870.25s 2023-05-16 18:44:46,419 - INFO - Saved model at 2 2023-05-16 18:44:46,419 - INFO - Val loss decrease from 9.5395 to 8.8201, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch2.tar 2023-05-16 18:48:48,057 - INFO - Training: task_level increase from 3 to 4 2023-05-16 18:48:48,058 - INFO - Current batches_seen is 7500 2023-05-16 18:58:40,821 - INFO - epoch complete! 2023-05-16 18:58:40,822 - INFO - evaluating now! 2023-05-16 18:59:16,571 - INFO - Epoch [3/200] (9120) train_loss: 1.4746, val_loss: 7.5734, lr: 0.000800, 870.15s 2023-05-16 18:59:16,620 - INFO - Saved model at 3 2023-05-16 18:59:16,620 - INFO - Val loss decrease from 8.8201 to 7.5734, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch3.tar 2023-05-16 19:04:38,854 - INFO - Training: task_level increase from 4 to 5 2023-05-16 19:04:38,855 - INFO - Current batches_seen is 10000 2023-05-16 19:13:11,483 - INFO - epoch complete! 2023-05-16 19:13:11,484 - INFO - evaluating now! 2023-05-16 19:13:47,456 - INFO - Epoch [4/200] (11400) train_loss: 1.5578, val_loss: 6.2103, lr: 0.000999, 870.84s 2023-05-16 19:13:47,504 - INFO - Saved model at 4 2023-05-16 19:13:47,505 - INFO - Val loss decrease from 7.5734 to 6.2103, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch4.tar 2023-05-16 19:20:29,006 - INFO - Training: task_level increase from 5 to 6 2023-05-16 19:20:29,006 - INFO - Current batches_seen is 12500 2023-05-16 19:27:40,824 - INFO - epoch complete! 2023-05-16 19:27:40,825 - INFO - evaluating now! 2023-05-16 19:28:16,715 - INFO - Epoch [5/200] (13680) train_loss: 1.6671, val_loss: 5.5062, lr: 0.000998, 869.21s 2023-05-16 19:28:16,764 - INFO - Saved model at 5 2023-05-16 19:28:16,764 - INFO - Val loss decrease from 6.2103 to 5.5062, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch5.tar 2023-05-16 19:36:20,079 - INFO - Training: task_level increase from 6 to 7 2023-05-16 19:36:20,079 - INFO - Current batches_seen is 15000 2023-05-16 19:42:11,589 - INFO - epoch complete! 2023-05-16 19:42:11,590 - INFO - evaluating now! 2023-05-16 19:42:47,715 - INFO - Epoch [6/200] (15960) train_loss: 1.7626, val_loss: 4.7724, lr: 0.000997, 870.95s 2023-05-16 19:42:47,765 - INFO - Saved model at 6 2023-05-16 19:42:47,765 - INFO - Val loss decrease from 5.5062 to 4.7724, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch6.tar 2023-05-16 19:52:12,913 - INFO - Training: task_level increase from 7 to 8 2023-05-16 19:52:12,913 - INFO - Current batches_seen is 17500 2023-05-16 19:56:43,899 - INFO - epoch complete! 2023-05-16 19:56:43,900 - INFO - evaluating now! 2023-05-16 19:57:19,904 - INFO - Epoch [7/200] (18240) train_loss: 1.8483, val_loss: 4.1567, lr: 0.000996, 872.14s 2023-05-16 19:57:19,954 - INFO - Saved model at 7 2023-05-16 19:57:19,954 - INFO - Val loss decrease from 4.7724 to 4.1567, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch7.tar 2023-05-16 20:08:04,890 - INFO - Training: task_level increase from 8 to 9 2023-05-16 20:08:04,891 - INFO - Current batches_seen is 20000 2023-05-16 20:11:14,477 - INFO - epoch complete! 2023-05-16 20:11:14,478 - INFO - evaluating now! 2023-05-16 20:11:50,326 - INFO - Epoch [8/200] (20520) train_loss: 1.9335, val_loss: 4.1311, lr: 0.000996, 870.37s 2023-05-16 20:11:50,374 - INFO - Saved model at 8 2023-05-16 20:11:50,374 - INFO - Val loss decrease from 4.1567 to 4.1311, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch8.tar 2023-05-16 20:23:55,479 - INFO - Training: task_level increase from 9 to 10 2023-05-16 20:23:55,479 - INFO - Current batches_seen is 22500 2023-05-16 20:25:45,186 - INFO - epoch complete! 2023-05-16 20:25:45,186 - INFO - evaluating now! 2023-05-16 20:26:20,922 - INFO - Epoch [9/200] (22800) train_loss: 2.0107, val_loss: 3.8494, lr: 0.000994, 870.55s 2023-05-16 20:26:20,969 - INFO - Saved model at 9 2023-05-16 20:26:20,970 - INFO - Val loss decrease from 4.1311 to 3.8494, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch9.tar 2023-05-16 20:39:45,983 - INFO - Training: task_level increase from 10 to 11 2023-05-16 20:39:45,983 - INFO - Current batches_seen is 25000 2023-05-16 20:40:15,208 - INFO - epoch complete! 2023-05-16 20:40:15,209 - INFO - evaluating now! 2023-05-16 20:40:50,902 - INFO - Epoch [10/200] (25080) train_loss: 2.0778, val_loss: 3.1183, lr: 0.000993, 869.93s 2023-05-16 20:40:50,950 - INFO - Saved model at 10 2023-05-16 20:40:50,951 - INFO - Val loss decrease from 3.8494 to 3.1183, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch10.tar 2023-05-16 20:54:45,733 - INFO - epoch complete! 2023-05-16 20:54:45,734 - INFO - evaluating now! 2023-05-16 20:55:21,374 - INFO - Epoch [11/200] (27360) train_loss: 2.1410, val_loss: 3.0148, lr: 0.000992, 870.42s 2023-05-16 20:55:21,428 - INFO - Saved model at 11 2023-05-16 20:55:21,428 - INFO - Val loss decrease from 3.1183 to 3.0148, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch11.tar 2023-05-16 20:56:12,575 - INFO - Training: task_level increase from 11 to 12 2023-05-16 20:56:12,575 - INFO - Current batches_seen is 27500 2023-05-16 21:09:12,687 - INFO - epoch complete! 2023-05-16 21:09:12,688 - INFO - evaluating now! 2023-05-16 21:09:48,311 - INFO - Epoch [12/200] (29640) train_loss: 2.1955, val_loss: 2.4845, lr: 0.000991, 866.88s 2023-05-16 21:09:48,358 - INFO - Saved model at 12 2023-05-16 21:09:48,358 - INFO - Val loss decrease from 3.0148 to 2.4845, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch12.tar 2023-05-16 21:23:41,755 - INFO - epoch complete! 2023-05-16 21:23:41,756 - INFO - evaluating now! 2023-05-16 21:24:17,422 - INFO - Epoch [13/200] (31920) train_loss: 2.1701, val_loss: 2.4204, lr: 0.000989, 869.06s 2023-05-16 21:24:17,470 - INFO - Saved model at 13 2023-05-16 21:24:17,470 - INFO - Val loss decrease from 2.4845 to 2.4204, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch13.tar 2023-05-16 21:38:10,876 - INFO - epoch complete! 2023-05-16 21:38:10,877 - INFO - evaluating now! 2023-05-16 21:38:46,653 - INFO - Epoch [14/200] (34200) train_loss: 2.1449, val_loss: 2.4278, lr: 0.000988, 869.18s 2023-05-16 21:52:40,800 - INFO - epoch complete! 2023-05-16 21:52:40,801 - INFO - evaluating now! 2023-05-16 21:53:16,431 - INFO - Epoch [15/200] (36480) train_loss: 2.1227, val_loss: 2.4205, lr: 0.000986, 869.78s 2023-05-16 22:07:09,616 - INFO - epoch complete! 2023-05-16 22:07:09,617 - INFO - evaluating now! 2023-05-16 22:07:45,443 - INFO - Epoch [16/200] (38760) train_loss: 2.1033, val_loss: 2.3595, lr: 0.000984, 869.01s 2023-05-16 22:07:45,491 - INFO - Saved model at 16 2023-05-16 22:07:45,491 - INFO - Val loss decrease from 2.4204 to 2.3595, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch16.tar 2023-05-16 22:21:38,284 - INFO - epoch complete! 2023-05-16 22:21:38,285 - INFO - evaluating now! 2023-05-16 22:22:13,827 - INFO - Epoch [17/200] (41040) train_loss: 2.0809, val_loss: 2.3248, lr: 0.000982, 868.34s 2023-05-16 22:22:13,874 - INFO - Saved model at 17 2023-05-16 22:22:13,875 - INFO - Val loss decrease from 2.3595 to 2.3248, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch17.tar 2023-05-16 22:36:07,377 - INFO - epoch complete! 2023-05-16 22:36:07,378 - INFO - evaluating now! 2023-05-16 22:36:43,059 - INFO - Epoch [18/200] (43320) train_loss: 2.0615, val_loss: 2.3307, lr: 0.000980, 869.18s 2023-05-16 22:50:37,685 - INFO - epoch complete! 2023-05-16 22:50:37,686 - INFO - evaluating now! 2023-05-16 22:51:13,458 - INFO - Epoch [19/200] (45600) train_loss: 2.0453, val_loss: 2.2941, lr: 0.000978, 870.40s 2023-05-16 22:51:13,505 - INFO - Saved model at 19 2023-05-16 22:51:13,506 - INFO - Val loss decrease from 2.3248 to 2.2941, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch19.tar 2023-05-16 23:05:05,970 - INFO - epoch complete! 2023-05-16 23:05:05,971 - INFO - evaluating now! 2023-05-16 23:05:41,536 - INFO - Epoch [20/200] (47880) train_loss: 2.0305, val_loss: 2.2664, lr: 0.000976, 868.03s 2023-05-16 23:05:41,584 - INFO - Saved model at 20 2023-05-16 23:05:41,584 - INFO - Val loss decrease from 2.2941 to 2.2664, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch20.tar 2023-05-16 23:19:36,350 - INFO - epoch complete! 2023-05-16 23:19:36,351 - INFO - evaluating now! 2023-05-16 23:20:12,128 - INFO - Epoch [21/200] (50160) train_loss: 2.0132, val_loss: 2.2610, lr: 0.000973, 870.54s 2023-05-16 23:20:12,175 - INFO - Saved model at 21 2023-05-16 23:20:12,175 - INFO - Val loss decrease from 2.2664 to 2.2610, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch21.tar 2023-05-16 23:34:07,301 - INFO - epoch complete! 2023-05-16 23:34:07,302 - INFO - evaluating now! 2023-05-16 23:34:42,942 - INFO - Epoch [22/200] (52440) train_loss: 1.9995, val_loss: 2.2296, lr: 0.000971, 870.77s 2023-05-16 23:34:43,003 - INFO - Saved model at 22 2023-05-16 23:34:43,004 - INFO - Val loss decrease from 2.2610 to 2.2296, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch22.tar 2023-05-16 23:48:38,090 - INFO - epoch complete! 2023-05-16 23:48:38,091 - INFO - evaluating now! 2023-05-16 23:49:13,883 - INFO - Epoch [23/200] (54720) train_loss: 1.9854, val_loss: 2.2307, lr: 0.000968, 870.88s 2023-05-17 00:03:08,257 - INFO - epoch complete! 2023-05-17 00:03:08,258 - INFO - evaluating now! 2023-05-17 00:03:43,444 - INFO - Epoch [24/200] (57000) train_loss: 1.9779, val_loss: 2.2211, lr: 0.000966, 869.56s 2023-05-17 00:03:43,491 - INFO - Saved model at 24 2023-05-17 00:03:43,491 - INFO - Val loss decrease from 2.2296 to 2.2211, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch24.tar 2023-05-17 00:17:39,708 - INFO - epoch complete! 2023-05-17 00:17:39,709 - INFO - evaluating now! 2023-05-17 00:18:14,936 - INFO - Epoch [25/200] (59280) train_loss: 1.9683, val_loss: 2.2709, lr: 0.000963, 871.44s 2023-05-17 00:32:14,015 - INFO - epoch complete! 2023-05-17 00:32:14,015 - INFO - evaluating now! 2023-05-17 00:32:49,374 - INFO - Epoch [26/200] (61560) train_loss: 1.9589, val_loss: 2.1978, lr: 0.000960, 874.44s 2023-05-17 00:32:49,421 - INFO - Saved model at 26 2023-05-17 00:32:49,422 - INFO - Val loss decrease from 2.2211 to 2.1978, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch26.tar 2023-05-17 00:46:48,453 - INFO - epoch complete! 2023-05-17 00:46:48,454 - INFO - evaluating now! 2023-05-17 00:47:23,585 - INFO - Epoch [27/200] (63840) train_loss: 1.9486, val_loss: 2.1839, lr: 0.000957, 874.16s 2023-05-17 00:47:23,634 - INFO - Saved model at 27 2023-05-17 00:47:23,635 - INFO - Val loss decrease from 2.1978 to 2.1839, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch27.tar 2023-05-17 01:01:19,459 - INFO - epoch complete! 2023-05-17 01:01:19,459 - INFO - evaluating now! 2023-05-17 01:01:54,780 - INFO - Epoch [28/200] (66120) train_loss: 1.9383, val_loss: 2.1941, lr: 0.000954, 871.14s 2023-05-17 01:15:53,618 - INFO - epoch complete! 2023-05-17 01:15:53,619 - INFO - evaluating now! 2023-05-17 01:16:28,775 - INFO - Epoch [29/200] (68400) train_loss: 1.9331, val_loss: 2.2043, lr: 0.000951, 873.99s 2023-05-17 01:30:26,629 - INFO - epoch complete! 2023-05-17 01:30:26,630 - INFO - evaluating now! 2023-05-17 01:31:01,952 - INFO - Epoch [30/200] (70680) train_loss: 1.9242, val_loss: 2.2516, lr: 0.000948, 873.18s 2023-05-17 01:44:59,157 - INFO - epoch complete! 2023-05-17 01:44:59,158 - INFO - evaluating now! 2023-05-17 01:45:34,587 - INFO - Epoch [31/200] (72960) train_loss: 1.9157, val_loss: 2.1706, lr: 0.000944, 872.63s 2023-05-17 01:45:34,644 - INFO - Saved model at 31 2023-05-17 01:45:34,644 - INFO - Val loss decrease from 2.1839 to 2.1706, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch31.tar 2023-05-17 01:59:31,641 - INFO - epoch complete! 2023-05-17 01:59:31,642 - INFO - evaluating now! 2023-05-17 02:00:07,036 - INFO - Epoch [32/200] (75240) train_loss: 1.9121, val_loss: 2.1828, lr: 0.000941, 872.39s 2023-05-17 02:14:03,423 - INFO - epoch complete! 2023-05-17 02:14:03,424 - INFO - evaluating now! 2023-05-17 02:14:38,757 - INFO - Epoch [33/200] (77520) train_loss: 1.9056, val_loss: 2.1502, lr: 0.000937, 871.72s 2023-05-17 02:14:38,822 - INFO - Saved model at 33 2023-05-17 02:14:38,822 - INFO - Val loss decrease from 2.1706 to 2.1502, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch33.tar 2023-05-17 02:28:36,322 - INFO - epoch complete! 2023-05-17 02:28:36,323 - INFO - evaluating now! 2023-05-17 02:29:11,663 - INFO - Epoch [34/200] (79800) train_loss: 1.8997, val_loss: 2.1762, lr: 0.000934, 872.84s 2023-05-17 02:43:08,245 - INFO - epoch complete! 2023-05-17 02:43:08,246 - INFO - evaluating now! 2023-05-17 02:43:43,356 - INFO - Epoch [35/200] (82080) train_loss: 1.8929, val_loss: 2.1455, lr: 0.000930, 871.69s 2023-05-17 02:43:43,403 - INFO - Saved model at 35 2023-05-17 02:43:43,403 - INFO - Val loss decrease from 2.1502 to 2.1455, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch35.tar 2023-05-17 02:57:31,112 - INFO - epoch complete! 2023-05-17 02:57:31,112 - INFO - evaluating now! 2023-05-17 02:58:06,218 - INFO - Epoch [36/200] (84360) train_loss: 1.8914, val_loss: 2.2126, lr: 0.000926, 862.81s 2023-05-17 03:11:52,314 - INFO - epoch complete! 2023-05-17 03:11:52,314 - INFO - evaluating now! 2023-05-17 03:12:27,361 - INFO - Epoch [37/200] (86640) train_loss: 1.8830, val_loss: 2.1460, lr: 0.000922, 861.14s 2023-05-17 03:26:13,606 - INFO - epoch complete! 2023-05-17 03:26:13,607 - INFO - evaluating now! 2023-05-17 03:26:48,747 - INFO - Epoch [38/200] (88920) train_loss: 1.8781, val_loss: 2.1369, lr: 0.000918, 861.39s 2023-05-17 03:26:48,794 - INFO - Saved model at 38 2023-05-17 03:26:48,794 - INFO - Val loss decrease from 2.1455 to 2.1369, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch38.tar 2023-05-17 03:40:35,048 - INFO - epoch complete! 2023-05-17 03:40:35,049 - INFO - evaluating now! 2023-05-17 03:41:10,160 - INFO - Epoch [39/200] (91200) train_loss: 1.8757, val_loss: 2.1640, lr: 0.000914, 861.37s 2023-05-17 03:54:57,035 - INFO - epoch complete! 2023-05-17 03:54:57,036 - INFO - evaluating now! 2023-05-17 03:55:32,192 - INFO - Epoch [40/200] (93480) train_loss: 1.8705, val_loss: 2.1320, lr: 0.000910, 862.03s 2023-05-17 03:55:32,239 - INFO - Saved model at 40 2023-05-17 03:55:32,239 - INFO - Val loss decrease from 2.1369 to 2.1320, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch40.tar 2023-05-17 04:09:19,678 - INFO - epoch complete! 2023-05-17 04:09:19,679 - INFO - evaluating now! 2023-05-17 04:09:54,837 - INFO - Epoch [41/200] (95760) train_loss: 1.8643, val_loss: 2.1617, lr: 0.000906, 862.60s 2023-05-17 04:23:41,532 - INFO - epoch complete! 2023-05-17 04:23:41,532 - INFO - evaluating now! 2023-05-17 04:24:16,451 - INFO - Epoch [42/200] (98040) train_loss: 1.8599, val_loss: 2.1853, lr: 0.000901, 861.61s 2023-05-17 04:38:03,730 - INFO - epoch complete! 2023-05-17 04:38:03,731 - INFO - evaluating now! 2023-05-17 04:38:38,742 - INFO - Epoch [43/200] (100320) train_loss: 1.8547, val_loss: 2.1437, lr: 0.000897, 862.29s 2023-05-17 04:52:26,737 - INFO - epoch complete! 2023-05-17 04:52:26,738 - INFO - evaluating now! 2023-05-17 04:53:01,766 - INFO - Epoch [44/200] (102600) train_loss: 1.8507, val_loss: 2.1166, lr: 0.000892, 863.02s 2023-05-17 04:53:01,812 - INFO - Saved model at 44 2023-05-17 04:53:01,813 - INFO - Val loss decrease from 2.1320 to 2.1166, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch44.tar 2023-05-17 05:06:49,558 - INFO - epoch complete! 2023-05-17 05:06:49,559 - INFO - evaluating now! 2023-05-17 05:07:24,561 - INFO - Epoch [45/200] (104880) train_loss: 1.8469, val_loss: 2.1393, lr: 0.000888, 862.75s 2023-05-17 05:21:12,109 - INFO - epoch complete! 2023-05-17 05:21:12,109 - INFO - evaluating now! 2023-05-17 05:21:47,143 - INFO - Epoch [46/200] (107160) train_loss: 1.8458, val_loss: 2.1486, lr: 0.000883, 862.58s 2023-05-17 05:35:34,622 - INFO - epoch complete! 2023-05-17 05:35:34,623 - INFO - evaluating now! 2023-05-17 05:36:09,641 - INFO - Epoch [47/200] (109440) train_loss: 1.8402, val_loss: 2.1275, lr: 0.000878, 862.50s 2023-05-17 05:49:57,223 - INFO - epoch complete! 2023-05-17 05:49:57,224 - INFO - evaluating now! 2023-05-17 05:50:32,287 - INFO - Epoch [48/200] (111720) train_loss: 1.8368, val_loss: 2.1223, lr: 0.000873, 862.65s 2023-05-17 06:04:19,910 - INFO - epoch complete! 2023-05-17 06:04:19,911 - INFO - evaluating now! 2023-05-17 06:04:54,820 - INFO - Epoch [49/200] (114000) train_loss: 1.8333, val_loss: 2.1259, lr: 0.000868, 862.53s 2023-05-17 06:18:41,490 - INFO - epoch complete! 2023-05-17 06:18:41,490 - INFO - evaluating now! 2023-05-17 06:19:16,434 - INFO - Epoch [50/200] (116280) train_loss: 1.8300, val_loss: 2.1208, lr: 0.000863, 861.61s 2023-05-17 06:33:03,641 - INFO - epoch complete! 2023-05-17 06:33:03,642 - INFO - evaluating now! 2023-05-17 06:33:38,629 - INFO - Epoch [51/200] (118560) train_loss: 1.8309, val_loss: 2.1078, lr: 0.000858, 862.20s 2023-05-17 06:33:38,676 - INFO - Saved model at 51 2023-05-17 06:33:38,676 - INFO - Val loss decrease from 2.1166 to 2.1078, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch51.tar 2023-05-17 06:47:26,017 - INFO - epoch complete! 2023-05-17 06:47:26,018 - INFO - evaluating now! 2023-05-17 06:48:00,987 - INFO - Epoch [52/200] (120840) train_loss: 1.8236, val_loss: 2.1036, lr: 0.000853, 862.31s 2023-05-17 06:48:01,034 - INFO - Saved model at 52 2023-05-17 06:48:01,034 - INFO - Val loss decrease from 2.1078 to 2.1036, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch52.tar 2023-05-17 07:01:47,189 - INFO - epoch complete! 2023-05-17 07:01:47,190 - INFO - evaluating now! 2023-05-17 07:02:22,223 - INFO - Epoch [53/200] (123120) train_loss: 1.8210, val_loss: 2.0906, lr: 0.000848, 861.19s 2023-05-17 07:02:22,270 - INFO - Saved model at 53 2023-05-17 07:02:22,270 - INFO - Val loss decrease from 2.1036 to 2.0906, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch53.tar 2023-05-17 07:16:08,383 - INFO - epoch complete! 2023-05-17 07:16:08,384 - INFO - evaluating now! 2023-05-17 07:16:43,427 - INFO - Epoch [54/200] (125400) train_loss: 1.8216, val_loss: 2.1170, lr: 0.000842, 861.16s 2023-05-17 07:30:29,588 - INFO - epoch complete! 2023-05-17 07:30:29,588 - INFO - evaluating now! 2023-05-17 07:31:04,653 - INFO - Epoch [55/200] (127680) train_loss: 1.8143, val_loss: 2.1087, lr: 0.000837, 861.23s 2023-05-17 07:44:51,629 - INFO - epoch complete! 2023-05-17 07:44:51,630 - INFO - evaluating now! 2023-05-17 07:45:26,746 - INFO - Epoch [56/200] (129960) train_loss: 1.8098, val_loss: 2.1113, lr: 0.000831, 862.09s 2023-05-17 07:59:13,897 - INFO - epoch complete! 2023-05-17 07:59:13,898 - INFO - evaluating now! 2023-05-17 07:59:49,006 - INFO - Epoch [57/200] (132240) train_loss: 1.8102, val_loss: 2.0985, lr: 0.000826, 862.26s 2023-05-17 08:13:36,261 - INFO - epoch complete! 2023-05-17 08:13:36,261 - INFO - evaluating now! 2023-05-17 08:14:11,416 - INFO - Epoch [58/200] (134520) train_loss: 1.8062, val_loss: 2.0996, lr: 0.000820, 862.41s 2023-05-17 08:27:58,914 - INFO - epoch complete! 2023-05-17 08:27:58,915 - INFO - evaluating now! 2023-05-17 08:28:34,079 - INFO - Epoch [59/200] (136800) train_loss: 1.8043, val_loss: 2.1167, lr: 0.000815, 862.66s 2023-05-17 08:42:21,363 - INFO - epoch complete! 2023-05-17 08:42:21,364 - INFO - evaluating now! 2023-05-17 08:42:56,530 - INFO - Epoch [60/200] (139080) train_loss: 1.7994, val_loss: 2.0866, lr: 0.000809, 862.45s 2023-05-17 08:42:56,576 - INFO - Saved model at 60 2023-05-17 08:42:56,576 - INFO - Val loss decrease from 2.0906 to 2.0866, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch60.tar 2023-05-17 08:56:43,749 - INFO - epoch complete! 2023-05-17 08:56:43,750 - INFO - evaluating now! 2023-05-17 08:57:18,951 - INFO - Epoch [61/200] (141360) train_loss: 1.7964, val_loss: 2.1088, lr: 0.000803, 862.37s 2023-05-17 09:11:05,590 - INFO - epoch complete! 2023-05-17 09:11:05,591 - INFO - evaluating now! 2023-05-17 09:11:40,470 - INFO - Epoch [62/200] (143640) train_loss: 1.7984, val_loss: 2.1438, lr: 0.000797, 861.52s 2023-05-17 09:25:27,584 - INFO - epoch complete! 2023-05-17 09:25:27,585 - INFO - evaluating now! 2023-05-17 09:26:02,601 - INFO - Epoch [63/200] (145920) train_loss: 1.7901, val_loss: 2.1244, lr: 0.000791, 862.13s 2023-05-17 09:39:50,325 - INFO - epoch complete! 2023-05-17 09:39:50,326 - INFO - evaluating now! 2023-05-17 09:40:25,368 - INFO - Epoch [64/200] (148200) train_loss: 1.7905, val_loss: 2.1085, lr: 0.000785, 862.77s 2023-05-17 09:54:13,025 - INFO - epoch complete! 2023-05-17 09:54:13,026 - INFO - evaluating now! 2023-05-17 09:54:48,002 - INFO - Epoch [65/200] (150480) train_loss: 1.7884, val_loss: 2.1377, lr: 0.000779, 862.63s 2023-05-17 10:08:35,610 - INFO - epoch complete! 2023-05-17 10:08:35,611 - INFO - evaluating now! 2023-05-17 10:09:10,558 - INFO - Epoch [66/200] (152760) train_loss: 1.7853, val_loss: 2.1008, lr: 0.000773, 862.56s 2023-05-17 10:22:56,965 - INFO - epoch complete! 2023-05-17 10:22:56,965 - INFO - evaluating now! 2023-05-17 10:23:31,931 - INFO - Epoch [67/200] (155040) train_loss: 1.7825, val_loss: 2.1053, lr: 0.000767, 861.37s 2023-05-17 10:37:18,435 - INFO - epoch complete! 2023-05-17 10:37:18,436 - INFO - evaluating now! 2023-05-17 10:37:53,532 - INFO - Epoch [68/200] (157320) train_loss: 1.7827, val_loss: 2.0982, lr: 0.000761, 861.60s 2023-05-17 10:51:42,313 - INFO - epoch complete! 2023-05-17 10:51:42,314 - INFO - evaluating now! 2023-05-17 10:52:17,394 - INFO - Epoch [69/200] (159600) train_loss: 1.7763, val_loss: 2.1192, lr: 0.000754, 863.86s 2023-05-17 11:06:06,285 - INFO - epoch complete! 2023-05-17 11:06:06,286 - INFO - evaluating now! 2023-05-17 11:06:41,390 - INFO - Epoch [70/200] (161880) train_loss: 1.7772, val_loss: 2.1660, lr: 0.000748, 863.99s 2023-05-17 11:20:29,703 - INFO - epoch complete! 2023-05-17 11:20:29,703 - INFO - evaluating now! 2023-05-17 11:21:04,797 - INFO - Epoch [71/200] (164160) train_loss: 1.7712, val_loss: 2.1008, lr: 0.000742, 863.41s 2023-05-17 11:34:52,800 - INFO - epoch complete! 2023-05-17 11:34:52,801 - INFO - evaluating now! 2023-05-17 11:35:27,932 - INFO - Epoch [72/200] (166440) train_loss: 1.7711, val_loss: 2.1248, lr: 0.000735, 863.13s 2023-05-17 11:49:16,901 - INFO - epoch complete! 2023-05-17 11:49:16,902 - INFO - evaluating now! 2023-05-17 11:49:52,092 - INFO - Epoch [73/200] (168720) train_loss: 1.7670, val_loss: 2.0882, lr: 0.000729, 864.16s 2023-05-17 12:03:43,363 - INFO - epoch complete! 2023-05-17 12:03:43,364 - INFO - evaluating now! 2023-05-17 12:04:19,420 - INFO - Epoch [74/200] (171000) train_loss: 1.7635, val_loss: 2.0958, lr: 0.000722, 867.33s 2023-05-17 12:18:27,214 - INFO - epoch complete! 2023-05-17 12:18:27,215 - INFO - evaluating now! 2023-05-17 12:19:03,868 - INFO - Epoch [75/200] (173280) train_loss: 1.7625, val_loss: 2.1180, lr: 0.000716, 884.45s 2023-05-17 12:33:14,925 - INFO - epoch complete! 2023-05-17 12:33:14,926 - INFO - evaluating now! 2023-05-17 12:33:51,592 - INFO - Epoch [76/200] (175560) train_loss: 1.7624, val_loss: 2.0894, lr: 0.000709, 887.72s 2023-05-17 12:47:59,740 - INFO - epoch complete! 2023-05-17 12:47:59,741 - INFO - evaluating now! 2023-05-17 12:48:36,104 - INFO - Epoch [77/200] (177840) train_loss: 1.7614, val_loss: 2.1040, lr: 0.000702, 884.51s 2023-05-17 13:02:40,588 - INFO - epoch complete! 2023-05-17 13:02:40,589 - INFO - evaluating now! 2023-05-17 13:03:16,945 - INFO - Epoch [78/200] (180120) train_loss: 1.7572, val_loss: 2.1399, lr: 0.000696, 880.84s 2023-05-17 13:17:20,430 - INFO - epoch complete! 2023-05-17 13:17:20,431 - INFO - evaluating now! 2023-05-17 13:17:56,828 - INFO - Epoch [79/200] (182400) train_loss: 1.7534, val_loss: 2.0870, lr: 0.000689, 879.88s 2023-05-17 13:31:59,081 - INFO - epoch complete! 2023-05-17 13:31:59,081 - INFO - evaluating now! 2023-05-17 13:32:35,349 - INFO - Epoch [80/200] (184680) train_loss: 1.7517, val_loss: 2.1126, lr: 0.000682, 878.52s 2023-05-17 13:46:43,964 - INFO - epoch complete! 2023-05-17 13:46:43,965 - INFO - evaluating now! 2023-05-17 13:47:20,321 - INFO - Epoch [81/200] (186960) train_loss: 1.7499, val_loss: 2.1120, lr: 0.000676, 884.97s 2023-05-17 14:01:26,649 - INFO - epoch complete! 2023-05-17 14:01:26,650 - INFO - evaluating now! 2023-05-17 14:02:03,016 - INFO - Epoch [82/200] (189240) train_loss: 1.7468, val_loss: 2.1199, lr: 0.000669, 882.69s 2023-05-17 14:16:08,420 - INFO - epoch complete! 2023-05-17 14:16:08,420 - INFO - evaluating now! 2023-05-17 14:16:44,657 - INFO - Epoch [83/200] (191520) train_loss: 1.7452, val_loss: 2.1142, lr: 0.000662, 881.64s 2023-05-17 14:30:49,352 - INFO - epoch complete! 2023-05-17 14:30:49,352 - INFO - evaluating now! 2023-05-17 14:31:25,432 - INFO - Epoch [84/200] (193800) train_loss: 1.7455, val_loss: 2.1162, lr: 0.000655, 880.77s 2023-05-17 14:45:28,241 - INFO - epoch complete! 2023-05-17 14:45:28,242 - INFO - evaluating now! 2023-05-17 14:46:04,309 - INFO - Epoch [85/200] (196080) train_loss: 1.7399, val_loss: 2.1086, lr: 0.000648, 878.88s 2023-05-17 15:00:06,280 - INFO - epoch complete! 2023-05-17 15:00:06,281 - INFO - evaluating now! 2023-05-17 15:00:42,352 - INFO - Epoch [86/200] (198360) train_loss: 1.7392, val_loss: 2.0793, lr: 0.000641, 878.04s 2023-05-17 15:00:42,399 - INFO - Saved model at 86 2023-05-17 15:00:42,399 - INFO - Val loss decrease from 2.0866 to 2.0793, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch86.tar 2023-05-17 15:14:42,240 - INFO - epoch complete! 2023-05-17 15:14:42,240 - INFO - evaluating now! 2023-05-17 15:15:18,252 - INFO - Epoch [87/200] (200640) train_loss: 1.7347, val_loss: 2.0783, lr: 0.000634, 875.85s 2023-05-17 15:15:18,299 - INFO - Saved model at 87 2023-05-17 15:15:18,300 - INFO - Val loss decrease from 2.0793 to 2.0783, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch87.tar 2023-05-17 15:29:19,518 - INFO - epoch complete! 2023-05-17 15:29:19,519 - INFO - evaluating now! 2023-05-17 15:29:55,474 - INFO - Epoch [88/200] (202920) train_loss: 1.7338, val_loss: 2.1070, lr: 0.000627, 877.17s 2023-05-17 15:43:56,421 - INFO - epoch complete! 2023-05-17 15:43:56,422 - INFO - evaluating now! 2023-05-17 15:44:32,378 - INFO - Epoch [89/200] (205200) train_loss: 1.7346, val_loss: 2.0908, lr: 0.000620, 876.90s 2023-05-17 15:58:34,411 - INFO - epoch complete! 2023-05-17 15:58:34,411 - INFO - evaluating now! 2023-05-17 15:59:10,635 - INFO - Epoch [90/200] (207480) train_loss: 1.7278, val_loss: 2.0817, lr: 0.000613, 878.26s 2023-05-17 16:13:16,930 - INFO - epoch complete! 2023-05-17 16:13:16,931 - INFO - evaluating now! 2023-05-17 16:13:53,088 - INFO - Epoch [91/200] (209760) train_loss: 1.7290, val_loss: 2.0902, lr: 0.000606, 882.45s 2023-05-17 16:27:58,351 - INFO - epoch complete! 2023-05-17 16:27:58,351 - INFO - evaluating now! 2023-05-17 16:28:34,289 - INFO - Epoch [92/200] (212040) train_loss: 1.7264, val_loss: 2.0866, lr: 0.000599, 881.20s 2023-05-17 16:42:39,151 - INFO - epoch complete! 2023-05-17 16:42:39,151 - INFO - evaluating now! 2023-05-17 16:43:15,296 - INFO - Epoch [93/200] (214320) train_loss: 1.7210, val_loss: 2.1437, lr: 0.000592, 881.01s 2023-05-17 16:57:17,571 - INFO - epoch complete! 2023-05-17 16:57:17,572 - INFO - evaluating now! 2023-05-17 16:57:53,465 - INFO - Epoch [94/200] (216600) train_loss: 1.7195, val_loss: 2.0910, lr: 0.000585, 878.17s 2023-05-17 17:11:55,282 - INFO - epoch complete! 2023-05-17 17:11:55,283 - INFO - evaluating now! 2023-05-17 17:12:31,190 - INFO - Epoch [95/200] (218880) train_loss: 1.7206, val_loss: 2.0787, lr: 0.000578, 877.72s 2023-05-17 17:26:32,304 - INFO - epoch complete! 2023-05-17 17:26:32,304 - INFO - evaluating now! 2023-05-17 17:27:08,109 - INFO - Epoch [96/200] (221160) train_loss: 1.7141, val_loss: 2.0896, lr: 0.000571, 876.92s 2023-05-17 17:41:09,366 - INFO - epoch complete! 2023-05-17 17:41:09,367 - INFO - evaluating now! 2023-05-17 17:41:45,162 - INFO - Epoch [97/200] (223440) train_loss: 1.7145, val_loss: 2.0827, lr: 0.000564, 877.05s 2023-05-17 17:55:46,033 - INFO - epoch complete! 2023-05-17 17:55:46,034 - INFO - evaluating now! 2023-05-17 17:56:21,906 - INFO - Epoch [98/200] (225720) train_loss: 1.7125, val_loss: 2.0895, lr: 0.000557, 876.74s 2023-05-17 18:10:20,615 - INFO - epoch complete! 2023-05-17 18:10:20,616 - INFO - evaluating now! 2023-05-17 18:10:56,391 - INFO - Epoch [99/200] (228000) train_loss: 1.7071, val_loss: 2.0814, lr: 0.000550, 874.48s 2023-05-17 18:24:57,419 - INFO - epoch complete! 2023-05-17 18:24:57,419 - INFO - evaluating now! 2023-05-17 18:25:33,224 - INFO - Epoch [100/200] (230280) train_loss: 1.7059, val_loss: 2.0693, lr: 0.000543, 876.83s 2023-05-17 18:25:33,271 - INFO - Saved model at 100 2023-05-17 18:25:33,272 - INFO - Val loss decrease from 2.0783 to 2.0693, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch100.tar 2023-05-17 18:39:33,675 - INFO - epoch complete! 2023-05-17 18:39:33,676 - INFO - evaluating now! 2023-05-17 18:40:09,496 - INFO - Epoch [101/200] (232560) train_loss: 1.7068, val_loss: 2.0860, lr: 0.000536, 876.22s 2023-05-17 18:54:10,075 - INFO - epoch complete! 2023-05-17 18:54:10,076 - INFO - evaluating now! 2023-05-17 18:54:45,825 - INFO - Epoch [102/200] (234840) train_loss: 1.7018, val_loss: 2.0875, lr: 0.000529, 876.33s 2023-05-17 19:08:46,035 - INFO - epoch complete! 2023-05-17 19:08:46,036 - INFO - evaluating now! 2023-05-17 19:09:21,868 - INFO - Epoch [103/200] (237120) train_loss: 1.6985, val_loss: 2.0918, lr: 0.000522, 876.04s 2023-05-17 19:23:21,887 - INFO - epoch complete! 2023-05-17 19:23:21,888 - INFO - evaluating now! 2023-05-17 19:23:57,741 - INFO - Epoch [104/200] (239400) train_loss: 1.6974, val_loss: 2.0792, lr: 0.000515, 875.87s 2023-05-17 19:37:53,728 - INFO - epoch complete! 2023-05-17 19:37:53,729 - INFO - evaluating now! 2023-05-17 19:38:29,591 - INFO - Epoch [105/200] (241680) train_loss: 1.6947, val_loss: 2.0851, lr: 0.000508, 871.85s 2023-05-17 19:52:26,497 - INFO - epoch complete! 2023-05-17 19:52:26,498 - INFO - evaluating now! 2023-05-17 19:53:02,400 - INFO - Epoch [106/200] (243960) train_loss: 1.6941, val_loss: 2.0836, lr: 0.000501, 872.81s 2023-05-17 20:07:00,213 - INFO - epoch complete! 2023-05-17 20:07:00,214 - INFO - evaluating now! 2023-05-17 20:07:36,172 - INFO - Epoch [107/200] (246240) train_loss: 1.6911, val_loss: 2.0833, lr: 0.000494, 873.77s 2023-05-17 20:21:34,881 - INFO - epoch complete! 2023-05-17 20:21:34,882 - INFO - evaluating now! 2023-05-17 20:22:10,759 - INFO - Epoch [108/200] (248520) train_loss: 1.6896, val_loss: 2.0751, lr: 0.000487, 874.59s 2023-05-17 20:36:08,973 - INFO - epoch complete! 2023-05-17 20:36:08,974 - INFO - evaluating now! 2023-05-17 20:36:44,834 - INFO - Epoch [109/200] (250800) train_loss: 1.6848, val_loss: 2.0802, lr: 0.000480, 874.07s 2023-05-17 20:50:42,940 - INFO - epoch complete! 2023-05-17 20:50:42,940 - INFO - evaluating now! 2023-05-17 20:51:18,704 - INFO - Epoch [110/200] (253080) train_loss: 1.6854, val_loss: 2.0953, lr: 0.000473, 873.87s 2023-05-17 21:05:08,919 - INFO - epoch complete! 2023-05-17 21:05:08,919 - INFO - evaluating now! 2023-05-17 21:05:44,958 - INFO - Epoch [111/200] (255360) train_loss: 1.6861, val_loss: 2.0919, lr: 0.000466, 866.25s 2023-05-17 21:19:21,191 - INFO - epoch complete! 2023-05-17 21:19:21,191 - INFO - evaluating now! 2023-05-17 21:19:57,606 - INFO - Epoch [112/200] (257640) train_loss: 1.6808, val_loss: 2.0879, lr: 0.000459, 852.65s 2023-05-17 21:33:59,900 - INFO - epoch complete! 2023-05-17 21:33:59,901 - INFO - evaluating now! 2023-05-17 21:34:36,339 - INFO - Epoch [113/200] (259920) train_loss: 1.6783, val_loss: 2.0895, lr: 0.000452, 878.73s 2023-05-17 21:48:38,724 - INFO - epoch complete! 2023-05-17 21:48:38,725 - INFO - evaluating now! 2023-05-17 21:49:15,330 - INFO - Epoch [114/200] (262200) train_loss: 1.6744, val_loss: 2.0912, lr: 0.000445, 878.99s 2023-05-17 22:03:17,558 - INFO - epoch complete! 2023-05-17 22:03:17,559 - INFO - evaluating now! 2023-05-17 22:03:54,218 - INFO - Epoch [115/200] (264480) train_loss: 1.6741, val_loss: 2.1169, lr: 0.000438, 878.89s 2023-05-17 22:17:57,583 - INFO - epoch complete! 2023-05-17 22:17:57,589 - INFO - evaluating now! 2023-05-17 22:18:34,137 - INFO - Epoch [116/200] (266760) train_loss: 1.6733, val_loss: 2.0685, lr: 0.000431, 879.92s 2023-05-17 22:18:34,217 - INFO - Saved model at 116 2023-05-17 22:18:34,217 - INFO - Val loss decrease from 2.0693 to 2.0685, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch116.tar 2023-05-17 22:32:26,980 - INFO - epoch complete! 2023-05-17 22:32:26,980 - INFO - evaluating now! 2023-05-17 22:33:02,926 - INFO - Epoch [117/200] (269040) train_loss: 1.6711, val_loss: 2.0782, lr: 0.000424, 868.71s 2023-05-17 22:46:57,112 - INFO - epoch complete! 2023-05-17 22:46:57,113 - INFO - evaluating now! 2023-05-17 22:47:34,005 - INFO - Epoch [118/200] (271320) train_loss: 1.6700, val_loss: 2.0847, lr: 0.000418, 871.08s 2023-05-17 23:01:28,402 - INFO - epoch complete! 2023-05-17 23:01:28,403 - INFO - evaluating now! 2023-05-17 23:02:05,229 - INFO - Epoch [119/200] (273600) train_loss: 1.6664, val_loss: 2.0803, lr: 0.000411, 871.22s 2023-05-17 23:16:00,041 - INFO - epoch complete! 2023-05-17 23:16:00,041 - INFO - evaluating now! 2023-05-17 23:16:36,959 - INFO - Epoch [120/200] (275880) train_loss: 1.6654, val_loss: 2.0658, lr: 0.000404, 871.73s 2023-05-17 23:16:37,019 - INFO - Saved model at 120 2023-05-17 23:16:37,019 - INFO - Val loss decrease from 2.0685 to 2.0658, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch120.tar 2023-05-17 23:30:31,821 - INFO - epoch complete! 2023-05-17 23:30:31,822 - INFO - evaluating now! 2023-05-17 23:31:08,694 - INFO - Epoch [121/200] (278160) train_loss: 1.6615, val_loss: 2.0541, lr: 0.000398, 871.67s 2023-05-17 23:31:08,770 - INFO - Saved model at 121 2023-05-17 23:31:08,770 - INFO - Val loss decrease from 2.0658 to 2.0541, saving to ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY_epoch121.tar 2023-05-17 23:45:02,769 - INFO - epoch complete! 2023-05-17 23:45:02,769 - INFO - evaluating now! 2023-05-17 23:45:39,585 - INFO - Epoch [122/200] (280440) train_loss: 1.6558, val_loss: 2.0611, lr: 0.000391, 870.81s 2023-05-17 23:59:34,347 - INFO - epoch complete! 2023-05-17 23:59:34,348 - INFO - evaluating now! 2023-05-18 00:00:11,210 - INFO - Epoch [123/200] (282720) train_loss: 1.6600, val_loss: 2.0771, lr: 0.000384, 871.62s 2023-05-18 00:14:04,147 - INFO - epoch complete! 2023-05-18 00:14:04,148 - INFO - evaluating now! 2023-05-18 00:14:40,521 - INFO - Epoch [124/200] (285000) train_loss: 1.6536, val_loss: 2.0622, lr: 0.000378, 869.31s 2023-05-18 00:28:36,732 - INFO - epoch complete! 2023-05-18 00:28:36,733 - INFO - evaluating now! 2023-05-18 00:29:13,031 - INFO - Epoch [125/200] (287280) train_loss: 1.6524, val_loss: 2.0747, lr: 0.000371, 872.51s 2023-05-18 00:43:16,798 - INFO - epoch complete! 2023-05-18 00:43:16,799 - INFO - evaluating now! 2023-05-18 00:43:53,091 - INFO - Epoch [126/200] (289560) train_loss: 1.6494, val_loss: 2.0735, lr: 0.000365, 880.06s 2023-05-18 00:57:47,952 - INFO - epoch complete! 2023-05-18 00:57:47,953 - INFO - evaluating now! 2023-05-18 00:58:24,235 - INFO - Epoch [127/200] (291840) train_loss: 1.6499, val_loss: 2.0730, lr: 0.000358, 871.14s 2023-05-18 01:12:24,066 - INFO - epoch complete! 2023-05-18 01:12:24,066 - INFO - evaluating now! 2023-05-18 01:12:59,897 - INFO - Epoch [128/200] (294120) train_loss: 1.6472, val_loss: 2.0647, lr: 0.000352, 875.66s 2023-05-18 01:26:55,153 - INFO - epoch complete! 2023-05-18 01:26:55,153 - INFO - evaluating now! 2023-05-18 01:27:30,838 - INFO - Epoch [129/200] (296400) train_loss: 1.6444, val_loss: 2.0793, lr: 0.000346, 870.94s 2023-05-18 01:41:02,394 - INFO - epoch complete! 2023-05-18 01:41:02,395 - INFO - evaluating now! 2023-05-18 01:41:38,290 - INFO - Epoch [130/200] (298680) train_loss: 1.6431, val_loss: 2.0859, lr: 0.000339, 847.45s 2023-05-18 01:55:12,028 - INFO - epoch complete! 2023-05-18 01:55:12,029 - INFO - evaluating now! 2023-05-18 01:55:47,827 - INFO - Epoch [131/200] (300960) train_loss: 1.6405, val_loss: 2.0828, lr: 0.000333, 849.54s 2023-05-18 02:09:38,691 - INFO - epoch complete! 2023-05-18 02:09:38,691 - INFO - evaluating now! 2023-05-18 02:10:14,985 - INFO - Epoch [132/200] (303240) train_loss: 1.6368, val_loss: 2.0670, lr: 0.000327, 867.16s 2023-05-18 02:24:05,175 - INFO - epoch complete! 2023-05-18 02:24:05,176 - INFO - evaluating now! 2023-05-18 02:24:41,468 - INFO - Epoch [133/200] (305520) train_loss: 1.6384, val_loss: 2.0803, lr: 0.000321, 866.48s 2023-05-18 02:38:32,108 - INFO - epoch complete! 2023-05-18 02:38:32,109 - INFO - evaluating now! 2023-05-18 02:39:08,394 - INFO - Epoch [134/200] (307800) train_loss: 1.6332, val_loss: 2.0753, lr: 0.000315, 866.93s 2023-05-18 02:53:00,205 - INFO - epoch complete! 2023-05-18 02:53:00,206 - INFO - evaluating now! 2023-05-18 02:53:36,494 - INFO - Epoch [135/200] (310080) train_loss: 1.6299, val_loss: 2.0762, lr: 0.000309, 868.10s 2023-05-18 03:07:28,770 - INFO - epoch complete! 2023-05-18 03:07:28,770 - INFO - evaluating now! 2023-05-18 03:08:05,000 - INFO - Epoch [136/200] (312360) train_loss: 1.6310, val_loss: 2.0664, lr: 0.000303, 868.51s 2023-05-18 03:21:54,431 - INFO - epoch complete! 2023-05-18 03:21:54,432 - INFO - evaluating now! 2023-05-18 03:22:30,737 - INFO - Epoch [137/200] (314640) train_loss: 1.6262, val_loss: 2.0756, lr: 0.000297, 865.74s 2023-05-18 03:36:21,077 - INFO - epoch complete! 2023-05-18 03:36:21,078 - INFO - evaluating now! 2023-05-18 03:36:56,998 - INFO - Epoch [138/200] (316920) train_loss: 1.6224, val_loss: 2.0740, lr: 0.000291, 866.26s 2023-05-18 03:50:42,598 - INFO - epoch complete! 2023-05-18 03:50:42,599 - INFO - evaluating now! 2023-05-18 03:51:18,293 - INFO - Epoch [139/200] (319200) train_loss: 1.6228, val_loss: 2.0650, lr: 0.000285, 861.29s 2023-05-18 04:05:04,089 - INFO - epoch complete! 2023-05-18 04:05:04,090 - INFO - evaluating now! 2023-05-18 04:05:39,881 - INFO - Epoch [140/200] (321480) train_loss: 1.6215, val_loss: 2.0630, lr: 0.000280, 861.59s 2023-05-18 04:19:25,767 - INFO - epoch complete! 2023-05-18 04:19:25,768 - INFO - evaluating now! 2023-05-18 04:20:01,517 - INFO - Epoch [141/200] (323760) train_loss: 1.6195, val_loss: 2.1005, lr: 0.000274, 861.64s 2023-05-18 04:33:47,323 - INFO - epoch complete! 2023-05-18 04:33:47,324 - INFO - evaluating now! 2023-05-18 04:34:22,833 - INFO - Epoch [142/200] (326040) train_loss: 1.6172, val_loss: 2.0658, lr: 0.000269, 861.32s 2023-05-18 04:48:09,093 - INFO - epoch complete! 2023-05-18 04:48:09,093 - INFO - evaluating now! 2023-05-18 04:48:44,707 - INFO - Epoch [143/200] (328320) train_loss: 1.6144, val_loss: 2.0692, lr: 0.000263, 861.87s 2023-05-18 05:02:17,837 - INFO - epoch complete! 2023-05-18 05:02:17,837 - INFO - evaluating now! 2023-05-18 05:02:52,848 - INFO - Epoch [144/200] (330600) train_loss: 1.6128, val_loss: 2.0725, lr: 0.000258, 848.14s 2023-05-18 05:16:08,712 - INFO - epoch complete! 2023-05-18 05:16:08,713 - INFO - evaluating now! 2023-05-18 05:16:43,826 - INFO - Epoch [145/200] (332880) train_loss: 1.6121, val_loss: 2.0680, lr: 0.000252, 830.98s 2023-05-18 05:30:01,605 - INFO - epoch complete! 2023-05-18 05:30:01,605 - INFO - evaluating now! 2023-05-18 05:30:36,782 - INFO - Epoch [146/200] (335160) train_loss: 1.6093, val_loss: 2.0608, lr: 0.000247, 832.96s 2023-05-18 05:43:52,809 - INFO - epoch complete! 2023-05-18 05:43:52,809 - INFO - evaluating now! 2023-05-18 05:44:27,849 - INFO - Epoch [147/200] (337440) train_loss: 1.6090, val_loss: 2.1063, lr: 0.000242, 831.07s 2023-05-18 05:57:48,552 - INFO - epoch complete! 2023-05-18 05:57:48,552 - INFO - evaluating now! 2023-05-18 05:58:23,755 - INFO - Epoch [148/200] (339720) train_loss: 1.6059, val_loss: 2.0572, lr: 0.000237, 835.90s 2023-05-18 06:12:21,451 - INFO - epoch complete! 2023-05-18 06:12:21,452 - INFO - evaluating now! 2023-05-18 06:12:56,954 - INFO - Epoch [149/200] (342000) train_loss: 1.6053, val_loss: 2.0595, lr: 0.000232, 873.20s 2023-05-18 06:26:45,317 - INFO - epoch complete! 2023-05-18 06:26:45,318 - INFO - evaluating now! 2023-05-18 06:27:20,771 - INFO - Epoch [150/200] (344280) train_loss: 1.6046, val_loss: 2.0791, lr: 0.000227, 863.82s 2023-05-18 06:41:05,331 - INFO - epoch complete! 2023-05-18 06:41:05,331 - INFO - evaluating now! 2023-05-18 06:41:40,864 - INFO - Epoch [151/200] (346560) train_loss: 1.5979, val_loss: 2.0599, lr: 0.000222, 860.09s 2023-05-18 06:54:55,870 - INFO - epoch complete! 2023-05-18 06:54:55,871 - INFO - evaluating now! 2023-05-18 06:55:31,155 - INFO - Epoch [152/200] (348840) train_loss: 1.5992, val_loss: 2.0651, lr: 0.000217, 830.29s 2023-05-18 07:08:47,060 - INFO - epoch complete! 2023-05-18 07:08:47,061 - INFO - evaluating now! 2023-05-18 07:09:22,163 - INFO - Epoch [153/200] (351120) train_loss: 1.5974, val_loss: 2.0681, lr: 0.000212, 831.01s 2023-05-18 07:22:38,970 - INFO - epoch complete! 2023-05-18 07:22:38,971 - INFO - evaluating now! 2023-05-18 07:23:13,814 - INFO - Epoch [154/200] (353400) train_loss: 1.5961, val_loss: 2.0632, lr: 0.000208, 831.65s 2023-05-18 07:36:30,700 - INFO - epoch complete! 2023-05-18 07:36:30,701 - INFO - evaluating now! 2023-05-18 07:37:05,290 - INFO - Epoch [155/200] (355680) train_loss: 1.5943, val_loss: 2.0565, lr: 0.000203, 831.48s 2023-05-18 07:50:18,770 - INFO - epoch complete! 2023-05-18 07:50:18,771 - INFO - evaluating now! 2023-05-18 07:50:53,320 - INFO - Epoch [156/200] (357960) train_loss: 1.5904, val_loss: 2.0657, lr: 0.000199, 828.03s 2023-05-18 08:04:09,713 - INFO - epoch complete! 2023-05-18 08:04:09,714 - INFO - evaluating now! 2023-05-18 08:04:44,256 - INFO - Epoch [157/200] (360240) train_loss: 1.5913, val_loss: 2.0644, lr: 0.000194, 830.94s 2023-05-18 08:18:14,205 - INFO - epoch complete! 2023-05-18 08:18:14,205 - INFO - evaluating now! 2023-05-18 08:18:49,082 - INFO - Epoch [158/200] (362520) train_loss: 1.5873, val_loss: 2.0810, lr: 0.000190, 844.83s 2023-05-18 08:32:20,910 - INFO - epoch complete! 2023-05-18 08:32:20,911 - INFO - evaluating now! 2023-05-18 08:32:55,738 - INFO - Epoch [159/200] (364800) train_loss: 1.5883, val_loss: 2.0798, lr: 0.000186, 846.65s 2023-05-18 08:46:34,198 - INFO - epoch complete! 2023-05-18 08:46:34,199 - INFO - evaluating now! 2023-05-18 08:47:09,175 - INFO - Epoch [160/200] (367080) train_loss: 1.5833, val_loss: 2.0764, lr: 0.000182, 853.44s 2023-05-18 09:00:55,853 - INFO - epoch complete! 2023-05-18 09:00:55,854 - INFO - evaluating now! 2023-05-18 09:01:31,148 - INFO - Epoch [161/200] (369360) train_loss: 1.5826, val_loss: 2.0744, lr: 0.000178, 861.97s 2023-05-18 09:15:23,087 - INFO - epoch complete! 2023-05-18 09:15:23,087 - INFO - evaluating now! 2023-05-18 09:15:57,878 - INFO - Epoch [162/200] (371640) train_loss: 1.5792, val_loss: 2.0719, lr: 0.000174, 866.73s 2023-05-18 09:29:48,824 - INFO - epoch complete! 2023-05-18 09:29:48,824 - INFO - evaluating now! 2023-05-18 09:30:23,352 - INFO - Epoch [163/200] (373920) train_loss: 1.5787, val_loss: 2.0644, lr: 0.000170, 865.47s 2023-05-18 09:44:15,466 - INFO - epoch complete! 2023-05-18 09:44:15,466 - INFO - evaluating now! 2023-05-18 09:44:49,961 - INFO - Epoch [164/200] (376200) train_loss: 1.5796, val_loss: 2.0624, lr: 0.000166, 866.61s 2023-05-18 09:58:42,108 - INFO - epoch complete! 2023-05-18 09:58:42,109 - INFO - evaluating now! 2023-05-18 09:59:16,632 - INFO - Epoch [165/200] (378480) train_loss: 1.5749, val_loss: 2.0590, lr: 0.000163, 866.67s 2023-05-18 10:13:08,396 - INFO - epoch complete! 2023-05-18 10:13:08,396 - INFO - evaluating now! 2023-05-18 10:13:42,927 - INFO - Epoch [166/200] (380760) train_loss: 1.5738, val_loss: 2.0624, lr: 0.000159, 866.29s 2023-05-18 10:27:34,775 - INFO - epoch complete! 2023-05-18 10:27:34,776 - INFO - evaluating now! 2023-05-18 10:28:09,382 - INFO - Epoch [167/200] (383040) train_loss: 1.5737, val_loss: 2.0596, lr: 0.000156, 866.45s 2023-05-18 10:41:59,555 - INFO - epoch complete! 2023-05-18 10:41:59,555 - INFO - evaluating now! 2023-05-18 10:42:34,140 - INFO - Epoch [168/200] (385320) train_loss: 1.5703, val_loss: 2.0679, lr: 0.000152, 864.76s 2023-05-18 10:56:32,846 - INFO - epoch complete! 2023-05-18 10:56:32,847 - INFO - evaluating now! 2023-05-18 10:57:08,163 - INFO - Epoch [169/200] (387600) train_loss: 1.5716, val_loss: 2.0674, lr: 0.000149, 874.02s 2023-05-18 11:11:09,760 - INFO - epoch complete! 2023-05-18 11:11:09,760 - INFO - evaluating now! 2023-05-18 11:11:45,145 - INFO - Epoch [170/200] (389880) train_loss: 1.5652, val_loss: 2.0562, lr: 0.000146, 876.98s 2023-05-18 11:25:47,139 - INFO - epoch complete! 2023-05-18 11:25:47,139 - INFO - evaluating now! 2023-05-18 11:26:22,519 - INFO - Epoch [171/200] (392160) train_loss: 1.5656, val_loss: 2.0601, lr: 0.000143, 877.37s 2023-05-18 11:26:22,520 - WARNING - Early stopping at epoch: 171 2023-05-18 11:26:22,520 - INFO - Trained totally 172 epochs, average train time is 831.283s, average eval time is 35.603s 2023-05-18 11:26:22,572 - INFO - Loaded model at 121 2023-05-18 11:26:22,572 - INFO - Saved model at ./libcity/cache/11302/model_cache/PDFormer_PEMSBAY.m 2023-05-18 11:26:22,617 - INFO - Start evaluating ... 2023-05-18 11:28:21,308 - INFO - Note that you select the average mode to evaluate! 2023-05-18 11:28:21,315 - INFO - Evaluate result is saved at ./libcity/cache/11302/evaluate_cache/2023_05_18_11_28_21_PDFormer_PEMSBAY_average.csv 2023-05-18 11:28:21,325 - INFO - MAE MAPE RMSE masked_MAE masked_MAPE masked_RMSE 1 0.873613 inf 1.658810 0.869416 0.016877 1.571239 2 1.013528 inf 2.034907 1.009336 0.020192 1.964113 3 1.124029 inf 2.354755 1.119844 0.022965 2.293847 4 1.214753 inf 2.623137 1.210573 0.025361 2.568609 5 1.290269 inf 2.846914 1.286094 0.027425 2.796754 6 1.354344 inf 3.034048 1.350173 0.029210 2.987044 7 1.409635 inf 3.192185 1.405468 0.030763 3.147555 8 1.457894 inf 3.326295 1.453729 0.032124 3.283493 9 1.500738 inf 3.441869 1.496577 0.033332 3.400532 10 1.539042 inf 3.541891 1.534883 0.034409 3.501742 11 1.573877 inf 3.630036 1.569720 0.035390 3.590883 12 1.606230 inf 3.710091 1.602076 0.036290 3.671806 ```

这个 step 3 的意思，不会是指前3个step的平均吧？

对。所以直接看第12行的结果就行，log有一句话，average mode。

---原始邮件--- 发件人: "Zheng @.> 发送时间: 2023年5月19日(周五) 上午10:48 收件人: @.>; 抄送: @.***>; 主题: Re: [BUAABIGSCity/PDFormer] 无法复现论文结果 (Issue #7)

这个 step 3 的意思，不会是指前3个step的平均吧？

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: @.***>

我去，这log也太反直觉了，一般都是第3第6第12步。个人建议最好在readme里说明一下。

因为其实采用的Libcity的结构，支持2类评估，也就是single和average。如果需要换成single就替换一下就行。

我一会在readme里边说明一下

设置 "mode": "average" "mode": "single"

好的，感谢

你好，请问我在自己的电脑上按作者的默认设置运行了两次模型，结果完全一致，小数点后面几位都是一致的。但是我自己写的模型每次运行结果都有偏差，请问您知道这是什么原因吗？

好的，感谢

你好，请问我在自己的电脑上按作者的默认设置运行了两次模型，结果完全一致，小数点后面几位都是一致的。但是我自己写的模型每次运行结果都有偏差，请问您知道这是什么原因吗？

你好，如果需要稳定的可重复性，建议在自己的代码中设定 random seed.

参考： https://www.kaggle.com/code/rhythmcam/random-seed-everything https://gist.github.com/ihoromi4/b681a9088f348942b01711f251e5f964

好的，感谢

你好，请问我在自己的电脑上按作者的默认设置运行了两次模型，结果完全一致，小数点后面几位都是一致的。但是我自己写的模型每次运行结果都有偏差，请问您知道这是什么原因吗？

你好，如果需要稳定的可重复性，建议在自己的代码中设定 random seed.

参考： https://www.kaggle.com/code/rhythmcam/random-seed-everything https://gist.github.com/ihoromi4/b681a9088f348942b01711f251e5f964

原来是这样，感谢指导！！

请问论文里面的结果都是用的MASK的吗

问题描述

按照 README 中的教程下载数据集，未改动任何超参，同样使用教程中给出的命令运行模型，得到的结果与论文不一致。不是变差，而是比论文要好不少。

数据集：PEMS08，PEMS04

PEMS08

我在不同的两台服务器上运行，得到了基本一致的结果。

服务器1结果：

          MAE  MAPE       RMSE  masked_MAE  masked_MAPE  masked_RMSE
1   11.744327   inf  19.637644   11.760401     0.077948    19.529140
2   11.975752   inf  20.247381   11.992254     0.079423    20.141357
3   12.196908   inf  20.769762   12.214051     0.080845    20.666855
4   12.393086   inf  21.220171   12.410814     0.082166    21.121649
5   12.565434   inf  21.609114   12.583639     0.083352    21.512920
6   12.720485   inf  21.951965   12.739080     0.084445    21.857193
7   12.865274   inf  22.262390   12.884212     0.085499    22.168737
8   13.001018   inf  22.545931   13.020285     0.086475    22.453295
9   13.128123   inf  22.803295   13.147656     0.087415    22.711014
10  13.249768   inf  23.042545   13.269598     0.088333    22.951004
11  13.386254   inf  23.260063   13.406418     0.089319    23.169426
12  13.558510   inf  23.498171   13.579021     0.090486    23.408354

手动计算第一列的非mask的mae 的均值，可以等效得到12步的总体mae。

计算结果为 12.75，比论文中标注的 13.583 要好上不少。其实只看step 12也能发现，最后一步的mae已经小于13.58了，整体算下来肯定是要小很多的。

服务器2结果：

          MAE  MAPE       RMSE  masked_MAE  masked_MAPE  masked_RMSE
1   11.807277   inf  19.672308   11.823156     0.078038    19.558163
2   12.032962   inf  20.286419   12.049309     0.079441    20.176619
3   12.252482   inf  20.817410   12.269487     0.080858    20.713125
4   12.448793   inf  21.282396   12.466464     0.082153    21.183153
5   12.624876   inf  21.685398   12.643200     0.083311    21.590067
6   12.784041   inf  22.039949   12.802795     0.084481    21.947374
7   12.932391   inf  22.358686   12.951554     0.085542    22.268145
8   13.071898   inf  22.650635   13.091407     0.086552    22.561703
9   13.202084   inf  22.917021   13.221920     0.087506    22.829172
10  13.329255   inf  23.166691   13.349422     0.088458    23.079922
11  13.472559   inf  23.394999   13.493119     0.089432    23.309145
12  13.643632   inf  23.640915   13.664682     0.090525    23.555864

同样计算得到总体mae为12.80，和服务器1基本一致。

PEMS04

只测了一次。

服务器1结果：

          MAE  MAPE       RMSE  masked_MAE  masked_MAPE  masked_RMSE
1   16.488237   inf  27.033958   16.616440     0.109217    26.958183
2   16.749134   inf  27.546349   16.873692     0.110845    27.452517
3   16.980698   inf  27.983900   17.102612     0.112179    27.875938
4   17.177589   inf  28.347172   17.297112     0.113359    28.227507
5   17.348188   inf  28.657040   17.466003     0.114348    28.527237
6   17.499729   inf  28.929235   17.615850     0.115272    28.789883
7   17.641754   inf  29.181208   17.756535     0.116133    29.032999
8   17.773027   inf  29.412554   17.886360     0.116933    29.255606
9   17.896318   inf  29.628168   18.008062     0.117671    29.462416
10  18.012926   inf  29.829189   18.122938     0.118415    29.654417
11  18.128349   inf  30.023409   18.236618     0.119193    29.839643
12  18.251507   inf  30.222811   18.357948     0.120025    30.030327

总体mae：17.50，同样比论文中给出的18.321好不少。

额外测试：PEMSBAY

我在libcity官方处下载了PEMSBAY的原子文件，放在PDFormer里也是兼容的，可以直接运行。

服务器1结果：

         MAE  MAPE      RMSE  masked_MAE  masked_MAPE  masked_RMSE
1   0.873613   inf  1.658810    0.869416     0.016877     1.571239
2   1.013528   inf  2.034907    1.009336     0.020192     1.964113
3   1.124029   inf  2.354755    1.119844     0.022965     2.293847
4   1.214753   inf  2.623137    1.210573     0.025361     2.568609
5   1.290269   inf  2.846914    1.286094     0.027425     2.796754
6   1.354344   inf  3.034048    1.350173     0.029210     2.987044
7   1.409635   inf  3.192185    1.405468     0.030763     3.147555
8   1.457894   inf  3.326295    1.453729     0.032124     3.283493
9   1.500738   inf  3.441869    1.496577     0.033332     3.400532
10  1.539042   inf  3.541891    1.534883     0.034409     3.501742
11  1.573877   inf  3.630036    1.569720     0.035390     3.590883
12  1.606230   inf  3.710091    1.602076     0.036290     3.671806

可以看到3 step mae=1.12，6 step mae=1.35，12 step mae=1.60。这个结果已经远超现在的SOTA了。

使用的超参（仿照其他数据集写的，没有刻意调）：

PEMSBAY.json

{
    "dataset_class": "PDFormerDataset",
    "input_window": 12,
    "output_window": 12,
    "train_rate": 0.7,
    "eval_rate": 0.1,
    "batch_size": 16,
    "add_time_in_day": true,
    "add_day_in_week": true,
    "step_size": 2500,
    "max_epoch": 200,
    "bidir": true,
    "far_mask_delta": 7,
    "geo_num_heads": 4,
    "sem_num_heads": 2,
    "t_num_heads": 2,
    "cluster_method": "kshape",
    "cand_key_days": 21,
    "seed": 1,
    "type_ln": "pre",
    "set_loss": "huber",
    "huber_delta": 2,
    "mode": "average"
}

我复现的结果也和你差不多，为什么会比论文中的效果要好呢

这个 step 3 的意思，不会是指前3个step的平均吧？对。所以直接看第12行的结果就行，log有一句话，average mode。

作者这不已经回复了，默认是算前x步的平均，所以应该看12行的结果，前11行都是废的。想看单步结果把average改成single。

设置 "mode": "average" "mode": "single"

BUAABIGSCity / PDFormer