search

DAMO-DI-ML / NeurIPS2023-One-Fits-All

The official code for "One Fits All: Power General Time Series Analysis by Pretrained LM (NeurIPS 2023 Spotlight)"
470 stars 66 forks source link

RuntimeError: mat1 and mat2 shapes cannot be multiplied (2304x768 and 2048x1) #10

Closed wsfzghnr closed 10 months ago

wsfzghnr commented 11 months ago

Hello,

I am encountering a runtime error in Short-term Forecasting, specifically in the seasonal_patterns of Quarterly、Daily、Weekly andHourly, and I'm seeking some assistance to troubleshoot it. Here's the error message I'm getting:

Args in experiment: Namespace(activation='gelu', anomaly_ratio=0.25, batch_size=16, c_out=1, checkpoints='./checkpoints/', d_ff=2048, d_layers=1, d_model=768, data='m4', data_path='ETTh1.csv', dec_in=1, des='Exp', devices='0,1,2,3', distil=True, dropout=0.1, e_layers=2, embed='timeF', enc_in=1, factor=1, features='M', freq='h', gpt_layers=6, gpu=0, is_training=1, itr=1, label_len=48, learning_rate=0.001, ln=0, loss='SMAPE', lradj='type1', mask_rate=0.25, mlp=0, model='GPT4TS', model_id='m4_Quarterly', moving_avg=25, n_heads=8, num_kernels=6, num_workers=10, output_attention=False, p_hidden_dims=[128, 128], p_hidden_layers=2, patch_size=1, patience=3, percent=5, pred_len=96, root_path='./dataset/m4', seasonal_patterns='Quarterly', seq_len=96, stride=1, target='OT', task_name='short_term_forecast', top_k=5, train_epochs=10, use_amp=False, use_gpu=True, use_multi_gpu=False, weight=0) Use GPU: cuda:0

start training : short_term_forecast_m4_Quarterly_GPT4TS_m4_ftM_sl96_ll48_pl96_dm768_nh8_el2_dl1_df2048_fc1_ebtimeF_dtTrue_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>> train 24000 val 24000 Traceback (most recent call last): File "run.py", line 153, in exp.train(setting) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/exp/exp_short_term_forecasting.py", line 88, in train outputs = self.model(batch_x, None, dec_inp, None) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, kwargs) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/models/GPT4TS.py", line 77, in forward dec_out = self.forecast(x_enc, x_mark_enc, x_dec, x_mark_dec) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/models/GPT4TS.py", line 146, in forecast dec_out = self.out_layer(dec_out) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, *kwargs) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (384x768 and 2048x1) Args in experiment: Namespace(activation='gelu', anomaly_ratio=0.25, batch_size=16, c_out=1, checkpoints='./checkpoints/', d_ff=2048, d_layers=1, d_model=768, data='m4', data_path='ETTh1.csv', dec_in=1, des='Exp', devices='0,1,2,3', distil=True, dropout=0.1, e_layers=2, embed='timeF', enc_in=1, factor=1, features='M', freq='h', gpt_layers=6, gpu=0, is_training=1, itr=1, label_len=48, learning_rate=0.001, ln=0, loss='SMAPE', lradj='type1', mask_rate=0.25, mlp=0, model='GPT4TS', model_id='m4_Daily', moving_avg=25, n_heads=8, num_kernels=6, num_workers=10, output_attention=False, p_hidden_dims=[128, 128], p_hidden_layers=2, patch_size=1, patience=3, percent=5, pred_len=96, root_path='./dataset/m4', seasonal_patterns='Daily', seq_len=96, stride=1, target='OT', task_name='short_term_forecast', top_k=5, train_epochs=10, use_amp=False, use_gpu=True, use_multi_gpu=False, weight=0) Use GPU: cuda:0 start training : short_term_forecast_m4_Daily_GPT4TS_m4_ftM_sl96_ll48_pl96_dm768_nh8_el2_dl1_df2048_fc1_ebtimeF_dtTrue_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>> train 4227 val 4227 Traceback (most recent call last): File "run.py", line 153, in exp.train(setting) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/exp/exp_short_term_forecasting.py", line 88, in train outputs = self.model(batch_x, None, dec_inp, None) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(input, kwargs) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/models/GPT4TS.py", line 77, in forward dec_out = self.forecast(x_enc, x_mark_enc, x_dec, x_mark_dec) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/models/GPT4TS.py", line 146, in forecast dec_out = self.out_layer(dec_out) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, kwargs) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (672x768 and 2048x1) Args in experiment: Namespace(activation='gelu', anomaly_ratio=0.25, batch_size=16, c_out=1, checkpoints='./checkpoints/', d_ff=2048, d_layers=1, d_model=768, data='m4', data_path='ETTh1.csv', dec_in=1, des='Exp', devices='0,1,2,3', distil=True, dropout=0.1, e_layers=2, embed='timeF', enc_in=1, factor=1, features='M', freq='h', gpt_layers=6, gpu=0, is_training=1, itr=1, label_len=48, learning_rate=0.001, ln=0, loss='SMAPE', lradj='type1', mask_rate=0.25, mlp=0, model='GPT4TS', model_id='m4_Weekly', moving_avg=25, n_heads=8, num_kernels=6, num_workers=10, output_attention=False, p_hidden_dims=[128, 128], p_hidden_layers=2, patch_size=1, patience=3, percent=5, pred_len=96, root_path='./dataset/m4', seasonal_patterns='Weekly', seq_len=96, stride=1, target='OT', task_name='short_term_forecast', top_k=5, train_epochs=10, use_amp=False, use_gpu=True, use_multi_gpu=False, weight=0) Use GPU: cuda:0 start training : short_term_forecast_m4_Weekly_GPT4TS_m4_ftM_sl96_ll48_pl96_dm768_nh8_el2_dl1_df2048_fc1_ebtimeF_dtTrue_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>> train 359 val 359 Traceback (most recent call last): File "run.py", line 153, in exp.train(setting) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/exp/exp_short_term_forecasting.py", line 88, in train outputs = self.model(batch_x, None, dec_inp, None) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, *kwargs) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/models/GPT4TS.py", line 77, in forward dec_out = self.forecast(x_enc, x_mark_enc, x_dec, x_mark_dec) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/models/GPT4TS.py", line 146, in forecast dec_out = self.out_layer(dec_out) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(input, kwargs) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (624x768 and 2048x1) Args in experiment: Namespace(activation='gelu', anomaly_ratio=0.25, batch_size=16, c_out=1, checkpoints='./checkpoints/', d_ff=2048, d_layers=1, d_model=768, data='m4', data_path='ETTh1.csv', dec_in=1, des='Exp', devices='0,1,2,3', distil=True, dropout=0.1, e_layers=2, embed='timeF', enc_in=1, factor=1, features='M', freq='h', gpt_layers=6, gpu=0, is_training=1, itr=1, label_len=48, learning_rate=0.001, ln=0, loss='SMAPE', lradj='type1', mask_rate=0.25, mlp=0, model='GPT4TS', model_id='m4_Hourly', moving_avg=25, n_heads=8, num_kernels=6, num_workers=10, output_attention=False, p_hidden_dims=[128, 128], p_hidden_layers=2, patch_size=1, patience=3, percent=5, pred_len=96, root_path='./dataset/m4', seasonal_patterns='Hourly', seq_len=96, stride=1, target='OT', task_name='short_term_forecast', top_k=5, train_epochs=10, use_amp=False, use_gpu=True, use_multi_gpu=False, weight=0) Use GPU: cuda:0 start training : short_term_forecast_m4_Hourly_GPT4TS_m4_ftM_sl96_ll48_pl96_dm768_nh8_el2_dl1_df2048_fc1_ebtimeF_dtTrue_Exp_0>>>>>>>>>>>>>>>>>>>>>>>>>> train 414 val 414 Traceback (most recent call last): File "run.py", line 153, in exp.train(setting) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/exp/exp_short_term_forecasting.py", line 88, in train outputs = self.model(batch_x, None, dec_inp, None) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(*input, *kwargs) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/models/GPT4TS.py", line 77, in forward dec_out = self.forecast(x_enc, x_mark_enc, x_dec, x_mark_dec) File "/NeurIPS2023-One-Fits-All-main/Short-term_Forecasting/models/GPT4TS.py", line 146, in forecast dec_out = self.out_layer(dec_out) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1130, in _call_impl return forward_call(input, **kwargs) File "/anaconda3/envs/OFA/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 114, in forward return F.linear(input, self.weight, self.bias) RuntimeError: mat1 and mat2 shapes cannot be multiplied (2304x768 and 2048x1)

PSacfc commented 11 months ago

Thank for your reminder! It is the missing arguments for d_ff. We have modified the script file.