YihongDong / FAN

MIT License
114 stars 10 forks source link

Why does the modified transformer model and transformer model have the same code in the timing prediction folder? #1

Open liu12138666 opened 1 month ago

liu12138666 commented 1 month ago

Why does the modified transformer model and transformer model have the same code in the timing prediction folder?

YihongDong commented 1 month ago

We only replace the MLP in Transformer with FAN, without making any other changes to the architecture of Transformer. For further details, please refer to Timeseries_Forecasting/layers/Transformer_EncDec.py , where the experimental setting exp_setting = 0 corresponds to the standard Transformer, while exp_setting = 2 refers to the Transformer with FAN (Gated), and exp_setting = 4 represents the Transformer with FAN.