YihongDong / FAN

MIT License
69 stars 6 forks source link

Why does the modified transformer model and transformer model have the same code in the timing prediction folder? #1

Open liu12138666 opened 4 weeks ago

liu12138666 commented 4 weeks ago

Why does the modified transformer model and transformer model have the same code in the timing prediction folder?

YihongDong commented 4 weeks ago

We only replace the MLP in Transformer with FAN, without making any other changes to the architecture of Transformer. For further details, please refer to Timeseries_Forecasting/layers/Transformer_EncDec.py , where the experimental setting exp_setting = 0 corresponds to the standard Transformer, while exp_setting = 2 refers to the Transformer with FAN (Gated), and exp_setting = 4 represents the Transformer with FAN.