search

romilbert / samformer

Official implementation of SAMformer, a transformer leveraging Sharpness-Aware Minimization and Channel-Wise Attention for Time Series Forecasting.
MIT License
131 stars 18 forks source link

could you share the code of random transformer mentioned in the paper #12

Closed unihe closed 1 week ago

unihe commented 3 months ago

really interested about the implemention of it

romilbert commented 3 months ago

Thanks for your feedback. To use random_transformer, you can pass transformer_random as an argument to --model. This will automatically set the trainable argument of the transformer to False in the initialize_model function.

unihe commented 3 months ago

Thanks for rapdly relpy. Also, I wanna consult that if all the parameters in attention layer are frozen when use random_transformer since self.attention_layer.trainable = trainable (which is false in random_transformer) in the code I found.