search

beanie00 / Decision-ConvFormer

[ICLR 2024 Spotlight] Code for the paper "Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making"
https://www.beanie00.com/publications/dc
MIT License
6 stars 1 forks source link

details question about attention map in Figure.3 (a) #2

Open dbdx-zwh opened 2 months ago

dbdx-zwh commented 2 months ago

Congratulations for publishing your work in ICLR2024! I have seen your figure.3(a) said original DT cannot focus on local information. However, I run your released code with your env, and my attention heatmap in DT is similar to figure.3(b). Can you release more details (code is better) to reproduce the heatmap in you figure 3?

beanie00 commented 1 month ago

Thank you for your interest in our work.

Regarding the attention map in Figure 3(a), we visualized it during the evaluation phase with a single batch size in the hopper-medium environment. Additionally, we used the default hyperparameters from https://github.com/kzl/decision-transformer (note that the default hyperparameters in our code differ slightly in terms of activation function and learning rate).