Attention mask in last Swin basic layer

[shkarupa-alex]

In the original Swin implementation last BasicLayer with 2 SwinTransformerBlock's does not uses attention mask:

• first: due to condition https://github.com/microsoft/Swin-Transformer/blob/main/models/swin_transformer.py#L192
• second: due to shift size equals 0

But your SwinTransformerBlock implementation does not uses such condition and first SwinTransformerBlock will be computed WITH attention mask.

Is it an error, or you did this on purpose? Will it harm the performance or boost it?

[bowenc0221]

The Swin Transformer backbone is from official implementation for semantic segmentation without any modification:

• https://github.com/SwinTransformer/Swin-Transformer-Semantic-Segmentation/blob/main/mmseg/models/backbones/swin_transformer.py

I would suggest redirect your question to the original Swin Transformer authors.