Numbers of heads in MHSA?

leaderj1001 / BottleneckTransformers

Bottleneck Transformers for Visual Recognition

MIT License

274 stars 50 forks source link

Open Hanqer opened 3 years ago

Hanqer commented 3 years ago

It seems that MHSA only has one head in the released code. But in the paper, 4 heads are used in MHSA. Is it a simplification for CIFAR dataset?