search

Project-MONAI / MONAI

AI Toolkit for Healthcare Imaging
https://monai.io/
Apache License 2.0
5.71k stars 1.04k forks source link

Unable to Set the bias term for the QKV linear layer in SAB #5148

Closed a-parida12 closed 2 years ago

a-parida12 commented 2 years ago

Describe the bug I pretrained a ViT using GMML method(https://github.com/Sara-Ahmed/GMML). I would like to be able to use this pretrained model as the ViT for UNETr. For every layer I am unable to load checkpoints forvit.blocks.1.attn.qkv.bias.

This largely due to on line https://github.com/Project-MONAI/MONAI/blob/4a38e1150ab098811a0985d9f53327bc3f70e30c/monai/networks/blocks/selfattention.py#L45 The bias term has been set False and their no way to access it from the higher level UnetR.

a-parida12 commented 2 years ago

I put a PR with my local changes that allows me to rectify this issue. I hope it is okay to merge it to dev.