search

NVlabs / GroupViT

Official PyTorch implementation of GroupViT: Semantic Segmentation Emerges from Text Supervision, CVPR 2022.
https://arxiv.org/abs/2202.11094
Other
725 stars 52 forks source link

Why is the model constrained to fixed Image sizes? #29

Open PardoAlejo opened 2 years ago

PardoAlejo commented 2 years ago

Hi guys, Thanks again for such a nice work. I want to ask you why is this assertion on the image size necessary? In case it is, any ideas on how to fix it? Thank you! https://github.com/NVlabs/GroupViT/blob/13b786155a1dfffe4703f40d028c92be58e1178d/models/group_vit.py#L596

xvjiarui commented 2 years ago

Hi @PardoAlejo

Thanks for your kind words. It's not necessarily asserted. It's just a sanity check. You may comment it out.

PardoAlejo commented 2 years ago

Thanks for your reply, @xvjiarui. Have you tried training the model with different image sizes? One would expect that larges image sizes or smaller patch sizes would give better segmentations.

xvjiarui commented 2 years ago

Hi @PardoAlejo

The smaller patch size will yield much more image tokens, and the self attention operator will require much more memory. We didn't do that since we would like to pre-train with large batch size.