search

OpenGVLab / unmasked_teacher

[ICCV2023 Oral] Unmasked Teacher: Towards Training-Efficient Video Foundation Models
https://arxiv.org/abs/2303.16058
MIT License
267 stars 13 forks source link

Wrong encoder width in bert_large #27

Closed HyeongminLEE closed 7 months ago

HyeongminLEE commented 7 months ago

Hi, I found that the encoder width option in https://github.com/OpenGVLab/unmasked_teacher/blob/main/multi_modality/configs/config_bert_large.json has been set to be 768. However, as far as I know, the encoder width of bert_large should be 1024. Please check this again.

Thanks!

Andy1621 commented 7 months ago

Actually, this is a placeholder, and the value will be changed in the code https://github.com/OpenGVLab/unmasked_teacher/blob/f6e0b283be2f544c83c7d1f767d5b406b0ec464c/multi_modality/models/backbones/bert/builder.py#L18C5-L18C30