Why MLP's parameters set 0

CASIA-IVA-Lab / Obj2Seq

Obj2Seq: Formatting Objects as Sequences with Class Prompt for Visual Tasks (NeurIPS2022)

Other

84 stars 10 forks source link

Why MLP's parameters set 0 #5

Closed DrRyanHuang closed 1 year ago

DrRyanHuang commented 1 year ago

https://github.com/CASIA-IVA-Lab/Obj2Seq/blob/a81484ef61d692e8d2b3f88e58c6df8344a50131/models/predictors/unified_seq_head.py#L106-L114

Hi, Thanks for your helpful code! Could you please tell me why the parameters of MLP are set to 0? Why not Normal init or others?

volgachen commented 1 year ago

Hello, we just follow the existing initialization as in Deformable DETR. I suppose these initializations would make little difference. With zero-init, the model will output exactly a bounding box around the reference point before training.