In lib.models/ops/modules/projattn.py, I noticed that the weights of self.sampling_offsets is set to constant 0, and the bias has no gradient backpropagation (line 94- line 105).
In my opinion, if the weights are set to 0 and the bias has no gradient, the sampled offsets will be always the same across different training samples. But it seems the offsetted points are informatively selected according to Figure 5 in your paper.
On the other hand, in the provided pretrained model, the weights and the bias are different from what they are initialized.
Could you please tell me what is the final initialization method of self.sampling_offsets?
Thank you very much!
Dear authors,
In lib.models/ops/modules/projattn.py, I noticed that the weights of self.sampling_offsets is set to constant 0, and the bias has no gradient backpropagation (line 94- line 105).
https://github.com/sail-sg/mvp/blob/80eecd012f51f49da357e337716d40a6398d520d/lib/models/ops/modules/projattn.py#L94
In my opinion, if the weights are set to 0 and the bias has no gradient, the sampled offsets will be always the same across different training samples. But it seems the offsetted points are informatively selected according to Figure 5 in your paper.
On the other hand, in the provided pretrained model, the weights and the bias are different from what they are initialized. Could you please tell me what is the final initialization method of self.sampling_offsets? Thank you very much!