Why the sampling_offset bias don't get updates during training

[Levishery]

in class MSDeformAttn reset_parameters line 69

        with torch.no_grad():
            self.sampling_offsets.bias = nn.Parameter(grid_init.view(-1))

Why the sampling_offset bias don't get updated during training? I saw that in some other implementations, this parameter would be updated.

Thanks!

[JGuillaumin]

Hi @Levishery

I asked myself the same question. I found a reason : it give to each ref point a favored direction and a favored radius :

• Refs points in head h (h in range(nb_head)) : preferred direction h *(2*Pi / nb_head)
• Then each point (4 by default ) has a preferred radio : k+1 for k in range(nb_ref_points)

I mean "favored" because it's independent of the data, it's sampling_offsets.bias. But based on the data and sampling_offsets.weight the real sampling offset might not respect the favored directions and radius.