sampling points

[zuo-ux]

How to get the sampling points? Below are the related code.

import torch import torch.nn as nn import torch.nn.functional as F

M=1 K=*

class DFMAtt(nn.Module): def init(self, in_ch,out_ch, k): super().init() self.conv = nn.Conv2d(in_ch, out_ch, 1, 1, 0,bias=True) self.k=k self.out_ch=out_ch offset_list=[] for x in range(k): conv = nn.Conv2d(in_ch, 2, 1, 1, 0,bias=True) offset_list.append(conv) self.offset_conv=nn.ModuleList(offset_list) self.weight_conv= nn.Sequential(nn.Conv2d(in_ch, k, 1, 1, 0,bias=True),nn.Softmax(1))

def forward(self,input):
    b, c, h, w = input.size()
    proj_feat=self.conv(input)
    offsets=[]
    for x in range(self.k):
        flow = self.offset_conv[x](input)
        offsets.append(flow)
    offsetweights= torch.repeat_interleave(self.weight_conv(input),self.out_ch,1)
    feats=[]
    for x in range(self.k):
        flow=offsets[x]
        flow = flow.permute(0, 2, 3, 1)
        grid_y, grid_x = torch.meshgrid(torch.arange(0, h), torch.arange(0, w))
        grid = torch.stack((grid_x, grid_y), 2).float()
        grid.requires_grad = False
        grid = grid.type_as(proj_feat)
        vgrid = grid + flow
        vgrid_x = 2.0 * vgrid[:, :, :, 0] / max(w - 1, 1) - 1.0
        vgrid_y = 2.0 * vgrid[:, :, :, 1] / max(h - 1, 1) - 1.0
        vgrid_scaled = torch.stack((vgrid_x, vgrid_y), dim=3)
        feat = F.grid_sample(proj_feat, vgrid_scaled, mode='bilinear', padding_mode='zeros')
        feats.append(feat)
    feat=torch.cat(feats,1)*offsetweights
    feat= sum(torch.split(feat,self.out_ch,1))
    return feat

[zuo-ux]

Below are the related code. https://github.com/Windaway/Deformable-Attention-for-Deformable-DETR/blob/main/DFMAtt.py

[duongnv0499]

You can get sampling point and attention weight here:

https://github.com/fundamentalvision/Deformable-DETR/blob/11169a60c33333af00a4849f1808023eba96a931/models/ops/modules/ms_deform_attn.py#L98-L111

[duongnv0499]

and if you want normalized coordinate (range in [0, 1]) of sampling points, get sampling_locations after L111

https://github.com/fundamentalvision/Deformable-DETR/blob/11169a60c33333af00a4849f1808023eba96a931/models/ops/modules/ms_deform_attn.py#L98-L111

[zuo-ux]

Thanks！Can you leave your WeChat? I have a few questions to consult you.

[duongnv0499]

I cannot visualize it yet. I plotted the sampling points in here https://github.com/duongnv0499/Explain-Deformable-DETR/blob/master/abcd.png but it seems to be wrong. Im on the way to fix it.

[zuo-ux]

Okay, I am currently applying deformable attention to semantic segmentation, and the code has run successfully.

[yairkit]

I cannot visualize it yet. I plotted the sampling points in here https://github.com/duongnv0499/Explain-Deformable-DETR/blob/master/abcd.png but it seems to be wrong. Im on the way to fix it.

Hi, is there a working visualization code for the deformable DETR by any chance? Thanks!

[sharat29ag]

I cannot visualize it yet. I plotted the sampling points in here https://github.com/duongnv0499/Explain-Deformable-DETR/blob/master/abcd.png but it seems to be wrong. Im on the way to fix it.

Hi, any success? Thanks

[quxu91]

I cannot visualize it yet. I plotted the sampling points in here https://github.com/duongnv0499/Explain-Deformable-DETR/blob/master/abcd.png but it seems to be wrong. Im on the way to fix it.

Hi, have you success?

[sharat29ag]

I cannot visualize it yet. I plotted the sampling points in here https://github.com/duongnv0499/Explain-Deformable-DETR/blob/master/abcd.png but it seems to be wrong. Im on the way to fix it.

Hi, have you success?

Not yet.

[quxu91]

I cannot visualize it yet. I plotted the sampling points in here https://github.com/duongnv0499/Explain-Deformable-DETR/blob/master/abcd.png but it seems to be wrong. Im on the way to fix it.

Hi, have you success?

Not yet.

Do you have the code of the picture generated above mentioned?

[2015wan]

@quxu91 @sharat29ag @yairkit Regarding Figure 5 and Figure 6 in the paper, have you had a successful code visualization?

[hwuidue]

@quxu91 @sharat29ag @yairkit Regarding Figure 5 and Figure 6 in the paper, have you had a successful code visualization?

hey bro ,have you success? thank you very much!