Confusion about your eight_eight_way_propagation

[dontLoveBugs]

In your paper, The convolutional transformation functional with a kernel size of k for each time step t could be written as:

But I find the implementation of Eq. (1) is different in your code. According to your code, k_i, j(a, b) = k^_i, j(a, b) / ∑_a, b|k^_i, j(a, b)|, the constraint of a,b ≠ 0 should be removed. I verified my assumption by modifying your code. In your code, eight_eight_way_propagation is:

def eight_way_propagation(weight_matrix, blur_matrix, kernel):
    [batch_size, channels, height, width] = weight_matrix.size()
    avg_conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=kernel, stride=1,
                              padding=(kernel - 1) // 2,
                              bias=False)
    weight = torch.ones(1, 1, kernel, kernel)
    weight[0, 0, (kernel - 1) // 2, (kernel - 1) // 2] = 0  # kernel中心元素等于0
    avg_conv.weight = nn.Parameter(weight)
    for param in avg_conv.parameters():
        param.requires_grad = False

    sum_conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=kernel, stride=1,
                              padding=(kernel - 1) // 2,
                              bias=False)
    sum_weight = torch.ones(1, 1, kernel, kernel)
    sum_conv.weight = nn.Parameter(sum_weight)
    for param in sum_conv.parameters():
        param.requires_grad = False
    weight_sum = sum_conv(weight_matrix)
    avg_sum = avg_conv((weight_matrix * blur_matrix))
    # 计算kernel
    out = (torch.div(weight_matrix, weight_sum)) * blur_matrix + torch.div(avg_sum, weight_sum)
    return out

I modified your code as:

def eight_way_propagation_v2(weight_matrix, blur_matrix, kernel):
    [batch_size, channels, height, width] = weight_matrix.size()
    avg_conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=kernel, stride=1,
                              padding=(kernel - 1) // 2,
                              bias=False)
    weight = torch.ones(1, 1, kernel, kernel)

    avg_conv.weight = nn.Parameter(weight)
    for param in avg_conv.parameters():
        param.requires_grad = False

    sum_conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=kernel, stride=1,
                              padding=(kernel - 1) // 2,
                              bias=False)
    sum_weight = torch.ones(1, 1, kernel, kernel)
    sum_conv.weight = nn.Parameter(sum_weight)
    for param in sum_conv.parameters():
        param.requires_grad = False
    weight_sum = sum_conv(weight_matrix)
    avg_sum = avg_conv((weight_matrix * blur_matrix))

    out = torch.div(avg_sum, weight_sum)
    return out

The test code is:

if __name__ == '__main__':
    weight_matrix = torch.randn(1, 1, 228, 304)
    blur_matrix = torch.randn(1, 1, 228, 304)
    spn_kernel = 3

    d0 = eight_way_propagation(weight_matrix, blur_matrix, spn_kernel)
    d1 = eight_way_propagation_v2(weight_matrix, blur_matrix, spn_kernel)

    print(d0)
    print(d1)

I get the same output!

[yellowYuga]

so v1 or v2 ,which one is right ?or both of v1 and v2 are wrong?

[yellowYuga]

Here is my code according to the paper:

    def eight_way_propagation_v3(self, weight_matrix, blur_matrix, kernel):
        [batch_size, channels, height, width] = weight_matrix.size()
        weight_abs = torch.abs(weight_matrix)
        self.avg_conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=kernel, stride=1,
                                  padding=(kernel - 1) // 2, bias=False)
        weight = torch.ones(1, 1, kernel, kernel).cuda()
        weight[0, 0, (kernel - 1) // 2, (kernel - 1) // 2] = 0  # 中心点置为0？   #kenel= [1,1,1]
        self.avg_conv.weight = nn.Parameter(weight)  # [1,0,1]
        for param in self.avg_conv.parameters():  # [1,1,1]
            param.requires_grad = False

        weight_sum = self.avg_conv(weight_matrix)
        abs_sum=self.avg_conv(weight_abs)
        out=torch.div((abs_sum-weight_sum)*blur_matrix+self.avg_conv(weight_matrix*blur_matrix),abs_sum)

        return out

Where weight_matrix can be negative

[JenningsL]

Here is my code according to the paper:

    def eight_way_propagation_v3(self, weight_matrix, blur_matrix, kernel):
        [batch_size, channels, height, width] = weight_matrix.size()
        weight_abs = torch.abs(weight_matrix)
        self.avg_conv = nn.Conv2d(in_channels=1, out_channels=1, kernel_size=kernel, stride=1,
                                  padding=(kernel - 1) // 2, bias=False)
        weight = torch.ones(1, 1, kernel, kernel).cuda()
        weight[0, 0, (kernel - 1) // 2, (kernel - 1) // 2] = 0  # 中心点置为0？   #kenel= [1,1,1]
        self.avg_conv.weight = nn.Parameter(weight)  # [1,0,1]
        for param in self.avg_conv.parameters():  # [1,1,1]
            param.requires_grad = False

        weight_sum = self.avg_conv(weight_matrix)
        abs_sum=self.avg_conv(weight_abs)
        out=torch.div((abs_sum-weight_sum)*blur_matrix+self.avg_conv(weight_matrix*blur_matrix),abs_sum)

        return out

Where weight_matrix can be negative

Did you find any performance difference between these two version?

[pengwangucla]

True, the released version is one version around ECCV 2018 submission time. We did 9 value normalization, rather than 8 values.