Figure 2 is different from code

[nachifur]

You work is great! I have some confusion. According to the code, these two connections need to be removed. According to the code, there is no need for otherwise in the second formula? Maybe my understanding is wrong, I look forward to your reply.

def build_aggasatt_joint(input,channel=64,vgg_19_path='None'):
    print("[i] Hypercolumn ON, building hypercolumn features ... ")
    vgg19_features=build_vgg19(input[:,:,:,0:3]*255.0,vgg_19_path)
    for layer_id in range(1,6):
        vgg19_f = vgg19_features['conv%d_2'%layer_id]
        input = tf.concat([tf.image.resize_bilinear(vgg19_f,(tf.shape(input)[1],tf.shape(input)[2]))/255.0,input], axis=3)

    sf=slim.conv2d(input,channel,[1,1],rate=1,activation_fn=lrelu,normalizer_fn=nm,weights_initializer=identity_initializer(),scope='g_sf')

    net0=slim.conv2d(sf,channel,[1,1],rate=1,activation_fn=lrelu,normalizer_fn=nm,weights_initializer=identity_initializer(),scope='g_conv0')
    net1=slim.conv2d(net0,channel,[3,3],rate=1,activation_fn=lrelu,normalizer_fn=nm,weights_initializer=identity_initializer(),scope='g_conv1')

    net = tf.concat([net0,net1],axis=3)
    netaggi_0 = agg(net,scope='g_aggi_0')
    netaggm_0 = agg(net,scope='g_aggm_0')

    net1 = netaggi_0 * tf.nn.sigmoid(netaggm_0)

    net2=slim.conv2d(net1,channel,[3,3],rate=2,activation_fn=lrelu,normalizer_fn=nm,weights_initializer=identity_initializer(),scope='g_conv2')
    net3=slim.conv2d(net2,channel,[3,3],rate=4,activation_fn=lrelu,normalizer_fn=nm,weights_initializer=identity_initializer(),scope='g_conv3')

    #agg
    netaggi_1 = agg(tf.concat([netaggi_0,net3,net2],axis=3),scope='g_aggi_1')
    netaggm_1 = agg(tf.concat([netaggm_0,net3,net2],axis=3),scope='g_aggm_1')

    net3 = netaggi_1 * tf.nn.sigmoid(netaggm_1)

    net4=slim.conv2d(net3,channel,[3,3],rate=8,activation_fn=lrelu,normalizer_fn=nm,weights_initializer=identity_initializer(),scope='g_conv4')
    net5=slim.conv2d(net4,channel,[3,3],rate=16,activation_fn=lrelu,normalizer_fn=nm,weights_initializer=identity_initializer(),scope='g_conv5')

    #agg
    netaggi_2 = agg(tf.concat([netaggi_1,net5,net4],axis=3),scope='g_aggi_2')
    netaggm_2 = agg(tf.concat([netaggm_1,net5,net4],axis=3),scope='g_aggm_2')

    net6 = netaggi_2 * tf.nn.sigmoid(netaggm_2)

    net6=slim.conv2d(net6,channel,[3,3],rate=32,activation_fn=lrelu,normalizer_fn=nm,weights_initializer=identity_initializer(),scope='g_conv6')
    net7=slim.conv2d(net6,channel,[3,3],rate=64,activation_fn=lrelu,normalizer_fn=nm,weights_initializer=identity_initializer(),scope='g_conv7')

    #agg
    netimg = agg(tf.concat([netaggi_1,netaggi_2,net6,net7],axis=3),scope='g_aggi_3')
    netmask = agg(tf.concat([netaggm_1,netaggm_2,net6,net7],axis=3),scope='g_aggm_3')

    netimg = spp(netimg,scope='g_imgpool')
    netmask = spp(netmask,scope='g_maskpool')

    netimg = netimg * tf.nn.sigmoid(netmask)

    netimg=slim.conv2d(netimg,3,[1,1],rate=1,activation_fn=None,scope='g_conv_img')
    netmask=slim.conv2d(netmask,1,[1,1],rate=1,activation_fn=None,scope='g_conv_mask')

    return netimg,netmask

[vinthony]

Thanks for your attention and feedback, this issue has been solved by #4 . I will check the formula ASAP.

[nachifur]

Thank you for your reply!

[vinthony]

Sorry for the reply late.

I think the otherwise situation is also necessary for the formulation.

We have solved the figure 2 issues in #4, thus, each component of the formula is necessary for our framework.

The main idea here is to use the hierarchical structure to aggregate the shadow removal branch and shadow detection(attention) branch with a shared backbone.