search

erikalu / class-agnostic-counting

63 stars 14 forks source link

Need to understand intuition behind the concat + matching_net steps #8

Open aurotripathy opened 3 years ago

aurotripathy commented 3 years ago

@erikalu or others

Could you please provide the intuition behind why the two steps below essentially perform a "learnt cross-correlation" with the exemplar patch. 

    # ==> concatenate exemplar and image features    
    outputs = keras.layers.Concatenate(axis=-1)([exemplar, image_f])
    # ==> matching module    
    outputs = matching_net(outputs)

Also, can the matching_net take up a deeper u-net like structure?
Note, I understand the broadcast step prior to the steps above.

erikalu commented 3 years ago

The matching_net is a network that takes as input the concatenated features for the exemplar patch and image. So the matching_net can therefore be used to compare the exemplar patch and image, or in other words, act as a learned similarity function.

The architecture of the matching_net can be modified to have more layers, yes.