Siamese networks

[kris314]

Hi,

Is it possible to implement a Siamese network using current dag branch of matconvnet? Is so could you please share some sample codes (if available) on it.

Thanks, kris314

[vedaldi]

Hi, you can implement siamese networks (we did) with simplenn as well. The trick is to pass pairs of images as consecutive entries in the sequence and then use a block such as vl_nnpdist that would pair them up in the loss.

On 29 Aug 2015, at 06:33, kris314 notifications@github.com wrote:

Hi,

Is it possible to implement a Siamese network using current dag branch of matconvnet? Is so could you please share some sample codes (if available) on it.

Thanks,

— Reply to this email directly or view it on GitHub https://github.com/vlfeat/matconvnet/issues/253.

[hyojinie]

@vedaldi Are you saying training one CNN using data where paired images are side-by-side?

[vedaldi]

Hi, almost. Simply pass paired images as consecutive indexes in the batch (e.g. images 1 and 2 in the batch are paired, then 3 and 4 and so on). Then the distance layer can take the difference between images 1 and 2 , 3 and 4 etc). Note that the output of this layer has a batch size exactly half than its input as for each pair of images you get one difference image (and then one distance value). You do get the same effect as a siamese architecture.

On 23 Sep 2015, at 20:20, hyojinie notifications@github.com wrote:

@vedaldi https://github.com/vedaldi Are you saying training one CNN using data where paired images are side-by-side?

— Reply to this email directly or view it on GitHub https://github.com/vlfeat/matconvnet/issues/253#issuecomment-142702706.

[hyojinie]

@vedaldi Let two different CNN learned through original Siamese network be f_a and f_b. Then wouldn't it be impossible to recover those functions in this implementation? For example, you might want to pre-compute features for one side before testing.

[vedaldi]

Hi, the assumption in this case is that f_a = f_b (usually this is what siamese means). Hence everything up to the layer that does the pairing would effectively be an implementation of f_a (and f_b being the latter identical).

If you want to train different f_a and f_b then you can still do it, but you need to switch to the new DAG wrapper.

On 24 Sep 2015, at 18:33, hyojinie notifications@github.com wrote:

@vedaldi https://github.com/vedaldi Let two different CNN learned through original Siamese network be f_a and f_b. Then wouldn't it be impossible to recover those functions in this implementation? For example, you might want to pre-compute features for one side before testing.

— Reply to this email directly or view it on GitHub https://github.com/vlfeat/matconvnet/issues/253#issuecomment-142997837.

[cheer37]

@vedaldi. what do you mean in pairing input images? I guess all pairs consist of two images in same class, because we have to reduce the distance between the same class and increase the distance between the different class. if i am right, it would be burden to prepare imdb. my database is structured folder by folder by classes, but not every class has the even number images, some has the odd number of images. so when pairing images, there should be pairs consisting of different class images. In my case, what can i do? Thanks in advance.

[cheer37]

nobody here?

[Mona77]

@vedaldi Thank you for your comments on building siamese network. However I get the following error in the backward path: size of derivatives backpropagated are incompatible

Which makes sense because the size of the derivative of pairing layer is 1 x 1 x d x batchSize/2. Please help me to fix this error.

Thank you.

[omair-kg]

@vedaldi Hi Andrea,

I want to implement the Triplet Siamese network given in 'Unsupervised Learning using Sequential Verification for Action Recognition' Fig. 2b

If i understand this correctly this network is just three parallel AlexNets. The output layers are modified and concatenated accordingly. For this i just need to define a dag network which consists of three alexnets in parallel. I don't understand how the parameters would be shared. Wouldn't the propagation step change the parameters differently in each network. Thank in anticipation.