search

happynear / caffe-windows

Configure Caffe in one hour for Windows users.
Other
1.32k stars 649 forks source link

Example of using the transformer layer #34

Open kevinlin311tw opened 8 years ago

kevinlin311tw commented 8 years ago

Hi Happynear,

I noticed that you have implemented the transformer layer recently. My understanding is that this layer is related to the spatial transformer layer from the recent arxiv "Spatial Transformer Network". It will be perfect if you could provide some examples of using this transformer layer.

Thank you Kevin

happynear commented 8 years ago

Hi @kevinlin311tw ,

It is only an experimental semi-finished realization. It only supports affine transform and suffers from a very low speed. I exploited it before I went out for business, and I don't have time to continue work on it. The codes is very easy to use. It takes a feature map blob and an affine tansform parameter blob, ouput the interpolated feature map. The affine tansform parameter can be got by a two-layer NN regression model as the paper describes.

It will be appreciated if you could take over the work, continueing to accomplish it.

ducha-aiki commented 8 years ago

@happynear, nice work! I will try to develop it further. For the begging - bug report #35 :)

wazhenzhen commented 8 years ago

Hi, @happynear , I have a question about the regression layer (localization network). I implement it with two fully-connected layers, and the outputs of the second fully-connected layer is 6. But in their paper, they claimed that a final regression layer should be appended after NN. I'm puzzled about which kind of regression layer is (like L2)? And as we cannot access to the groundtruth of transformation \theta, how can we do regression? Thanks a lot!

happynear commented 8 years ago

@wazhenzhen , The gradient of the second fc layer is from transformer_layer. The foumula of calculating the gradient is written in the paper, and that is the most important highlight of the paper. I have implemented the codes in transformer_layer.cpp and transformer_layer.cu.

wazhenzhen commented 8 years ago

@happynear , Thanks! As I'm a freshman on caffe, I'm still not clear about how to implement the whole spatial transformer network to a standard CNN. Could you share a .prototxt file to show how to use the transformer layer, it will be much clear I think. Thanks a lot!

happynear commented 8 years ago

I am busy for cvpr deadline now. After that, I will continue to work on the transformer layer.