search

Qualcomm-AI-research / InverseForm

BSD 3-Clause Clear License
144 stars 21 forks source link

I have trained stn with resnet using the ImageNet , then how to train inverseForm net #8

Open Liupengshuaige opened 3 years ago

Liupengshuaige commented 3 years ago

Hi! I have read your awesome paper and the appendices, and I have a few questions:

  1. Which dataset is used to train the stn ? ImageNet or mnist. if I use ImageNet to train stn, how many times should I down sample to get the feature before flatten the feature, and how about the learning rate should I choose in stn, is it the same with classification net?

2.if I have trained stn with resnet using the ImageNet , the how to train inverseForm net, In my opinion it looks like this: a) first get the rgb input image from ImageNet, then get the gray edges using sobel filter.
b) Freeze the stn, Using rgb input image to get the theta and the affine matrix, and use gray edges to get the affine edges by the affine matrix. c) Get the theta hat from the inverseForm Net by using the gray edges and the affine gray edges. Then get the loss from theta and theta hat Is that correct?

  1. Does it matter if I use different net(resnet or mobilenet) to train the stn net?
  2. Whats's the loss function between the theta from stn and the theta hat from inverseForm, L2? or L1 Looking forward to your reply. Thank you!
yeshwanth95 commented 3 years ago

Hi @Liupengshuaige . I am also presently trying to train the InverseForm Net on a custom dataset of mine. I was wondering if you could share more details on how you trained the STN with resnet on Imagenet? I would appreciate it if you could point me to some resources. Even better, if you could also share the pretrained STN weights it would save me a great deal of time.

Looking forward to your reply. Thank you.

zzx0836 commented 2 years ago

Hi @yeshwanth95 could you tell me how to train inverseForm net in my datasets?