How ROI proposal works when shift to new class in few shot learning?

[HochCC]

Hi@jshtok, thanks for sharing your work! But one question is really confusing for me, How can ROI proposal still work well when shift to object from new categories in few shot learning phase? In the normal Faster R-CNN, the ROI proposal layer generate proposals by classifying fore-/background, when a new category comes (with high visual difference), I think it may classified as background. And in your code, it seems your ROI proposal layer is pretty the same as normal Faster R-CNN. (maybe you can tell me where is the difference)

In the paper, you start from pooled feature vector directly as in below picture.

So how can the network generate good proposals for new categories in few shot learning with out fine-turning?

thanks!

[jshtok]

Hi,

The assumption is that new categories come from the same visual domain as the training categories. Also, the set of training categories should be rich to contain many objects. Then the RPN works well

Your remark is justified. When we tried to use new categories radically different from training data, RPN was not generating good proposals.

Regards, Joseph

On Sat, 28 Dec 2019, 20:37 XinjieInformatik, notifications@github.com wrote:

Hi@jshtok, thanks for sharing your work! But one question is really confusing for me, How can ROI proposal still work well when shift to object from new categories in few shot learning phase? In the normal Faster R-CNN, the ROI proposal layer generate proposals by classifying fore-/background, when a new category comes (with high visual difference), I think it may classified as background. And in your code, it seems your ROI proposal layer is pretty the same as normal Faster R-CNN. (maybe you can tell me where is the difference)

In the paper, you start from pooled feature vector directly as in below picture. [image: Screenshot from 2019-12-28 19-24-40] https://user-images.githubusercontent.com/20421610/71547997-dea73a00-29a7-11ea-9296-29f681cfb2b2.png

So how can the network generate good proposals for new categories in few shot learning with out fine-turning?

thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jshtok/RepMet/issues/20?email_source=notifications&email_token=ACOBU6TJQZU2MA7CHP2VJ5TQ26MGJA5CNFSM4KAQS3K2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IDBS7GA, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACOBU6T3V7IHCAZQ3JZRREDQ26MGJANCNFSM4KAQS3KQ .

[HochCC]

Hi Joseph, thanks for your very clear reply!

Best, Xinjie