search

naver / r2d2

Other
461 stars 86 forks source link

Learn from Correspondences instead of aflow #26

Closed ShrutheeshIR closed 3 years ago

ShrutheeshIR commented 3 years ago

Hello,

While training the reliability AP loss, is it possible to feed correspondences instead of aflow? This is because the optical flow is mostly dense while the correspondences are sparse in nature. Have you tried using correspondences to train the network? Would this affect the sampler and the AP loss? Thanks!

humenbergerm commented 3 years ago

Hi! We have not tried that extensively (unless I missed it) but we would be interested in the performance. Best, Martin

jerome-revaud commented 3 years ago

Hi. Same answer. Actually, R2D2 was really meant from the start to be trained from dense correspondences. I guess it would be possible to train it from sparse correspondences, because there is a correspondence validity mask anyway that indicates which correspondence is valid or not, but we never trained it this way. I am not sure it would converge as well as before, because there would be much less samples in the end...

hichem-abdellali commented 3 years ago

Hi @ShrutheeshIR

I would be curious to find out if you managed to train your network from sparse correspondences? Thanks

ShrutheeshIR commented 3 years ago

Hi @djidan10 no I did not get to it. I realized that it might be counter intuitive to train using sparse correspondences, since the problem would cease to be self-supervised in some sense, and be more supervised.

@humenbergerm and @jerome-revaud thank you for your response, apologies I must have missed it! Thanks