search

lmb-freiburg / flownet2

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
https://lmb.informatik.uni-freiburg.de/Publications/2017/IMKDB17/
Other
1k stars 318 forks source link

About the preprocessing before warping #10

Closed ptriantd closed 7 years ago

ptriantd commented 7 years ago

It was mentioned in a previous issue that the occluded areas are filled with 0s before warping. Is this filling with 0s performed somewhere in this code? And is the location of the occluded areas available in your datasets?

eddy-ilg commented 7 years ago

It is just the initialization with 0s. It was not entirely correct what I said, in this case we perform backward warping (warping image 2 to image 1). Since the flow field in image 1 is available for every pixel, we have a value to assign for every pixel (except for vectors pointing out of the image 2, these locations are left as 0). In the case of occlusion two different image locations from image 1 point to image 2. Both locations in image 1 will be assigned the same value after warping, but this value is only correct for one of the locations in image 1. In the other case the assigned color is wrong due to the occlusion. It is left to the network to deal with this. Some, but not all datasets include occlusion ground truth.