search

dongliangcao / OCFlow

A neural network for self-supervised learning of optical flow and occlusion mask
3 stars 0 forks source link

Issues #4

Closed mosjoker closed 3 years ago

mosjoker commented 3 years ago

Hi，good job. I have some questions.

  1. Correlation or Cost Volume，Which is better? or consider the dataset? Have you tried adding Normalization in front of Cost Volume like What Matters in Unsupervised Optical Flow?
  2. Occlusion Estimation: Related approaches estimate occlusions by (i)learning a model for occlusion estimation，(ii)checking for consistent forward and backward flow, (iii) using the range map of the backward flow. I only use (ii)，I don't understand the meaning of learning a model for occlusion estimation(self-supervised manner)?
  3. Have you tried some joint training? (such as semantic segmentation)
dongliangcao commented 3 years ago

Hi, thanks for your questions. I will answer your questions as you listed.

  1. The Correlation and Cost Volume calculate the correlation layer in different way. Currently, we are using the Correlation, which is in consistent with the original PWC-Net. We didn't try the normalization in front of Cost Volume, but we also want to have a look at the paper: What Matters in Unsupervised Optical Flow to find a better way to train the optical flow network in an unsupervised way.
  2. Occlusion Estimation: Our work wants to train a network to estimate the occlusion map rather than derived from backward flow like (ii) and (iii)
  3. Unfortunately, no. We are also beginner in this area, and our project is still in progress.