Explain Occupancy Map

[soulslicer]

In your code you have the following lines:

                if self.cfg.occ_from_back: # True
                    occu_mask1 = 1 - get_occu_mask_backward(flow[:, 2:], th=0.2)
                    occu_mask2 = 1 - get_occu_mask_backward(flow[:, :2], th=0.2)
                else:
                    occu_mask1 = 1 - get_occu_mask_bidirection(flow[:, :2], flow[:, 2:])
                    occu_mask2 = 1 - get_occu_mask_bidirection(flow[:, 2:], flow[:, :2])

And in the training json it says:

occ_from_back": true

in one part and

           "stage1": {"epoch": 50,
                      "loss": {"occ_from_back": false,
                               "w_l1": 0.0,
                               "w_ssim": 0.0,
                               "w_ternary": 1.0}},

in the later part.

Does that mean that at epoch 50, you switch to occ_from_back to false? What is the difference between get_occu_mask_backward and get_occu_mask_bidirection? Is it because at the start we dont have any logical flow so you just use a threshold?

[lliuz]

When the optical flow predictions are totally inaccurate, the occlusion map by bidirectional reasoning will be all zeros, thus the photometric loss is invalid. To avoid this problem, some previous work set all pixels are non-occluded in the first tens of thousands of iterations.

In our implementation, we find that the way proposed in "Occlusion aware unsupervised learning of optical flow. CVPR 2018" that estimates occlusion from the backward flow can avoid this problem and generate a more accurate occlusion when the flow is inaccurate. So we adopt this way for the first 50 epochs as a warm-up and then switch to the bidirectional reasoning without stopping training.