how does the auto masking work int the code?

gaosanyuan commented 2 years ago

I have read the code, but I still didn't know how the auto masking work in the code. I only found the mask is computed here:

if not self.opt.disable_automasking: outputs["identity_selection/{}".format(scale)] = ( idxs > identity_reprojection_loss.shape[1] - 1).float()

But, I don't know where it works.

Can anyone explain it for me, thanks

kebijuelun commented 2 years ago

firstly, pred(reprojected image) will be compared to target image

reprojection_losses.append(self.compute_reprojection_loss(pred, target))

then, pred(source image) will be compared to target image

identity_reprojection_losses.append(
                    self.compute_reprojection_loss(pred, target))

choose minimal loss between identity_reprojection_losses and reprojection_losses, that makes the similar regions(between source image and target image) will be set to identity_reprojection_losses, the pixels set to identity_reprojection_losses will be discard area in auto masking process
```
combined = torch.cat((identity_reprojection_loss, reprojection_loss), dim=1)
to_optimise, idxs = torch.min(combined, dim=1)
```

dantkz commented 2 years ago

nianticlabs / monodepth2