How to understand the "training for multiple flow" and the "div_flow"

[PkuRainBow]

issue 1

As in the FlowNetC, FlowNetS, we will return a list of flow during training:

        if self.training:
            return flow2,flow3,flow4,flow5,flow6

So do we need to compute EPE loss for all the five flow predictions? If needed, could you help me point out where we do such operations for the five predicted flow maps and where have we scale the groud truth flow to match the different scales corresponding to the five predicted flow maps?

As I find that in the main.py, we only have a class like below:

        class ModelAndLoss(nn.Module):
            def __init__(self, args):
                super(ModelAndLoss, self).__init__()
                kwargs = tools.kwargs_from_args(args, 'model')
                self.model = args.model_class(args, **kwargs)
                kwargs = tools.kwargs_from_args(args, 'loss')
                self.loss = args.loss_class(args, **kwargs)

            def forward(self, data, target, inference=False ):
                output = self.model(data)

                loss_values = self.loss(output, target)

                if not inference :
                    return loss_values
                else :
                    return loss_values, output

        model_and_loss = ModelAndLoss(args)

issue 2

I find that there exist a factor "div_flow" set as 20, could you share me how to understand this value and why choose 20?

Thanks for your kind help!

[PkuRainBow]

I guess that the "div_flow" is in fact to controll the previously learned flow as the main component and keep the later flow as the residual. So the "div_flow" must be larger than 1, how to choose the value of "div_flow" is a hyperparamter choice. Do I get it right?

[JiamingSuen]

Just provide some information here, for issue1 MultiScale loss is accepting a tuple of flow predictions which should be the returned flows you mentioned. The ground truth scaling is proceeded here with AvgPool2d. For issue2 it's still not clear to me but I found this explanation in @ClementPinard 's original implementation. Maybe one of the authors would explain.

Hope this will help.

[ClementPinard]

div_flow is used for 2 reasons :

• Training is deemed more stable for low output target values.
• it was used by initial caffe code, and thus it makes this training compatible with a pretrained model from caffe.

[fitsumreda]

I will close this, as sufficient explanation was given by @ClementPinard.