How to evaluate the flownets

[AndrewZhao]

Sorry to bother you again. I just want to use the trained model to predict the optical flow. Could you share me the command only to predict. Thanks very much.

[ClementPinard]

I add a run_inference.py in the repo (update it!) to help you figure it out.

what it basically does, from a img pair:

• construct a tensor from each img array, don't forget to normalize values and transpose the dimension
• concatenate the tensors in the channel dimension and unsqueeze it, to get a 1x6xHxW tensor
• make it a volatile Variable
• run the network on this Variable
• convert the output back to a normal image array with flow2rgb

you can note that batch size is 1 because all images of the same batch must be the same size werehas here, there can images of the size you want (provided it doesn't make your GPU run out of memory)

Clément

[bkvie]

Tried it with the provided weight and ran into following error : Error in `python': corrupted size vs. prev_size: 0x0000557bc0334fc0

Any ideas?

[ClementPinard]

Is there a python stack trace ? Seems like a low level bug more than something related to python code. What's your pytorch version ? Are you on linux or windows ? Does it crash when giving it your own weights ?

[bkvie]

Interestingly flow result is still calculated! On linux using pytorch '0.3.1.post2'. Haven't trained weights just jet. The resulting image is several magnitudes smaller than the input image. Will the result have to be upsampled?

[ClementPinard]

This is expected as FlowNet's output size is H/4xW/4

Main motivation from original author here is that regular upsampling methods (like bilinear) doesn't yield worse results than FlowNet with up convolutions until full resolution. I'll admit that I didn't check this statement.

To check this, you can slightly modify FlowNetS architecture by adding 2 upconvolutions the same style as the others to get Flow1 and Flow0 (which would be your desired full resolution flow)

[bkvie]

right so in :

def realEPE(output, target, sparse=False): b, _, h, w = target.size() upsampled_output = nn.functional.upsample(output, size=(h,w), mode='bilinear') return EPE(upsampled_output, target, sparse, mean=True)

bilinear upsampling is performed. I could just save THAT result for presentation purposes. I will perform convoluational upsampling and report back.