search

lmb-freiburg / flownet2

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
https://lmb.informatik.uni-freiburg.de/Publications/2017/IMKDB17/
Other
1k stars 318 forks source link

Large transformations in data augmentation #30

Closed sampepose closed 7 years ago

sampepose commented 7 years ago

Hi there,

Thank you for your time with my previous questions. I attempted to rewrite the data augmentation code in TensorFlow.

Here are my original two images along with the corresponding flow:

Image 0 Image 1 Flow
img0 img1 flow0

I then augment only with an x,y translation. I'm using the same uniform_bernoulli / gaussian_bernoulli distribution options as in the models for FlowNetS.

Image 0 Image 1 Flow
img0_aug img1_aug flow_aug

This seems like a large amount of information lost from such a huge translation. Half of the flow field is gone. Is this size of augmentation normal? As an extreme, sometimes I end up with an empty, noisy image after all of the transformations.

screenshot 2017-06-13 02 29 32

Thanks for your help!

Sam

nikolausmayer commented 7 years ago

Hi Sam,

you're bound to lose information with a shift augmentation, there's no way around that. However, the FlowNet trained on crops, not whole images. If you crop out a smaller region from both images, you can safely apply this geometric stuff without losing information in the visible part (of course it gets a little harder because the allowed augmentations depend on each input's crop settings).

Best, Nikolaus