Augmentation issues?

[PkuRainBow]

I find that in the original version of the FlowNet/ FlowNet-2，the data augmentation techniques include :

translation rotation scaling gaussian noise contrast multiplicative color changes

But it seems that there only exists the center crop augmentation op in the current implementations.

[fitsumreda]

we've not implemented data augmentations yet. Currently, random crop is supported. Feel free to do a PR to port the data augmentations from flownet2-caffe

[PkuRainBow]

@fitsumreda We will try to reimplement it. I recommend a torch version below(transform it to pytorch will be more convenient):

https://github.com/anuragranj/spynet/blob/master/transforms.lua

https://github.com/anuragranj/spynet/blob/master/donkey.lua

[ClementPinard]

You can find a flow augmentation here : https://github.com/ClementPinard/FlowNetPytorch/blob/master/flow_transforms.py

As for flownet2 implementation, they set up a very elegant solution, based on affine transformation matrices.

You can find how they construct the flow map here and how they construct the affine matrices here

I tried to explain the general workflow with LaTeX equations here : to view it, copy paste in e.g. stackedit

So basically,

• construct 2 affine matrices A1 and A2 the way you want (should be easy), and an identity one (to get a perfect (x,y) grid, you could also get it from constructing it with torch.arange)
• from affine_grid and grid_sample, you can easily apply such transformation : apply A1 to both I1 and FlowMap, and A2 to I2
• invert R2 from A2
• multiply every flow value by iR2 with @ and flattened Flowmap (flattend_flow = flow.view(flow.size(0), 2, -1) if considering batchwise flow augmentation)
• multiply every value of (x,y) grid by (iR2*R1 - I3)
• add these two together, along with iR2(T1 - T2) to get your final flow

It may not be the simplest way to integrate it, but this is the most flexible, because it works for every possible value of affine matrix, provided that R2 is invertible. That includes shearing, flip, and every other affine transormation.

[fitsumreda]

Thanks @PkuRainBow and @ClementPinard

[harshdeepsokhey]

Just curious if this issue has been fixed/ is required any longer. I would like to take this one up. Thanks. -H