search

lmb-freiburg / flownet2

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
https://lmb.informatik.uni-freiburg.de/Publications/2017/IMKDB17/
Other
1k stars 318 forks source link

Correlation Layer Fast Implementation #27

Closed sampepose closed 7 years ago

sampepose commented 7 years ago

Hi,

I'm trying to rewrite FlownetC in TensorFlow (@el3ment I think you're working on this?). Here's my current code for the correlation layer: (https://gist.github.com/sampepose/1244694a546ed173b2f38d1bb3e6a433)

It's unfortunately slow because it's so many nested loops. Also, the output feature maps don't match your Caffe model, so my logic is either incorrect or the ordering of my output differs from yours. I'm having some trouble understanding the CUDA for the correlation layer. Is there a way to calculate this using batch matrix operations?

Thank you for your help!

eddy-ilg commented 7 years ago

As far as I can see it is not possible with batch matrix operation. Would it maybe be possible to just take the CUDA code from caffe and transplant it into tensor flow? I would assume that only the interface to the data is different, but the cuda functions take a pointer anyway.