Flow returning 0

[CCareaga]

I'm trying to use this library but when I run my images through I get all zeros... Here is my code:

f0 = cv2.cvtColor(cv2.imread("frames/frame7.png"), cv2.COLOR_BGR2RGB).astype(np.float32) / 255.
f1 = cv2.cvtColor(cv2.imread("frames/frame8.png"), cv2.COLOR_BGR2RGB).astype(np.float32) / 255.

optflow = farneback3d.Farneback(
        pyr_scale=0.9,         # Scaling between multi-scale pyramid levels
        levels=8,              # Number of multi-scale levels
        num_iterations=5,      # Iterations on each multi-scale level
        winsize=9,             # Window size for Gaussian filtering of polynomial coefficients
        poly_n=5,              # Size of window for weighted least-square estimation of polynomial coefficients
        poly_sigma=1.2,        # Sigma for Gaussian weighting of least-square estimation of polynomial coefficients
)

flow = optflow.calc_flow(f0, f1)
print(flow.max())

This prints:

Scale 8
Scale 7
Scale 6
Scale 5
Scale 4
Scale 3
Scale 2
Scale 1
Scale 0
0.0

Is it maybe a problem with how I'm loading the images? Or am I mistaken and this is not supposed to be used on images? I am slightly confused because you stated that this repo is based off the OpenCV implementation, which does work on images...

[theHamsta]

Hi! The framework is made for volumes not for 2d images. I should really yield a warning if someone tries to apply it on images (it thinks your y,x,channel image is a z,y,x volume)

[CCareaga]

Ahhh that makes sense, I was skeptical about it being for images but wasn't sure. It is very hard to find a GPU implementation of optical flow that is written in Python...

Thank you!

[theHamsta]

Did you try cppimport? It's usally easier to just call a C++ implementation of an optical flow (e.g. from OpenCV). pybind11 makes it easy to use Numpy-Arrays in C++ (https://pybind11.readthedocs.io/en/stable/advanced/pycpp/numpy.html).