NVIDIA / flownet2-pytorch

Pytorch implementation of FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Other
3.12k stars 739 forks source link

Wrong calculation of EPE #284

Open sueskind opened 1 year ago

sueskind commented 1 year ago

Usually the (average) end-point error (EPE) is defined as such:

$$EPE = \frac{1}{n} \sum_{i,j}\lVert v(i,j) - g(i,j) \rVert$$

where

Notice that the flows are 2-dimensional as they have an entry $v_x,g_x$ for $x$ and $v_y,g_y$ for $y$ disposition. Therefore, the norm here is the square root of the sum of two squares:

$$EPE = \frac{1}{n} \sum_{i,j} \sqrt{ \left( v_x(i,j) - g_x(i,j) \right)^2 + \left( v_y(i,j) - g_y(i,j) \right)^2 }$$

In losses.py however, the EPE is calculated wrongly as the MSE:

$$MSE = \frac{1}{n} \sum_{i,j,k} \left( v_k(i,j) -gk(i,j) \right)^2 = \frac{1}{n} \sum{i,j} \left( v_x(i,j) -g_x(i,j) \right)^2 + \left( v_y(i,j) -g_y(i,j) \right)^2$$

Although they look very similar, the actual EPE involves a square root which is missing in the MSE.

sueskind commented 1 year ago

Well I got a bit confused there... Depending on the shapes of the tensors it looks like the error is implemented correctly after all.

However, the L2Loss is usually implemented as the MSE, which is not the case in this code. At least the EPE and L2 having the same implementation is a contradiction.