flow results too small?

[billamiable]

I tested pd-flow on my RGBD dataset collected from xtion, but in the output txt file, i found the optical flow values are extremely small --- none of them are larger than 0.5 or less than -0.5. I wonder if i need to amplify by a factor? By the way, i got five columns in the output txt file. I am not sure about the exact meaning of the later three columns. Anyone can offer help? Thanks!

[MarianoJT88]

Hi, if you look at the method PD_flow_opencv::saveResults(...) you will see that it saves the pixels coordinates first and then the 3D components of the motion. I tend to work in meters so those are probably expressed in meters (having values between -0.5m and 0.5m is very reasonable)

[billamiable]

thanks for your quick reply! I have a weird result using depth and image pairs. Do you know why? Thanks!

[MarianoJT88]

It also looks weird to me but I cannot say much more than that. Make sure that you provide depth images with the right format (specified in the README file).

[billamiable]

I observed that the format is different, i'm using ppm, pgm image format for rgb, depth respectively. After transforming into png, the results remain the same. So i'm confused... Meanwhile, could you provide demo image pairs for reference？

[MarianoJT88]

Try with this: https://www.dropbox.com/s/t7w7n4ewg9yziti/Person%20moving%202.rar?dl=0

Don't expect it to be very precise for this because the motion is large (I used these images for a more recent paper)

[billamiable]

It works better for the provided image pairs, however i still observed similar patterns (like purple dots) over the image. Do you have this kind of effect? And are the results of dx, dy, dz defined in world coordinate or camera coordinate? Thanks!

[MarianoJT88]

That effect is normal on distant regions because depth flickers there (RGB-D cameras are not very precise to measure distances beyond 2 meters or so)

[billamiable]

Got it, i still have one more question about how to obtain optical flow result from dx, dy, dz. Suppose pixel position is u,v in image coordinate, using depth and camera intrinsic parameter i can obtain corresponding 3d position X,Y, Z in camera coordinate. Can i simply add dx, dy, dz to X,Y, Z to obtain X', Y', Z' and project back to obtain final pixel position u', v'? Or i need to use the following equation to obtain du, dv, and then add them to obtain u', v'? Thanks a lot!

[MarianoJT88]

Hi, yes that´s the precise way to do it. However you might want to dig into the code to see where the optical flow is stored because that's the one which is actually estimated (and later transformed into 3D motion). As you prefer!

[billamiable]

Hi, I've looked at your code, and found out the way to implement. However, I wonder if the motion field M is ordered as (dy_dev, dz_dev, dx_dev)' according to the formula above and code below (which is a little weird). If correct, then the saved results' order, which are (dx, dy, dz), is different from the order of three components of M. Am i right? Thanks!

` device void CSF_cuda::computeMotionField(unsigned int index) { const float inv_f = 2.ftanf(0.5ffovh)/float(cols);

//Fill the matrices dx,dy,dz with the scene flow estimate
if (depth_old_dev[level_image][index] > 0)
{
    dx_dev[index] = dw_l_dev[index];
    dy_dev[index] = depth_old_dev[level_image][index]*du_l_dev[index]*inv_f + dw_l_dev[index]*xx_old_dev[level_image][index]/depth_old_dev[level_image][index];
    dz_dev[index] = depth_old_dev[level_image][index]*dv_l_dev[index]*inv_f + dw_l_dev[index]*yy_old_dev[level_image][index]/depth_old_dev[level_image][index];
}
else
{
    dx_dev[index] = 0.f;
    dy_dev[index] = 0.f;
    dz_dev[index] = 0.f;
}

}`

[MarianoJT88]

Hi, dx stores motion in the depth direction, dy stores motion in the direction lateral to the camera (horizontal) and dz stores the vertical motion. It might be strange if compared to reference systems commonly used in computer vision but felt natural to me from the point of view of physics (x to the front, y to the side and z looking upwards). I hope this helps!

[billamiable]

Thanks a lot!