Question about scene flow generation

princeton-vl / RAFT-3D

BSD 3-Clause "New" or "Revised" License

230 stars 26 forks source link

Question about scene flow generation #1

Closed ootts closed 3 years ago

ootts commented 3 years ago

I have a question about here. When we are generating scene flow using optical flow and depth maps, shouldn't we interpolate on depth1 using x1 and y1, and then backproject using interpolated depths? Thank you!

zachteed commented 3 years ago

Hi, this function is using the disparity_change to compute 3D flow. In sceneflow.py

depth1 = torch.from_numpy(intrinsics[0] / disp1).float()
depth2 = torch.from_numpy(intrinsics[0] / disp2).float()

# transformed depth
depth12 = torch.from_numpy(intrinsics[0] / (disp1 + disparity_change)).float()

flow3d = pops.backproject_flow3d(flow2d, depth1, depth12, intrinsics)

depth12 is already represented in the coordinates of the first frame, so no interpolation is necessary.

Bayrambai commented 3 years ago

Hi, how do you fine disparity_change in the above code snippet? Thanks