search

uber-research / DeepPruner

DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch (ICCV 2019)
Other
354 stars 41 forks source link

Reconstructed point cloud accuracy #17

Closed paulacarbo07 closed 3 years ago

paulacarbo07 commented 4 years ago

Hello and thanks for the source code!

I have been using DeepPruner with the KITTI dataset to reconstruct point clouds using the disparity images from DeepPruner together with the KITTI camera parameters. The figure below illustrates the consecutive accumulation of reconstructed stereo point clouds with DeepPruner over the KITTI ground truth trajectory.

stereo_misalignment_2_edit

This point cloud has been cropped to remove the ground plane and data above a certain limit. The objects in these point clouds belong to the following real life objects:

misalignment_image

As a reference, these are the accumulated point clouds for LiDAR data.

stereo_misalignment_2_lidar_edit

My question is regarding the accuracy of the point clouds, since there seems to be around 1m total variance in between the estimated depth of an object, from the first until the last time it is seen, and around a 1m error in spacial accuracy, given the mean position for stereo depths with respect to the LiDAR depths. Is this bias and variance reasonable for the DeepPruner performance, or might I have introduced some other source of error in the process? Thanks in advance!

ShivamDuggal4 commented 4 years ago

Hi @paulacarbo07 thanks for the interest in our work. I don't know if you would have mistakenly introduced some other source of error in the process of creating this. But it has been observed in other papers also that stereo estimation methods optimized with disparity loss, are not perfect for predicting depth (reconstructing point cloud). Especially at larger depths, slight change in the predicted disparity value (in pixels) could mean larger change in depth (in meters).

Optimizing for depth would potentially reconstruct better point cloud.

Also you can check our supplementary results of reconstructing the point cloud using DeepPruner's disparity and uncertainty estimates. As shown in Fig. 4, pruning the uncertain pixels in the predicted disparity map before reconstructing the point cloud, drastically reduce the smearing effect and the depth variance that happens frequently at the object boundaries.

paulacarbo07 commented 4 years ago

Thanks for your answer, @ShivamDuggal4 . I have been cropping the point clouds as to not get depth estimations from too far away from the car, so currently I was considering points whose distance from the car is less than 15m. I have however not tried removing points according to their estimated uncertainty. Is there any code in the repo to get this information directly for two images? Thanks!

ShivamDuggal4 commented 3 years ago

Sorry, I don't have a script to visualize this. Please refer to supplementary for some example visualizations on this. Closing the issue due to inactivity. Feel free to reopen or create a new one.

a-akram-98 commented 3 years ago

@ShivamDuggal4 @paulacarbo07 will it be good to use DeepPruner to optimize depth?