question about depth_dense_twise datasets?

[xiaqing10]

Nice work! And I just get good results in kitti .

2022-06-30 02:18:54,741   INFO  recall_roi_0.3: 0.979423
2022-06-30 02:18:54,742   INFO  recall_rcnn_0.3: 0.979423
2022-06-30 02:18:54,742   INFO  recall_roi_0.5: 0.966354
2022-06-30 02:18:54,742   INFO  recall_rcnn_0.5: 0.967883
2022-06-30 02:18:54,742   INFO  recall_roi_0.7: 0.771846
2022-06-30 02:18:54,742   INFO  recall_rcnn_0.7: 0.807577
2022-06-30 02:18:54,745   INFO  Average predicted number of objects(3769 samples): 5.277
2022-06-30 02:19:25,714   INFO  Car AP@0.70, 0.70, 0.70:
bbox AP:96.2890, 89.2454, 88.6888
bev  AP:90.2766, 87.8979, 86.6158
3d   AP:88.9786, 78.3266, 77.2474
aos  AP:96.25, 89.10, 88.44
Car AP_R40@0.70, 0.70, 0.70:
bbox AP:98.6695, 92.3087, 91.7850
bev  AP:95.5831, 89.0949, 88.4404
3d   AP:91.7081, 81.6186, 77.5079
aos  AP:98.63, 92.14, 91.50
Car AP@0.70, 0.50, 0.50:
bbox AP:96.2890, 89.2454, 88.6888
bev  AP:96.4604, 89.5325, 89.1255
3d   AP:96.3678, 89.4688, 89.0301
aos  AP:96.25, 89.10, 88.44
Car AP_R40@0.70, 0.50, 0.50:
bbox AP:98.6695, 92.3087, 91.7850
bev  AP:98.7339, 94.5900, 94.2001
3d   AP:98.7011, 94.4395, 92.2569
aos  AP:98.63, 92.14, 91.50

But I have a question about the depth_dense_twise datasets, these are depth images and add rgb imgs concat to pc_velo_rgb_seg_uv. where can I get depth images , are these second 12G datasets from the web Download right color images, if you want to use stereo information?

Thanks.

[LittlePey]

Hi, the depth images are generated by depth completion network whose inputs are left color images and raw point clouds. If you want to generate by yourself, you can refer to TWISE or PENet.

[xiaqing10]

I Got it. Thanks

[yujmo]

Can you provide more details about depth completion? I use the TWISE, and get the following image

The original image corresponding to this image is as follows

So I don't know what to do. Can you help me?

[LittlePey]

Hi, is the second image the dense depth map we provided? How do you get the first image? In our experiments, we first project the raw point clouds to image space, getting a sparse dense depth map, then we feed the sparse depth map and corresponding RGB image to TWISE, getting a dense depth map (the second image).

[yujmo]

My code for projecting point clouds into depth images:

I use get_ lidar_ in_ image_ FOV function to project the point cloud in the camera's perspective into a depth image. The shape of the depth image is 352 * 1216.

First, I project the point cloud into a sparse depth image and then use the depth completion method to complete the sparse depth image. However, the depth value of the depth image after I complete is different from the depth image you provide, and the difference is quite large.

For example, the maximum depth value of the sample 000085.png in the training folder you provided is 21299. But the maximum depth value after I complete is 19962.

I have verified whether there is any problem in the coordinate calculation of my point cloud to depth image projection. I binarize the depth values of both the dense depth map you provided and the dense depth map I completed myself, 0 or 255. Then I found that the depth map I completed is exactly the same as the depth map you provided.

So I don't know where there is a problem with my data preprocessing.I'm looking forward to your answer.

[yujmo]

The following figure is provided in your dataset.

The following figure is the sparse depth image I converted from point cloud through matrix calculation.

The following figure is the dense depth image I generated from the sparse depth image through TWISE.

[LittlePey]

Hi, we will release the codes about how to generate pseudo point clouds in Here this month. Be waiting~

[yujmo]

好的，谢了大兄弟。

[LittlePey]

Hi, the codes that generate pseudo point clouds have been released, please refer to SFD-TWISE.

[yujmo]

Thank you for sharing the code. I compared my code with yours and found the problem with my original code.