search

TRI-ML / packnet-sfm

TRI-ML Monocular Depth Estimation Repository
https://tri-ml.github.io/packnet-sfm/
MIT License
1.23k stars 242 forks source link

About input depth map and gt depth map #201

Open mzy97 opened 2 years ago

mzy97 commented 2 years ago

First, thank you for sharing this great project. I have two questions. In dgp_dataset.py, it seems input depth map and gt depth map is the same, in packnet-SAN, are you using some downsample method when input sparse depth to the model? image

And after browse the code, I want to check the input depth map and gt depth map are made by projecting single LiDAR scan to camera plane, are you using some densify skills such as accumulate one sequence point cloud scan to make depth map denser or other techniques?

mzy97 commented 2 years ago

plus, when view DDAD depth map using dgp, I found the depth map is fairly sparse, it it normal or I forgot to do something? image

VitorGuizilini-TRI commented 2 years ago

Thank you for your interest in our work! About the depth maps, for DGP they are indeed the same, but we sparsify the input depth map during the augmentation stage. About the sparsity of the DDAD depth map, seems like you are resizing it "naively", with nearest-neighbor interpolation. This will eliminate a lot of the valid depth pixels, you should use this one: https://github.com/TRI-ML/packnet-sfm/blob/f59b1d615777a9987285a10e45b5d87b0369fa7d/packnet_sfm/datasets/augmentations.py#L56

Good luck!

haoweiz23 commented 2 years ago

Thank you for your interest in our work! About the depth maps, for DGP they are indeed the same, but we sparsify the input depth map during the augmentation stage. About the sparsity of the DDAD depth map, seems like you are resizing it "naively", with nearest-neighbor interpolation. This will eliminate a lot of the valid depth pixels, you should use this one:

https://github.com/TRI-ML/packnet-sfm/blob/f59b1d615777a9987285a10e45b5d87b0369fa7d/packnet_sfm/datasets/augmentations.py#L56

Good luck!

Hi, could you please provide more details about how do you sparsify the input depth map and how can we get the dense depth map which will be used to compute prediction metrics like abs.rel?

jbndr commented 1 year ago

@mzy97 @LionRoarRoar have you had success training the network with the down-sampled depth inputs? If so, did you scale up the predictions to GT resolution for the loss calculation?

DongyangHuLi commented 1 year ago

@mzy97 @LionRoarRoar have you had success training the network with the down-sampled depth inputs? If so, did you scale up the predictions to GT resolution for the loss calculation?

Hi, @jbndr . I have the same problem. Have you solved it? One more thing, is the downsampled depth aligned with RGB?