search

fangchangma / self-supervised-depth-completion

ICRA 2019 "Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera"
MIT License
623 stars 135 forks source link

training with vlp-16 dataset #19

Open tzuchiehTT opened 5 years ago

tzuchiehTT commented 5 years ago

Hi Fangchang,

Our lab is currently working on a project which requires generating depth maps from our vlp-16 lidar and camera setting. Your work looks great as the depth map solution. Since we got different size images as input, I think what we need to do to use this network is (1) read in our own calibration information (K) and (2) crop input images as (width, high) both multiples of 16 (since we got errors when going through decode layers with some other sizes), is that right?

We've tested with a rather small dataset (only ~700 frames) and got results like the figure showing below. We are wondering if the dataset is too small or the depth info from vlp-16 is too sparse since the results remain clear projected lines. It would be great if you have any suggestions, thanks!

comparison_best

fangchangma commented 5 years ago

For cameras with different intrinsic parameters than the KITTI cameras, there is a slightly more complicated image resizing process than simple cropping. In particular, we need to resize the image in a way such that the resized image has the same effective intrinsics as KITTI.

XiaotaoGuo commented 5 years ago

Hi Fangchang, I have the same question. Why can't we just use our own intrinsic parameters instead of resizing images? Is it related to some hyperparameters needed to tune?

fangchangma commented 5 years ago

@XiaotaoGuo It might or might not work well with a different set of intrinsics, but there is simply no guarantee that the trained network would transfer directly to this new set of intrinsics (and image sizes). My suggestion is to keep the test images as close to the train images as possible.

XiaotaoGuo commented 5 years ago

Thanks! What if we use our own dataset to train the network and test with it?

fangchangma commented 5 years ago

What if we use our own dataset to train the network and test with it?

Then there is no need for any resizing

christian-lanius commented 5 years ago

Hi @MelvinTT, Did you manage to work this out? I am fighting similar problems with my own dataset, generated with a VLP32 and a Zed camera.

longyangqi commented 4 years ago

What if we use our own dataset to train the network and test with it?

Then there is no need for any resizing

Hi， When we use 'resize' operation, how to deal with the sparse depth? Could it be correct to just resize the sparse depth input like the rgb images, although it is sparse? Thanks!